e-discovery 2.0

HP announced today that it has signed an agreement with Clearwell to resell the Clearwell E-Discovery Platform. The two companies have been partners for some time and have many joint customers such as Constellation Energy, Del Monte, and Universal Music. But, under this new agreement, thousands of HP sales people will now be compensated for selling Clearwell, giving them a powerful incentive to introduce their customer base to Clearwell’s e-discovery solution.

To my knowledge, this is the first time that a major archiving vendor has agreed to resell a partner’s e-discovery solution, and it raises a couple of interesting questions: why did HP do this deal? And, what does it mean for HP customers?

Ask anyone who tracks the email archiving market, and they will tell you that e-discovery is a major driver of archive purchases. As Gartner’s Carolyn DiCenzo observes: “Legal discovery is being mentioned by almost every client evaluating an e-mail archiving solution.” That’s because whenever a company has litigation, regulatory inquiries or internal investigations, IT is required to provide relevant electronic information to legal or information security. Far better to have it in one repository than spread out on user desktops, email servers, and file shares. So, CIOs are partnering with General Counsels to deploy email archives, much as they did – in years gone by – with the VP of Sales to implement CRM systems.

The problem is that, when you look at archives as e-discovery solutions, they only solve 50% of the pain. In EDRM terms, archives are a very effective solution for collection and preservation, but awful for processing, analysis, and review. They provide a bulletproof way to capture and preserve every message, but do not make it easy to perform early case analyses and cull down data to the very small set of documents relevant to the case at hand.

That’s why enterprise customers find it so compelling to pair up an archive, such as HP’s Integrated Archive Platform, with an e-discovery solution, such as the Clearwell E-Discovery Platform. So to summarize, HP is doing this deal because it’s the best way to provide HP customers with an end-to-end solution for e-discovery. The two products integrate out-of-the-box, have been proven to work together at several large enterprises, and can be purchased from a single supplier (HP). That’s a much easier, lower risk decision for many enterprises than purchasing separate point solutions and cobbling them together.

Very few companies have as much mindshare with corporate CIOs as HP. It can only be good news for the e-discovery market as a whole to have one the largest technology companies in the world out there educating its customers on the value of lowering the costs and risks of e-discovery.

Most large companies face a dilemma. Should they open up their products and invite others to build features on them, creating a “platform” or ecosystem around themselves? Or would that be inviting the proverbial fox into the hen-house, meaning they should instead prevent others from integrating with their product or leveraging it to create add-on functionality?

In the internet world, there is no doubt about the answer: throw open the doors via easy-to-use APIs (“application programming interfaces”) and let a thousand flowers bloom. That’s what FaceBook did a couple of weeks back with their announcement of the FaceBook Platform, and it has already led to hundreds of new applications for their users. It is what Skype did so effectively, creating a mini-industry around themselves of voicemail, skins, ring-tones, and more. Even eBay, which has jealously guarded its feedback ratings and has habitually crushed smaller companies in its cross-hairs, is embracing the open platform mantra, announcing this week that third-party companies can build features that work with eBay in new ways.

By contrast, telecom companies live in a world of closed standards. Even in the wireless industry, which is arguably the most competitive part of the telecom world, the carriers (Cingular, T-Mobile, Verizon, etc.) exact a heavy toll on any application trying to reach their handsets. As friends in the industry tell me, “There’s a reason why there has never been a billion dollar mobile application company.”

In e-discovery, the large technology vendors like EMC, HP, Symantec, and ZANTAZ face the same choice. Their email archiving products store huge amounts of data. Should they let 3^rd party e-discovery software analyze that data, giving their customers more choice? Or should they slam the door shut, and try to force customers to use their own proprietary e-discovery applications?

The answer, it seems, depends on what they want to be when they grow up. As the market leader, Symantec is confident enough to open its archive (Enterprise Vault) to 3^rd party applications while offering customers its own Discovery Accelerator for litigations holds and some document review. Similarly, perhaps because of its powerful brand, HP focuses on storage optimization with HP RISS and partners with e-discovery software, often with huge savings for its customers. On the other side of the coin, smaller companies like ZANTAZ and Mimosa see themselves as e-discovery companies: they seek to leverage their storage products to get customers to also buy their e-discovery applications.

In the long-run, my feeling is that any archive of any stature will have to adopt open standards. Customers will demand it, and (unlike telecom companies) the archive vendors do not have the market power to resist. Over time, they will also come to appreciate (as HP and Symantec do now) that enabling 3^rd party applications to analyze the data they store is to their advantage, since it creates a powerful, additional incentive to store more information in the archive.

In a previous post, I wrote about the forces transforming e-discovery, a phenomenon that has received increasing attention from the press, most recently in this week’s Economist magazine. While everyone agrees that something big has changed, and (generally speaking) on the reasons why, people struggle to put their finger on exactly what e-discovery has become.

That’s why I think the concept of “E-Discovery 2.0” is so helpful. Analogous to Web 2.0, E-Discovery 2.0 is a set of new processes, technologies, and services that enable companies to manage huge volumes of data, lower costs, and meet tight deadlines.

New Processes

When e-discovery meant handing over a few boxes of paper, companies did not need much of a process. But in today’s world, where it involves terabytes of data, teams of reviewers, and precious little time, it is a very different story. To cope with the growing volume and complexity of e-discovery issues, companies have had no choice but to adopt new processes. These include:

• Collect and Preserve: Most companies have now established procedures so that, when the need arises, they can collect all data relevant to a case and ensure that it cannot be changed or deleted.
• Analyze Up Front: When presented with more work than can be done, a company’s only option is to work smarter, not harder. That means analyzing the collected data up front, to cull it down to only those emails and documents directly relevant to the case at hand.
• Collaborate Efficiently: E-Discovery has become a team sport. And whenever you have a team, you need a playbook, or a process, to ensure work is not repeated and that everyone is marching towards the same goal.

New Technologies

If technology created this problem, by making electronic communication so pervasive and voluminous, then it can also solve it. In recent years, several new technologies have arisen that enable companies to store and sift through their data to fulfill e-discovery obligations. The most significant of these trends include:

• From tape to disk: As the cost of disk storage has continued to decline, more and more companies are abandoning tapes and instead keeping their data online. Email archiving software optimizes for storage efficiency, allowing companies to keep hundreds of terabytes of data readily available for e-discovery.
• From search to analysis: Basic keyword search has evolved into sophisticated analysis technology that mines email meta-data for relevance, links messages together into discussion threads, and groups them by topics. These analysis applications allow users to sift through millions of messages in minutes, to rapidly identify, tag, and export relevant data.
• From closed systems to open standards: Until recently, technology providers made no effort to integrate their applications, leaving customers to fend for themselves. But that has started to change. Symantec Enterprise Vault and HP RISS now have open APIs, creating pressure on others to follow suit. George Socha’s Electronic Discovery Reference Model (EDRM), a standards body, has received widespread support, accelerating progress towards creation of an open e-discovery platform.

To anyone working in litigation support, legal, or information security, all this is quite unremarkable. Of course they use technology to address e-discovery. Obviously, there has to be a process. From the company’s perspective, e-discovery has become no different to HR or finance – it is a core competency, part of doing business.

And that, perhaps, is the most remarkable thing about E-Discovery 2.0 – in only a few short years, it has become so widespread and deeply entrenched within the enterprise, that people barely notice it.