e-discovery 2.0

At some point in his blog last year, David Hornik, a venture capitalist, lamented the fact that VideoEgg, one of his investments, had 38 competitors in the online video market – and those were only the ones that he knew about.

A casual observer could be forgiven for seeing the same thing happening in e-discovery. Barely a day goes by without some company in a completely different market announcing that they too now have an “e-discovery solution”. Debra Logan at Gartner, who is fast emerging as one of the leading lights of the e-discovery world, tells me she is speaking to 30 vendors for her forthcoming research – and could easily have covered twice that number. Brian Babinau, the insightful and witty analyst at Enterprise Strategy Group, jokes that: “nowadays, people either build a social networking product or do e-discovery.”

For example, last week Zimbra, an open source email platform which has nothing to do with e-discovery, announced its new “e-discovery features”, which sound a lot like keyword search. Kazeon, which wins the prize for creating the world’s most complex e-discovery workflow diagram, has added e-discovery as one of its primary “solutions”, while Endeca takes a more measured approach, proposing only that its financial services customers use it for e-discovery. The list goes on and on.

Despite the worsening signal-to-noise ratio, all the activity will ultimately make it easier for customers to figure out which e-discovery solution makes sense for them. There’s more coverage from leading analysts, who can help explain the different products; large vendors such as EMC, Symantec, and HP are gradually educating the market; and the industry is coalescing around the Electronic Discovery Reference Model, which breaks e-discovery down into its key elements and explains how they fit together.

If e-discovery follows the path of online video and other fast-growing categories, lots of companies will continue to throw their hat into the ring. But for every hundred “VideoEggs”, there will only be one YouTube.

Most large companies face a dilemma. Should they open up their products and invite others to build features on them, creating a “platform” or ecosystem around themselves? Or would that be inviting the proverbial fox into the hen-house, meaning they should instead prevent others from integrating with their product or leveraging it to create add-on functionality?

In the internet world, there is no doubt about the answer: throw open the doors via easy-to-use APIs (“application programming interfaces”) and let a thousand flowers bloom. That’s what FaceBook did a couple of weeks back with their announcement of the FaceBook Platform, and it has already led to hundreds of new applications for their users. It is what Skype did so effectively, creating a mini-industry around themselves of voicemail, skins, ring-tones, and more. Even eBay, which has jealously guarded its feedback ratings and has habitually crushed smaller companies in its cross-hairs, is embracing the open platform mantra, announcing this week that third-party companies can build features that work with eBay in new ways.

By contrast, telecom companies live in a world of closed standards. Even in the wireless industry, which is arguably the most competitive part of the telecom world, the carriers (Cingular, T-Mobile, Verizon, etc.) exact a heavy toll on any application trying to reach their handsets. As friends in the industry tell me, “There’s a reason why there has never been a billion dollar mobile application company.”

In e-discovery, the large technology vendors like EMC, HP, Symantec, and ZANTAZ face the same choice. Their email archiving products store huge amounts of data. Should they let 3^rd party e-discovery software analyze that data, giving their customers more choice? Or should they slam the door shut, and try to force customers to use their own proprietary e-discovery applications?

The answer, it seems, depends on what they want to be when they grow up. As the market leader, Symantec is confident enough to open its archive (Enterprise Vault) to 3^rd party applications while offering customers its own Discovery Accelerator for litigations holds and some document review. Similarly, perhaps because of its powerful brand, HP focuses on storage optimization with HP RISS and partners with e-discovery software, often with huge savings for its customers. On the other side of the coin, smaller companies like ZANTAZ and Mimosa see themselves as e-discovery companies: they seek to leverage their storage products to get customers to also buy their e-discovery applications.

In the long-run, my feeling is that any archive of any stature will have to adopt open standards. Customers will demand it, and (unlike telecom companies) the archive vendors do not have the market power to resist. Over time, they will also come to appreciate (as HP and Symantec do now) that enabling 3^rd party applications to analyze the data they store is to their advantage, since it creates a powerful, additional incentive to store more information in the archive.

In a previous post, I wrote about the forces transforming e-discovery, a phenomenon that has received increasing attention from the press, most recently in this week’s Economist magazine. While everyone agrees that something big has changed, and (generally speaking) on the reasons why, people struggle to put their finger on exactly what e-discovery has become.

That’s why I think the concept of “E-Discovery 2.0” is so helpful. Analogous to Web 2.0, E-Discovery 2.0 is a set of new processes, technologies, and services that enable companies to manage huge volumes of data, lower costs, and meet tight deadlines.

New Processes

When e-discovery meant handing over a few boxes of paper, companies did not need much of a process. But in today’s world, where it involves terabytes of data, teams of reviewers, and precious little time, it is a very different story. To cope with the growing volume and complexity of e-discovery issues, companies have had no choice but to adopt new processes. These include:

• Collect and Preserve: Most companies have now established procedures so that, when the need arises, they can collect all data relevant to a case and ensure that it cannot be changed or deleted.
• Analyze Up Front: When presented with more work than can be done, a company’s only option is to work smarter, not harder. That means analyzing the collected data up front, to cull it down to only those emails and documents directly relevant to the case at hand.
• Collaborate Efficiently: E-Discovery has become a team sport. And whenever you have a team, you need a playbook, or a process, to ensure work is not repeated and that everyone is marching towards the same goal.

New Technologies

If technology created this problem, by making electronic communication so pervasive and voluminous, then it can also solve it. In recent years, several new technologies have arisen that enable companies to store and sift through their data to fulfill e-discovery obligations. The most significant of these trends include:

• From tape to disk: As the cost of disk storage has continued to decline, more and more companies are abandoning tapes and instead keeping their data online. Email archiving software optimizes for storage efficiency, allowing companies to keep hundreds of terabytes of data readily available for e-discovery.
• From search to analysis: Basic keyword search has evolved into sophisticated analysis technology that mines email meta-data for relevance, links messages together into discussion threads, and groups them by topics. These analysis applications allow users to sift through millions of messages in minutes, to rapidly identify, tag, and export relevant data.
• From closed systems to open standards: Until recently, technology providers made no effort to integrate their applications, leaving customers to fend for themselves. But that has started to change. Symantec Enterprise Vault and HP RISS now have open APIs, creating pressure on others to follow suit. George Socha’s Electronic Discovery Reference Model (EDRM), a standards body, has received widespread support, accelerating progress towards creation of an open e-discovery platform.

To anyone working in litigation support, legal, or information security, all this is quite unremarkable. Of course they use technology to address e-discovery. Obviously, there has to be a process. From the company’s perspective, e-discovery has become no different to HR or finance – it is a core competency, part of doing business.

And that, perhaps, is the most remarkable thing about E-Discovery 2.0 – in only a few short years, it has become so widespread and deeply entrenched within the enterprise, that people barely notice it.