e-discovery 2.0

Clearwell just announced major enhancements to our Identification and Collection Module that together usher in a new generation of targeted collection capabilities for e-discovery. Why are we excited about this? Because it promises to provide our customers with a dramatic increase in their ability to perform quick and efficient collections across the enterprise with a small fraction of the cost and effort traditionally required.

Before Clearwell, vendors could only rely on building their own indexes when attempting to collect content by keyword from unstructured document sources. They did this in one of two ways.

The first method was to build one-off indexes with each collection, indexing content and then discarding the index after collection is complete. This minimized the amount of infrastructure required to maintain the index, but was painfully slow and wasteful of computing and network resources. These sorts of solutions came from vendors who originally focused on the forensic investigation side of the world, whose tools had been designed around small-scale collection from individual devices and hard drives. Unfortunately, they simply don’t scale to meet the demands of today’s large enterprises with their ever-increasing data volumes.

The second method was to attempt to create an uber-index of all of the information in an enterprise and keep it continually updated so that it would be ready at a moment’s notice for your collection needs. This approach proved to be incredibly challenging to implement, required a huge amount of infrastructure to maintain, and, worst of all, didn’t really work: creating the uber-index, as it turns out, was uber-difficult.

In talking with hundreds of customers over the last couple of years, we realized that there was a better “third way,” which combined the lightweight nature of the first method with the comprehensiveness of the second. How? By leveraging the indexes that enterprises already have in place. From comprehensive, robust archiving solutions like Symantec Enterprise Vault to the fully-searchable indexes found on Microsoft SharePoint, Exchange, and file servers, the way of finding the information you need quickly for e-discovery is, by and large, already out there. It’s simply a matter of building an e-discovery platform sophisticated enough to leverage those indexes and, when necessary, be intelligent enough to build its own when not available from another source. That’s exactly what we’ve done with Clearwell’s targeted keyword collection feature.

One of the most exciting things about this approach is that, while it works great for today’s enterprise information infrastructure, it is perhaps even more powerful in tomorrow’s. As your company’s information stores gradually shift toward the cloud, leveraging the indexes in the cloud becomes essential to being able to access the information that lives there in a fast and efficient manner. It’s simply not feasible to be able to use the “one-off” or “uber-index” approaches when data is living in a cloud infrastructure, since data access rates are often slower because they are occurring over a wider-area network.  Last year, Clearwell was the first e-discovery platform to support direct access of cloud Exchange and SharePoint environments, and now with keyword collection we have made another great stride forward in achieving our customer’s vision for next generation e-discovery. And there’s still more to come as we accelerate our product development by integrating with Symantec’s world-class information management team. Stay tuned!

I am thrilled to announce that Clearwell has signed an agreement to be acquired by Symantec for $410 million ($390 million, net of our cash balance of $20 million). By bringing together Clearwell’s market leading e-discovery platform with Symantec’s market-leading archiving solution, we are uniquely positioned to provide customers with the next generation of information management solutions.

The e-discovery software industry has matured rapidly in the 6 years since Clearwell was founded. As electronic information has become a key part of all litigation, regulatory inquiries, and internal investigations, companies have had no choice but to adopt e-discovery software to keep their costs down. Some have done so by bringing e-discovery in-house; others prefer to work with law firms and litigation support companies who provide cloud-based solutions. Either way, e-discovery software has become widely adopted by corporations, government agencies, and law firms around the world.

Clearwell has been a major beneficiary of these trends. Our annual sales have grown rapidly to over $50 million, and the company has been profitable since 2009. Today, we have over 400 customers and 75 partners in 14 different countries.

Many of these customers are using Clearwell together with Symantec Enterprise Vault in a single integrated workflow, and they have often requested that we couple our products more tightly to better serve their information management needs. That’s what led us to partner with Symantec for the past several years and ultimately led to this transaction. Over time, we see corporations and government agencies increasingly seeking information management solutions that encompass both e-discovery and archiving, making the combination of Clearwell with Enterprise Vault incredibly compelling.

In the near term, we expect very little to change for our existing customers. The product will continue to be sold on a standalone basis and supported by the Clearwell team. We remain committed to serving law firms and litigation support partners, who are absolutely critical to our success in more ways than we can describe.

This is an exciting time for the e-discovery industry. Last week, Gartner published its first ever Magic Quadrant For eDiscovery Software. Today, Symantec and Clearwell join forces to deliver a seamless, integrated archiving and e-discovery management workflow, benefitting all our customers. You can find more information about the acquisition at: http://www.symantec.com/clearwell.  There are exciting times ahead.

Earlier today, Gartner published its eDiscovery MarketScope for 2009. Written by Debra Logan, John Bace, and Whit Andrews, it is perhaps the most comprehensive “buyers guide” available for companies interested in using electronic discovery technology to lower costs.

The eDiscovery MarketScope analyzes about 20 software companies focused on electronic data discovery. Based on extensive interviews with end customers and data from the companies themselves, Gartner rates the companies using criteria similar to those used in its famous Magic Quadrant reports. It also identifies market trends, and makes predictions for 2009 and beyond.

This report is required reading for anyone considering an investment in eDiscovery software, and I strongly recommend that you get a copy, either from Gartner or some other authorized source. To give you a flavor for Gartner’s analysis, a few of its main conclusions are as follows:

1. Bringing eDiscovery In-House Dramatically Reduces Cost

This is a claim that electronic discovery software vendors often make, and prospective customers rightly question. Gartner investigates and finds that many of its corporate clients are saving large amounts of money by using eDiscovery software to reduce the amount they spend on lawyers and legal service providers. It reports that customers typically recover their money from buying eDiscovery software within 3-6 months of implementation.

2. There’s No Single, End-To-End Solution For eDiscovery

Gartner addresses what is probably the most common question I get asked by corporate counsels and litigation support managers – namely, “Isn’t there a single product I can buy that will do end-to-end eDiscovery, covering all aspects of the EDRM?” The answer, of course, is “no” and Gartner goes further by predicting that the answer will remain “no” until at least 2011. So, for the foreseeable future, customers will need to buy best-of-breed products from different vendors for different stages of the EDRM model, and ensure they integrate smoothly.

3. There Are 4 Leading eDiscovery Software Companies

List of vendors achieving highest rating of “strong positive” (from Figure 2, page 10)

Of all the companies it analyzed, Gartner only gives 4 its highest rating of “strong positive”. Each of the four has different strengths. For processing, analysis and review, Clearwell is “fast-to-install and easy-to-use” (page 12) , while FTI’s ability to offer Attenex / RingTail either hosted or on-premise “positions it well for the future” (page 13) . Symantec’s leadership in email archiving makes Discovery Accelerator a good option for its customers who need to search and export data from Enterprise Vault. Finally, Zylab is well-known within law-enforcement circles and has a strong presence in Europe and Asia.

4. There Will Be Consolidation In The Next 12 Months

As the market matures, Gartner predicts that as many as 25% of eDiscovery software providers will either merge, be acquired, or exit the business. Access Data’s ambitious bid for Guidance has publicly put Guidance in play. Beyond that, Gartner suggests that Kazeon and several other players are all likely acquisition targets for larger companies wishing to enter the eDiscovery space.

Of course, Gartner is not the only influential voice in eDiscovery. Earlier this year, George Socha and Tom Gelbmann published their Socha-Gelbmann Survey, which also provides a valuable perspective on the market. How do the two reports compare? That will be the subject of my next post.

Yesterday, George Socha and Tom Gelbmann published summary results for their 2008 EDD survey. George and Tom gathered self-reported data from 85 electronic data discovery service providers and 40 e-discovery litigation software companies. To help vendors resist the temptation to “exaggerate” their accomplishments, they then cross-referenced the responses against independent surveys submitted by 29 law firms and 19 corporations, and applied a healthy dose of their own good judgment. The outcome, which they will publish in-full next month, is a great snapshot of the industry, and probably the most objective ranking of e-discovery vendors that you can find.

By comparing this year’s results to the 2007 survey, you get a sense for how much has changed in the e-discovery world over the past 12 months:

Top E-Discovery Software Companies

Note: arrows show change to rankings from last year’s Socha-Gelbmann Survey

Autonomy and Clearwell move up to the Top 5, overtaking Attenex and CT Summation which slip back to the second tier. There are also 3 new names ranked 6 through 10 (Epiq, iConect and Symantec) who displace Cataphora, Doculex, ISYS, and Oracle, none of whom even make it into the top 15. In other words, 70% of the rankings have changed since last year.

If a litigation support manager were to focus only on the Top 5 in making her ediscovery software decision, she would have a choice of some very different solutions. Autonomy positions itself as a high-end (expensive) platform for corporations, while Lexis offers a comprehensive toolset for law firms. Guidance and Clearwell are complementary in that both provide best-of-breed solutions for parts of the EDRM model: Guidance is the leader in collection and preservation, while Clearwell is the leader in processing, analysis and review. Finally, FTI takes a services-based approach which centers around RingTail, its hosted review application.

Looking lower down the list, there were some other interesting results, primarily around which companies were NOT ranked. Kazeon made it into the third tier (ranked 11-15) whereas StoredIQ, its main competitor, did not. Nor did Recommind break into the rankings, despite making a major push into e-discovery from knowledge management over the past year. But the most striking absentees are PSS Systems and Exterro, which have pioneered litigation hold management for Fortune 100 companies. I can only guess that they cover too much of niche market to warrant inclusion in an industry-wide report.

Top E-Discovery Service Providers

In contrast to the world of software, e-discovery services saw much less movement in this year’s rankings:

Note: arrows show change to rankings from last year’s Socha-Gelbmann Survey

There was only one change to the top 5: Fios moved up, displacing Guidance which plummeted 10-20 places down to a 16-25 ranking. In addition, there were two new players in the top 10, Epiq and Huron, who edged out Electronic Evidence Discovery and Ernst & Young.

Conclusion

Changes to the software rankings reflect broader changes in the litigation software market. As litigation discovery has moved in-house, corporations have become a major driver of purchase decisions that were previously left to law firms. Many software companies, such as Attenex, have struggled to make this transition, while others, such as Clearwell, have capitalized on it. There has been no such change in the service provider world and, as a result, the rankings are relatively stable.

It will be interesting to see what happens next year. Every other software space is dominated by a small number of players, like Oracle for databases or VMWare for virtualization. If the same is true for top ediscovery, then we can expect many fewer changes to the software rankings in future surveys as the leaders pull away from the pack.

Learn More On Litigation Support Software & Frcp Electronic Discovery.

Working my way through a backlog of articles and reports this week, I came across Forrester’s Archiving Wave Report for Q1 2008, which was published last month. I spoke to Barry Murphy, the Forrester analyst who wrote the report, around the time of its publication and it’s clear he views e-discovery as a major driver of archiving sales. It was, therefore, no surprise that Forrester’s evaluation criteria heavily weighted e-discovery. To quote the report: “We focused on value-add functionality like records and retention management and eDiscovery support…vendors need to offer real solutions for eDiscovery” (p.4).

The report’s most striking conclusion is that ZANTAZ is rated ahead of Symantec. Every previous report from Gartner and Forrester has identified Symantec as the market leader, so this is potentially a significant shift. Forrester’s primary criticism of Symantec is that it has “product gaps”, more specifically “a reliance on the end-of-lifed Alta Vista search engine (a major issue, given the importance of search to eDiscovery).” (p.9) By contrast, Forrester writes that ZANTAZ offers: “an EAS product with strong search, analytics,…and advanced eDiscovery capabilities – the value-add features customers are demanding.” (p.8)

It’s worth taking a moment to understand Forrester’s criticism of Symantec in more detail. Companies buy archives for 2 reasons: mailbox management and e-discovery. For mailbox management, search is becoming less important since most users will rely on Windows search in Vista or Google desktop. Rather than provide their own search functionality, archives will instead just integrate with Microsoft and/or Google.

But the reverse is true for e-discovery, where search has become increasingly important. The growing volume of litigation, regulatory inquiries, and corporate investigations requires companies to continually comb through their archives in a highly iterative process that often involves IT and legal. Forrester’s analysis shows that – for this purpose – Symantec’s functionality is insufficient.

There’s no doubt Enterprise Vault is a great product. It has long been a market leader, and generates a huge amount of revenue for Symantec. But archiving is an intensely competitive market. So, if Symantec does not improve its search/indexing capabilities for e-discovery, it leaves itself vulnerable to others who will. To me, that’s the most important point that Forrester is making in its report.

For those in regulated industries like financial services, where data retention policies are mandated, every keystroke is tracked and every phone call recorded, the question of how long you should keep data is moot: you keep it for as long as regulations demand.

But for the rest of us in manufacturing, media, technology, government, and elsewhere, it remains an open question. The answer to “what should our email and document retention policy be?” is often a political hot potato, pitting legal and IT’s goal of lower costs against the broader population’s desire to hang on to all their email, just in case they need it later. In fact, the only thing harder than agreeing a retention policy is enforcing it afterwards, as corporate users habitually keep more data than allowed, unless physically prevented from doing so.The reason this matters is that many people believe creating a data retention policy is a key part of implementing an e-discovery solution. I too used to think this way, viewing retention-policy-creation as a necessary rite of passage for legal, IT, and information security people who want to lower e-discovery costs. After all, if the #1 cause of higher e-discovery costs is too much data, then a policy reducing the amount of data looks like a low cost, no-brainer solution.

But life just does not work that way. Outside of the command-and-control environment of regulated industries, retention policies simply do not work. You cannot fight human nature and force people to delete information they want to keep – especially when Gmail, Yahoo Mail, Hotmail and others are training them to do precisely the opposite (i.e., never delete, keep everything) in their personal email accounts.

So, I have changed my mind: to anyone engaged in implementing an e-discovery solution in a non-regulated industry, I say: forget data retention policies, it is a red herring. Too much data is a fact of life that will only get worse. You can no more get people to delete email and documents than you can stop someone writing them in the first place. Instead, focus on the battle you can win by putting in an e-discovery solution that enables you to do two things:

1. Collect data efficiently, so that you have a reliable (defensible) way of getting the data you need. Implementing an email archive from HP, Symantec or others is a great way of approaching this, as is leveraging forensics tools from Guidance or Access Data.

2. Analyze the data up front, so that you can cull it down to only those documents relevant to the case before a human being has to review them. Clearwell’s e-discovery solution is one approach which has worked for a large number of enterprises.

If your experiences, or conclusions, differ from mine, then feel free to post a comment. I am particularly interested to hear about successful examples of data retention policies at non-regulated companies, since I have yet to see one.

Today, Symantec announced 3 connectors for Enterprise Vault, for analytics, review and content collection. According to the announcement, these will “provide tight integration with third-party case management, review, analytics, forensics and desktop collection tools.”

The idea that archives should integrate with third party products is one I whole-heartedly support and have written about before. My company, Clearwell, has been working with Nick, Scott, and the gang at Symantec on this for over a year. They tell us that we were the first to integrate with Enterprise Vault and, to our knowledge, we are the only ones who have deployed fully integrated e-discovery solutions with Enterprise Vault at several enterprises.

Having said all that (and climbing down from my soapbox), I think Symantec’s customers will need to read this announcement very carefully to understand what it means. To give them a helping hand, let me translate it from corporate-marketing-speak into plain English:

Symantec is releasing 3 connectors which enable customers to ingest files from EnCase and export files to Summation and Ringtail. It is also exposing a new application programming interface (API) so that third-party vendors can more easily build their own connectors to Enterprise Vault.

At this point, most people’s eyes glaze over and they ask “who cares”? Surely, only techies get excited about something as esoteric as a new API. But as the recent excitement over FaceBook’s API has shown, opening up a platform – even in a limited way, as Symantec is doing – can unlock tremendous value. For those customers with Discovery Accelerator v.7.5, the new API will have a huge impact for 2 reasons:

1. It makes integration with Enterprise Vault much easier, so lots more vendors will do it. In their press release, Symantec mentions a handful of companies who are building connectors to the new API and I’m sure more will follow. This increases customer choice, and makes it more likely that Symantec customers will be able to select related products that closely fit their needs;
2. It enables enterprises to have a smooth workflow across all aspects of e-discovery, from collection/preservation to analysis/review to production/presentation. For example, companies can now collect information in Enterprise Vault, preserve it by placing a litigation hold on key information via Discovery Accelerator, and then seamlessly hand off that information to a third party application (like Clearwell) for review and analysis. This saves a lot of time that would otherwise be wasted on importing/exporting data from different systems, and reduces the risk that something gets lost in the shuffle.

Net net: companies do well by giving customers what they want, and customers want end-to-end e-discovery solutions. Symantec is not the only one to have figured this out; stay tuned for more announcements like this from other archiving vendors.

At some point in his blog last year, David Hornik, a venture capitalist, lamented the fact that VideoEgg, one of his investments, had 38 competitors in the online video market – and those were only the ones that he knew about.

A casual observer could be forgiven for seeing the same thing happening in e-discovery. Barely a day goes by without some company in a completely different market announcing that they too now have an “e-discovery solution”. Debra Logan at Gartner, who is fast emerging as one of the leading lights of the e-discovery world, tells me she is speaking to 30 vendors for her forthcoming research – and could easily have covered twice that number. Brian Babinau, the insightful and witty analyst at Enterprise Strategy Group, jokes that: “nowadays, people either build a social networking product or do e-discovery.”

For example, last week Zimbra, an open source email platform which has nothing to do with e-discovery, announced its new “e-discovery features”, which sound a lot like keyword search. Kazeon, which wins the prize for creating the world’s most complex e-discovery workflow diagram, has added e-discovery as one of its primary “solutions”, while Endeca takes a more measured approach, proposing only that its financial services customers use it for e-discovery. The list goes on and on.

Despite the worsening signal-to-noise ratio, all the activity will ultimately make it easier for customers to figure out which e-discovery solution makes sense for them. There’s more coverage from leading analysts, who can help explain the different products; large vendors such as EMC, Symantec, and HP are gradually educating the market; and the industry is coalescing around the Electronic Discovery Reference Model, which breaks e-discovery down into its key elements and explains how they fit together.

If e-discovery follows the path of online video and other fast-growing categories, lots of companies will continue to throw their hat into the ring. But for every hundred “VideoEggs”, there will only be one YouTube.

Most large companies face a dilemma. Should they open up their products and invite others to build features on them, creating a “platform” or ecosystem around themselves? Or would that be inviting the proverbial fox into the hen-house, meaning they should instead prevent others from integrating with their product or leveraging it to create add-on functionality?

In the internet world, there is no doubt about the answer: throw open the doors via easy-to-use APIs (“application programming interfaces”) and let a thousand flowers bloom. That’s what FaceBook did a couple of weeks back with their announcement of the FaceBook Platform, and it has already led to hundreds of new applications for their users. It is what Skype did so effectively, creating a mini-industry around themselves of voicemail, skins, ring-tones, and more. Even eBay, which has jealously guarded its feedback ratings and has habitually crushed smaller companies in its cross-hairs, is embracing the open platform mantra, announcing this week that third-party companies can build features that work with eBay in new ways.

By contrast, telecom companies live in a world of closed standards. Even in the wireless industry, which is arguably the most competitive part of the telecom world, the carriers (Cingular, T-Mobile, Verizon, etc.) exact a heavy toll on any application trying to reach their handsets. As friends in the industry tell me, “There’s a reason why there has never been a billion dollar mobile application company.”

In e-discovery, the large technology vendors like EMC, HP, Symantec, and ZANTAZ face the same choice. Their email archiving products store huge amounts of data. Should they let 3^rd party e-discovery software analyze that data, giving their customers more choice? Or should they slam the door shut, and try to force customers to use their own proprietary e-discovery applications?

The answer, it seems, depends on what they want to be when they grow up. As the market leader, Symantec is confident enough to open its archive (Enterprise Vault) to 3^rd party applications while offering customers its own Discovery Accelerator for litigations holds and some document review. Similarly, perhaps because of its powerful brand, HP focuses on storage optimization with HP RISS and partners with e-discovery software, often with huge savings for its customers. On the other side of the coin, smaller companies like ZANTAZ and Mimosa see themselves as e-discovery companies: they seek to leverage their storage products to get customers to also buy their e-discovery applications.

In the long-run, my feeling is that any archive of any stature will have to adopt open standards. Customers will demand it, and (unlike telecom companies) the archive vendors do not have the market power to resist. Over time, they will also come to appreciate (as HP and Symantec do now) that enabling 3^rd party applications to analyze the data they store is to their advantage, since it creates a powerful, additional incentive to store more information in the archive.

In a previous post, I wrote about the forces transforming e-discovery, a phenomenon that has received increasing attention from the press, most recently in this week’s Economist magazine. While everyone agrees that something big has changed, and (generally speaking) on the reasons why, people struggle to put their finger on exactly what e-discovery has become.

That’s why I think the concept of “E-Discovery 2.0” is so helpful. Analogous to Web 2.0, E-Discovery 2.0 is a set of new processes, technologies, and services that enable companies to manage huge volumes of data, lower costs, and meet tight deadlines.

New Processes

When e-discovery meant handing over a few boxes of paper, companies did not need much of a process. But in today’s world, where it involves terabytes of data, teams of reviewers, and precious little time, it is a very different story. To cope with the growing volume and complexity of e-discovery issues, companies have had no choice but to adopt new processes. These include:

• Collect and Preserve: Most companies have now established procedures so that, when the need arises, they can collect all data relevant to a case and ensure that it cannot be changed or deleted.
• Analyze Up Front: When presented with more work than can be done, a company’s only option is to work smarter, not harder. That means analyzing the collected data up front, to cull it down to only those emails and documents directly relevant to the case at hand.
• Collaborate Efficiently: E-Discovery has become a team sport. And whenever you have a team, you need a playbook, or a process, to ensure work is not repeated and that everyone is marching towards the same goal.

New Technologies

If technology created this problem, by making electronic communication so pervasive and voluminous, then it can also solve it. In recent years, several new technologies have arisen that enable companies to store and sift through their data to fulfill e-discovery obligations. The most significant of these trends include:

• From tape to disk: As the cost of disk storage has continued to decline, more and more companies are abandoning tapes and instead keeping their data online. Email archiving software optimizes for storage efficiency, allowing companies to keep hundreds of terabytes of data readily available for e-discovery.
• From search to analysis: Basic keyword search has evolved into sophisticated analysis technology that mines email meta-data for relevance, links messages together into discussion threads, and groups them by topics. These analysis applications allow users to sift through millions of messages in minutes, to rapidly identify, tag, and export relevant data.
• From closed systems to open standards: Until recently, technology providers made no effort to integrate their applications, leaving customers to fend for themselves. But that has started to change. Symantec Enterprise Vault and HP RISS now have open APIs, creating pressure on others to follow suit. George Socha’s Electronic Discovery Reference Model (EDRM), a standards body, has received widespread support, accelerating progress towards creation of an open e-discovery platform.

To anyone working in litigation support, legal, or information security, all this is quite unremarkable. Of course they use technology to address e-discovery. Obviously, there has to be a process. From the company’s perspective, e-discovery has become no different to HR or finance – it is a core competency, part of doing business.

And that, perhaps, is the most remarkable thing about E-Discovery 2.0 – in only a few short years, it has become so widespread and deeply entrenched within the enterprise, that people barely notice it.