e-discovery 2.0

Over the past 4 years, I have had hundreds of conversations with corporate counsel and “legal IT”, meaning technical folks charged with supporting the legal team. More and more of them are looking to lower their costs by bringing e-discovery in-house. But as they work through that process, there’s one question that consistently comes up, even today – namely, “When [insert name of software company] says they “do” e-discovery, what exactly does that mean?”

There has been progress towards answering this question, thanks mainly to the analyst community. George Socha and Tom Gelbmann’s EDRM framework has been immensely helpful in breaking down electronic discovery into its component steps. Other analysts, like Debra Logan at Gartner, were quick to embrace the framework, prompting every software provider to follow suit. As a result, there is today a common language that everyone uses to describe the e-discovery process.

The Electronic Discovery Reference Model (EDRM) breaks down the e-discovery process into a series of steps. Companies looking to buy e-discovery software to lower costs typically map different software products to each of these steps, to make sure that they cover the entire process.

But having a universally-agreed framework is only half the answer. To eliminate customer confusion, there also needs to be agreement on how different software products fit into the framework. This is especially important since there is no single, end-to-end solution for e-discovery which covers all aspects of EDRM. So customers are forced to think about how different software solutions fit together. And that is where things begin to fall apart.

Many software vendors feel it is advantageous to claim that they do everything, even though they do not. Customers are rightly suspicious of those claims, and so press vendors to provide more detailed information – hence the question, “when you say you do e-discovery, what exactly does that mean?”

In light of that, how can litigation support teams, corporate counsel, or legal IT people figure out which e-discovery solution best meets their needs? From observing this decision-making process hundreds of times, I have found 3 simple steps are incredibly helpful.

Step 1: Read the analyst reports

Two reports in particular make for required reading. One is Gartner’s MarketScope Report, which is available for free at certain sites; the other is the 451Group’s recent e-discovery report, which is summarized in a publicly available presentation. The helpful thing about the 451 Group’s report is that it tells you which software companies do which parts of the EDRM process. You do have to buy the report to get the full picture (it’s well worth it!), but the publicly available presentation will give you a flavor for their analyis, and I have drawn from that presentation in the figure below:

Analyst firms like the 451 Group map software vendors to the EDRM framework according to what they actually do, which is often different from what software vendors claim they do.

The 451 Group’s analysis highlights several important points. First, it shows that there is no single end-to-end solution. Even the products of giants like EMC (SourceOne), HP (IAP), and IBM (CommonStore) only solve one piece of the puzzle, information management. Second, it shows that customers have choices at each stage of the EDRM process. For example, to solve the problem of identification, collection, and preservation of electronic information, customers can choose from solutions as diverse as Guidance EnCase (forensic collection), Index Engines (back-up tapes) and Mimosa NearPoint (email archive). Third, it provides an independent assessment of what vendors do, as opposed to what they may claim. For example, Kazeon claims analysis and review capabilities, whereas the report shows its product does identification, collection, and preservation; Recommind claims its Axcelerate eDiscovery and MindServer products do processing, whereas the report finds that they do not.

Step 2: Evaluate the products prior to purchase

Just as anyone would test-drive a car prior to purchase, it’s critical to test-drive e-discovery software. Any vendor should be willing to provide their software free of charge for an evaluation on-premise. The most effective evaluations are when the customer uses the product themselves, either on a live case or test data. This is far preferable to just sending the data to the vendor who then loads it into their system, as in that scenario there are too many opportunities for the vendor to hide their product’s shortcomings.

Step 3: Check references carefully

The trick with references is to insist on relevant references. It’s not good enough for the vendor to dredge up some random person who says nice things; or even a credible knowledgeable person who is using the product in a completely different way. For example, if a company is happy with Autonomy’s IDOL for enterprise search, that does not tell you much about what Autonomy might be like for e-discovery. What really counts are references from other customers who are using the product for the same application that you are.

All this can sound like a lot of work, but I have seen people go through the process in as little as a month, and be much happier for it. A little work up front can save a lot of time (and heart-ache!) later on.

When we first started brainstorming about our “Five Questions” feature a few months back, Tom Gelbmann was on our short list of “must-have” interviewees. As most readers are probably aware, Tom Gelbmann, together with George Socha, founded the Electronic Discovery Reference Model (EDRM) project and, also with George, directs the annual Socha-Gelbmann Electronic Discovery Survey. Since 1993 he has been managing director of Gelbmann and Associates, a consultancy based in Saint Paul, Minnesota.

1. There’s been a lot of buzz (particularly within the vendor community!) about the changes to the Socha-Gelbmann Electronic Discovery Survey. You and George have written about this a bit before, but we’ll try to take a slightly different angle: Beyond concerns about people taking rankings out of context or as an excuse for not doing due diligence, were there specific trends and drivers among end-users of e-discovery products and services that motivated the change?

The most important reason for killing the rankings was that they were too often interpreted by end-users as being a “one size fits all” evaluation of top product and vendors, and we thought this type of decision-making was becoming increasingly dangerous as electronic discovery was increasingly in its visibility and importance. It was similar to a prospective car buyer focusing in on the cars that earned the Car & Driver Car of the Year award. They may not need a BMW 3-Series or a Porsche Boxster if they need something to haul lumber or transport the kids to hockey practice. We also saw the opportunity to deliver something that could be useful in identifying software and service providers that meet a set of requirements based on the EDRM model. The result of the overhaul of the Survey we are currently working on will be the capability to specify requirements, dial up/down weighting of criteria and see which providers fit those parameters. Our objective is to deliver something more useful than the rankings.

2. Your other main project is, of course, the Electronic Discovery Reference Model (EDRM). How did EDRM get started? Was it you and George scribbling on the back of a napkin in a coffee shop?

It wasn’t quite on the back of a napkin, but close – it was an Etch-a-Sketch®! Actually, the idea came out of the 2004 Socha-Gelbmann Electronic Discovery Survey. After the dust settled in the fall of 2004, we took a step back to look at what we learned. It wasn’t difficult to spot the #1 issue concerning consumers and providers: confusion and frustration over the lack of standards in the electronic discovery industry. Many people told us this was the Wild West and something needed to be done. That something was a standards initiative. The reference model approach came from looking at the tremendous value the Open Systems Interconnection Reference Model had on computer communications. Developed in the late 1970s, this model provided a common structure for development of products. We thought a similar approach would work for electronic discovery. We started talking to providers and consumers about this idea and initially thought we may get 15 or so organizations interested… 20 tops. We now list the EDRM alumni of 500+ individuals from 140+ organizations.

3. EDRM will be celebrating its 5th year at the upcoming May meeting in Saint Paul. What’s surprised you most about the first five years of the Project?

The biggest surprise by far is that there was a second year, to say nothing of a fifth year that we are now planning. We started out with the objective of keeping the project to a single year, thinking that having a hard deadline would motivate everyone to complete the project on time. We also knew that placing all content in the public domain would attract interest and participation in this initiative that would help the entire industry.

4. The global recession is having a tremendous impact on the electronic discovery community. Any thoughts on how the landscape will be reshaped coming out of the storm?

Potential influencers keep surfacing, making it difficult to anticipate what is around the corner. Certainly, we have seen and will continue to see consolidation of providers. Acquisitions will likely continue as will dissolutions. It is clear that everyone is tightening the belt. Some corporations and law firms who had begun to expand internal electronic discovery operations will sharply curtail or stop the expansion. Experienced, highly capable people will be out of a job and looking for new opportunities, which means short term turmoil for these folks, but hopefully long term success within well managed organizations. The volume of work doesn’t seem to be ebbing yet, so there is great opportunity for the survivors.

5. You and George Socha are the “Dynamic Duo” of electronic discovery and have been working together for a long time.  Anything you want to reveal about George that might not be generally known?  Hidden talents? Secret ambitions?

Interesting question. George is a man of many talents. One pleasant surprise to me was learning of his expertise in baking. I thought I was good at making good pie crusts, after a dozen years making Thanksgiving pies with my daughters. My crusts have gotten rave reviews from the family. But George is the real expert. His creations are a work of art. When it comes to secret ambitions, I think he would like to assemble a massively parallel computer complex in his home office. He has a good start with a half dozen monitors, several computers, servers, etc. and shows no signs of slowing down.

Earlier today, Gartner published its eDiscovery MarketScope for 2009. Written by Debra Logan, John Bace, and Whit Andrews, it is perhaps the most comprehensive “buyers guide” available for companies interested in using electronic discovery technology to lower costs.

The eDiscovery MarketScope analyzes about 20 software companies focused on electronic data discovery. Based on extensive interviews with end customers and data from the companies themselves, Gartner rates the companies using criteria similar to those used in its famous Magic Quadrant reports. It also identifies market trends, and makes predictions for 2009 and beyond.

This report is required reading for anyone considering an investment in eDiscovery software, and I strongly recommend that you get a copy, either from Gartner or some other authorized source. To give you a flavor for Gartner’s analysis, a few of its main conclusions are as follows:

1. Bringing eDiscovery In-House Dramatically Reduces Cost

This is a claim that electronic discovery software vendors often make, and prospective customers rightly question. Gartner investigates and finds that many of its corporate clients are saving large amounts of money by using eDiscovery software to reduce the amount they spend on lawyers and legal service providers. It reports that customers typically recover their money from buying eDiscovery software within 3-6 months of implementation.

2. There’s No Single, End-To-End Solution For eDiscovery

Gartner addresses what is probably the most common question I get asked by corporate counsels and litigation support managers – namely, “Isn’t there a single product I can buy that will do end-to-end eDiscovery, covering all aspects of the EDRM?” The answer, of course, is “no” and Gartner goes further by predicting that the answer will remain “no” until at least 2011. So, for the foreseeable future, customers will need to buy best-of-breed products from different vendors for different stages of the EDRM model, and ensure they integrate smoothly.

3. There Are 4 Leading eDiscovery Software Companies

List of vendors achieving highest rating of “strong positive” (from Figure 2, page 10)

Of all the companies it analyzed, Gartner only gives 4 its highest rating of “strong positive”. Each of the four has different strengths. For processing, analysis and review, Clearwell is “fast-to-install and easy-to-use” (page 12) , while FTI’s ability to offer Attenex / RingTail either hosted or on-premise “positions it well for the future” (page 13) . Symantec’s leadership in email archiving makes Discovery Accelerator a good option for its customers who need to search and export data from Enterprise Vault. Finally, Zylab is well-known within law-enforcement circles and has a strong presence in Europe and Asia.

4. There Will Be Consolidation In The Next 12 Months

As the market matures, Gartner predicts that as many as 25% of eDiscovery software providers will either merge, be acquired, or exit the business. Access Data’s ambitious bid for Guidance has publicly put Guidance in play. Beyond that, Gartner suggests that Kazeon and several other players are all likely acquisition targets for larger companies wishing to enter the eDiscovery space.

Of course, Gartner is not the only influential voice in eDiscovery. Earlier this year, George Socha and Tom Gelbmann published their Socha-Gelbmann Survey, which also provides a valuable perspective on the market. How do the two reports compare? That will be the subject of my next post.

Earlier this month, I attended the 4th annual EDRM kickoff meeting in Saint Paul. For those unfamiliar with it, the E-Discovery Reference Model (EDRM) project is one of the foundational organizations in electronic discovery, started in 2005 by George Socha and Tom Gelbmann. It has grown rapidly over the past 3 years and now boasts a roster of 80+ companies that represent the best and brightest in the e-discovery world, both vendors and (increasingly) practitioners.

What I love about EDRM, and what differentiates it so much from other (equally worthy) groups that focus on issues around electronic discovery, is its relentless focus on solving practical problems, usually in a very concrete and actionable way. There’s clearly a place for deep thinking about e-discovery case law and strategizing about best practices for e-discovery policies and practice, but Tom and George saw a growing need for a complementary group that focused on e-discovery nuts-and-bolts, with a particular emphasis on technology-based solutions. If nothing else, EDRM’s rapid growth has certainly proven the merit of their idea.

So, like the mighty Mississippi River that flows by a few hundred yards from the doors of the St. Paul Hotel, when you dive into EDRM, you know it’s going to take you somewhere. There is exciting follow-on work that is happening across a range of current projects, most notably Evergreen, XML, Metrics, and Code of Conduct. But what I found most exciting this year is a completely new project that will be tackling one of the most important and challenging aspects of e-discovery: search.

Launched by a grassroots group of conference participants, the Search Project seeks to tackle critical problems around e-discovery search from a practical, actionable perspective. Given the criticality of search in e-discovery, it is something that many participants felt was sorely needed.

For example, the project is discussing how to develop a common language for talking about search across various vendor implementations. It’s also tackling the problem of ensuring consistency across searches in different steps of the EDRM process. This whole area is ripe for innovation, and I am delighted to see the e-discovery community collaborating to make more progress than any one participant could alone.