e-discovery 2.0

Earlier this month, Gartner published its third annual MarketScope For E-Discovery Product Vendors. Written by Debra Logan, Whit Andrews, and John Bace, the report is an excellent survey of this rapidly evolving market. It is also a useful buyer’s guide for anyone considering a purchase of electronic discovery software, since it analyzes and rates various e-discovery players. You can buy the report at Gartner’s site, or access a complimentary copy here.

The report covers 18 e-discovery software vendors. Missing from the report are e-discovery hosted/software-as-a-service (SaaS) providers and small e-discovery software vendors. Gartner believes the market is maturing and only larger companies are viable in the long run. So it increased the minimum annual revenue requirement for inclusion in the report to $15 million.

My guess is that next year Gartner will discontinue the MarketScope and move instead to a Magic Quadrant for e-discovery software. Doing so would be very helpful for the entire industry. Now that George Socha and Tom Gelbmann no longer publish their annual rankings, Gartner’s report is the only way for people to get a sense for how different products compare against each other. That alone makes it required reading for anyone considering an investment in e-discovery software.

Over the past 4 years, I have had hundreds of conversations with corporate counsel and “legal IT”, meaning technical folks charged with supporting the legal team. More and more of them are looking to lower their costs by bringing e-discovery in-house. But as they work through that process, there’s one question that consistently comes up, even today – namely, “When [insert name of software company] says they “do” e-discovery, what exactly does that mean?”

There has been progress towards answering this question, thanks mainly to the analyst community. George Socha and Tom Gelbmann’s EDRM framework has been immensely helpful in breaking down electronic discovery into its component steps. Other analysts, like Debra Logan at Gartner, were quick to embrace the framework, prompting every software provider to follow suit. As a result, there is today a common language that everyone uses to describe the e-discovery process.

The Electronic Discovery Reference Model (EDRM) breaks down the e-discovery process into a series of steps. Companies looking to buy e-discovery software to lower costs typically map different software products to each of these steps, to make sure that they cover the entire process.

But having a universally-agreed framework is only half the answer. To eliminate customer confusion, there also needs to be agreement on how different software products fit into the framework. This is especially important since there is no single, end-to-end solution for e-discovery which covers all aspects of EDRM. So customers are forced to think about how different software solutions fit together. And that is where things begin to fall apart.

Many software vendors feel it is advantageous to claim that they do everything, even though they do not. Customers are rightly suspicious of those claims, and so press vendors to provide more detailed information – hence the question, “when you say you do e-discovery, what exactly does that mean?”

In light of that, how can litigation support teams, corporate counsel, or legal IT people figure out which e-discovery solution best meets their needs? From observing this decision-making process hundreds of times, I have found 3 simple steps are incredibly helpful.

Step 1: Read the analyst reports

Two reports in particular make for required reading. One is Gartner’s MarketScope Report, which is available for free at certain sites; the other is the 451Group’s recent e-discovery report, which is summarized in a publicly available presentation. The helpful thing about the 451 Group’s report is that it tells you which software companies do which parts of the EDRM process. You do have to buy the report to get the full picture (it’s well worth it!), but the publicly available presentation will give you a flavor for their analyis, and I have drawn from that presentation in the figure below:

Analyst firms like the 451 Group map software vendors to the EDRM framework according to what they actually do, which is often different from what software vendors claim they do.

The 451 Group’s analysis highlights several important points. First, it shows that there is no single end-to-end solution. Even the products of giants like EMC (SourceOne), HP (IAP), and IBM (CommonStore) only solve one piece of the puzzle, information management. Second, it shows that customers have choices at each stage of the EDRM process. For example, to solve the problem of identification, collection, and preservation of electronic information, customers can choose from solutions as diverse as Guidance EnCase (forensic collection), Index Engines (back-up tapes) and Mimosa NearPoint (email archive). Third, it provides an independent assessment of what vendors do, as opposed to what they may claim. For example, Kazeon claims analysis and review capabilities, whereas the report shows its product does identification, collection, and preservation; Recommind claims its Axcelerate eDiscovery and MindServer products do processing, whereas the report finds that they do not.

Step 2: Evaluate the products prior to purchase

Just as anyone would test-drive a car prior to purchase, it’s critical to test-drive e-discovery software. Any vendor should be willing to provide their software free of charge for an evaluation on-premise. The most effective evaluations are when the customer uses the product themselves, either on a live case or test data. This is far preferable to just sending the data to the vendor who then loads it into their system, as in that scenario there are too many opportunities for the vendor to hide their product’s shortcomings.

Step 3: Check references carefully

The trick with references is to insist on relevant references. It’s not good enough for the vendor to dredge up some random person who says nice things; or even a credible knowledgeable person who is using the product in a completely different way. For example, if a company is happy with Autonomy’s IDOL for enterprise search, that does not tell you much about what Autonomy might be like for e-discovery. What really counts are references from other customers who are using the product for the same application that you are.

All this can sound like a lot of work, but I have seen people go through the process in as little as a month, and be much happier for it. A little work up front can save a lot of time (and heart-ache!) later on.

Earlier today, Gartner published its eDiscovery MarketScope for 2009. Written by Debra Logan, John Bace, and Whit Andrews, it is perhaps the most comprehensive “buyers guide” available for companies interested in using electronic discovery technology to lower costs.

The eDiscovery MarketScope analyzes about 20 software companies focused on electronic data discovery. Based on extensive interviews with end customers and data from the companies themselves, Gartner rates the companies using criteria similar to those used in its famous Magic Quadrant reports. It also identifies market trends, and makes predictions for 2009 and beyond.

This report is required reading for anyone considering an investment in eDiscovery software, and I strongly recommend that you get a copy, either from Gartner or some other authorized source. To give you a flavor for Gartner’s analysis, a few of its main conclusions are as follows:

1. Bringing eDiscovery In-House Dramatically Reduces Cost

This is a claim that electronic discovery software vendors often make, and prospective customers rightly question. Gartner investigates and finds that many of its corporate clients are saving large amounts of money by using eDiscovery software to reduce the amount they spend on lawyers and legal service providers. It reports that customers typically recover their money from buying eDiscovery software within 3-6 months of implementation.

2. There’s No Single, End-To-End Solution For eDiscovery

Gartner addresses what is probably the most common question I get asked by corporate counsels and litigation support managers – namely, “Isn’t there a single product I can buy that will do end-to-end eDiscovery, covering all aspects of the EDRM?” The answer, of course, is “no” and Gartner goes further by predicting that the answer will remain “no” until at least 2011. So, for the foreseeable future, customers will need to buy best-of-breed products from different vendors for different stages of the EDRM model, and ensure they integrate smoothly.

3. There Are 4 Leading eDiscovery Software Companies

List of vendors achieving highest rating of “strong positive” (from Figure 2, page 10)

Of all the companies it analyzed, Gartner only gives 4 its highest rating of “strong positive”. Each of the four has different strengths. For processing, analysis and review, Clearwell is “fast-to-install and easy-to-use” (page 12) , while FTI’s ability to offer Attenex / RingTail either hosted or on-premise “positions it well for the future” (page 13) . Symantec’s leadership in email archiving makes Discovery Accelerator a good option for its customers who need to search and export data from Enterprise Vault. Finally, Zylab is well-known within law-enforcement circles and has a strong presence in Europe and Asia.

4. There Will Be Consolidation In The Next 12 Months

As the market matures, Gartner predicts that as many as 25% of eDiscovery software providers will either merge, be acquired, or exit the business. Access Data’s ambitious bid for Guidance has publicly put Guidance in play. Beyond that, Gartner suggests that Kazeon and several other players are all likely acquisition targets for larger companies wishing to enter the eDiscovery space.

Of course, Gartner is not the only influential voice in eDiscovery. Earlier this year, George Socha and Tom Gelbmann published their Socha-Gelbmann Survey, which also provides a valuable perspective on the market. How do the two reports compare? That will be the subject of my next post.

Yesterday’s post about the top e-discovery software vendors prompted a couple of interesting comments. George Socha posted a response here, disagreeing with my conclusions; and someone else (“top8”, whoever that is) asked whether one should “always listen to the top 5-10 songs on the list…[or] use the top 5 software products, regardless of one’s situation.”

To clarify, I whole-heartedly agree with George that there is no such thing as a “best” e-discovery service provider – as George says, it really does depend on your situation and I can think of many cases where a smaller, less well-known firm is a better choice than a national brand.

But e-discovery software is different for 2 reasons. First, and most importantly, in software there are increasing returns to scale which do not exist for service providers. The more companies that use a particular software product, the better that product becomes. Speaking from personal experience, when you have a large number of demanding customers, they force you to make your product better – and give you the money to do it. That’s why most technology markets are incredibly concentrated: everything from databases (Oracle) to search engines (Google) have a single dominant player. We are still in the early days of the e-discovery software market, but ultimately I expect it will follow suit and consolidate around a very small number of players.

The second difference between e-discovery software and service providers is that enterprises cannot change their software vendors as easily as they can change their service providers. Once software is deployed behind the firewall, it is fiendishly difficult to get it out, requiring enterprises to pick a single product for all cases. By contrast, it is easy to change service providers, so enterprises can pick the most relevant expertise on a case-by-case basis.

To answer the question posed by “top8”, I am not suggesting that everyone should only read Harry Potter, watch American Idol, and (Heaven forbid!) listen to Britney Spears. Those are matters of personal taste where diversity is what makes for a rich, vibrant society. But there are very good reasons why so many corporations rely on Veritas for backup software, Oracle for databases, Symantec/McAfee for anti-virus, IBM for developer tools, and so on. In software, the best products only get better. That’s why, 5 years from now, the list of top e-discovery software vendors will be even shorter.

There are two independent analyst reports identifying the top e-discovery software vendors.

The first, published in June 2007, is the Socha-Gelbmann Annual Electronic Discovery Survey. The authors, George Socha and Tom Gelbmann, probably know more about e-discovery than anyone else you are likely to meet. As someone who has filled out their 178-page survey, I can tell you it is excruciating in its detail and incredibly rigorous. According to the report, George and Tom contacted nearly 1,000 individuals and collected detailed data from 115 organizations.

The second analyst report is Gartner’s MarketScope, which is published today (December 2007). Its author, Debra Logan, is fast emerging as one of the leading lights of e-discovery and has great instincts about the market. For her report, Debra tells me that surveyed 30 vendors and checked over 90 customer references.

The results from the two reports are as follows:

(1) Companies listed as “Top Electronic Discovery Software Providers Based on 7 Criteria” (Table 19 and 20), listed in alphabetical order. (2) Companies awarded ratings of “Positive” or “Strong Positive” (Figure 1), listed in alphabetical order.

Why are the lists so different? Primarily because of two main factors:

1. Gartner’s list mixes service providers and software companies whereas Socha breaks them out separately. The Socha report has an entirely separate list for service providers.
2. Socha’s report was completed 6 months earlier than Gartner’s. In that intervening period, several new players entered the e-discovery market. For example, Kazeon was ranked by Gartner earlier this year a “niche player” (lower left quadrant) in the enterprise search market, and has not been in e-discovery long enough to participate in the Socha study (or, if they did participate, they did not have enough e-discovery customers to gain a high ranking).

Conclusions

The first conclusion to draw from these lists is that any vendor not in them is probably not worth considering for e-discovery. If neither Socha nor Gartner ranked them highly, then the vendor either could not provide compelling customer references or has lost competitive bake-offs to someone who is on the list. Either way, they are best avoided.

The second thing that stands out is how different these lists are. Of the 21 vendors identified by Socha and Gartner, only 5 are ranked as top e-discovery software vendors by both of them. Those 5 are Attenex, Clearwell, FTI, Guidance, and LexisNexis. So, if you are an enterprise looking for an e-discovery solution, it is clear who you should call first.

Finally, it is worth noting that both these analyst reports are relatively new. This is the third annual survey for Socha, and the first MarketScope for Gartner. That speaks to the fact that e-discovery software is a new, fast-growing product area. More and more enterprises are adopting e-discovery software solutions, and asking analysts about them, because they offer such a compelling ROI.