e-discovery 2.0

Over the past 4 years, I have had hundreds of conversations with corporate counsel and “legal IT”, meaning technical folks charged with supporting the legal team. More and more of them are looking to lower their costs by bringing e-discovery in-house. But as they work through that process, there’s one question that consistently comes up, even today – namely, “When [insert name of software company] says they “do” e-discovery, what exactly does that mean?”

There has been progress towards answering this question, thanks mainly to the analyst community. George Socha and Tom Gelbmann’s EDRM framework has been immensely helpful in breaking down electronic discovery into its component steps. Other analysts, like Debra Logan at Gartner, were quick to embrace the framework, prompting every software provider to follow suit. As a result, there is today a common language that everyone uses to describe the e-discovery process.

The Electronic Discovery Reference Model (EDRM) breaks down the e-discovery process into a series of steps. Companies looking to buy e-discovery software to lower costs typically map different software products to each of these steps, to make sure that they cover the entire process.

But having a universally-agreed framework is only half the answer. To eliminate customer confusion, there also needs to be agreement on how different software products fit into the framework. This is especially important since there is no single, end-to-end solution for e-discovery which covers all aspects of EDRM. So customers are forced to think about how different software solutions fit together. And that is where things begin to fall apart.

Many software vendors feel it is advantageous to claim that they do everything, even though they do not. Customers are rightly suspicious of those claims, and so press vendors to provide more detailed information – hence the question, “when you say you do e-discovery, what exactly does that mean?”

In light of that, how can litigation support teams, corporate counsel, or legal IT people figure out which e-discovery solution best meets their needs? From observing this decision-making process hundreds of times, I have found 3 simple steps are incredibly helpful.

Step 1: Read the analyst reports

Two reports in particular make for required reading. One is Gartner’s MarketScope Report, which is available for free at certain sites; the other is the 451Group’s recent e-discovery report, which is summarized in a publicly available presentation. The helpful thing about the 451 Group’s report is that it tells you which software companies do which parts of the EDRM process. You do have to buy the report to get the full picture (it’s well worth it!), but the publicly available presentation will give you a flavor for their analyis, and I have drawn from that presentation in the figure below:

Analyst firms like the 451 Group map software vendors to the EDRM framework according to what they actually do, which is often different from what software vendors claim they do.

The 451 Group’s analysis highlights several important points. First, it shows that there is no single end-to-end solution. Even the products of giants like EMC (SourceOne), HP (IAP), and IBM (CommonStore) only solve one piece of the puzzle, information management. Second, it shows that customers have choices at each stage of the EDRM process. For example, to solve the problem of identification, collection, and preservation of electronic information, customers can choose from solutions as diverse as Guidance EnCase (forensic collection), Index Engines (back-up tapes) and Mimosa NearPoint (email archive). Third, it provides an independent assessment of what vendors do, as opposed to what they may claim. For example, Kazeon claims analysis and review capabilities, whereas the report shows its product does identification, collection, and preservation; Recommind claims its Axcelerate eDiscovery and MindServer products do processing, whereas the report finds that they do not.

Step 2: Evaluate the products prior to purchase

Just as anyone would test-drive a car prior to purchase, it’s critical to test-drive e-discovery software. Any vendor should be willing to provide their software free of charge for an evaluation on-premise. The most effective evaluations are when the customer uses the product themselves, either on a live case or test data. This is far preferable to just sending the data to the vendor who then loads it into their system, as in that scenario there are too many opportunities for the vendor to hide their product’s shortcomings.

Step 3: Check references carefully

The trick with references is to insist on relevant references. It’s not good enough for the vendor to dredge up some random person who says nice things; or even a credible knowledgeable person who is using the product in a completely different way. For example, if a company is happy with Autonomy’s IDOL for enterprise search, that does not tell you much about what Autonomy might be like for e-discovery. What really counts are references from other customers who are using the product for the same application that you are.

All this can sound like a lot of work, but I have seen people go through the process in as little as a month, and be much happier for it. A little work up front can save a lot of time (and heart-ache!) later on.

It’s been a little over a month since the news first broke that Guidance Software was the frog in an electronic discovery kettle whose water had just reached the boiling point, with the arbitrator in an employment case demanding, “I want this game-playing stopped.” We thought that, with a little time between the initial story and now, it would be worth taking a step back and looking at possible lessons learned — not so much for Guidance specifically, but for enterprises who find themselves in similar situations, as well as the electronic discovery community that serves them.

First, a quick summary. Based on published accounts, it seems like a classic discovery situation (that’s just plain old discovery, without the “e”): a party is sued and fails to produce a document that, lo and behold, surfaces via some other source, throwing the integrity of the sued party into question. After all, if one potentially incriminating document wasn’t discovered, then who knows what else could be out there?

Guidance contended that it did everything that was required of it, and that it didn’t have (or couldn’t find, despite good faith efforts) the emails in question. But, of course, that didn’t stop the litigation support community (via forums such as the Litigation Support List) from pouncing on the perceived hypocrisy.

After all, how could a leading, publically-traded electronic discovery company get caught up in such a mess? How could their cutting-edge electronic discovery technology not have saved them? Or their (hopefully) best-in-class internal electronic discovery processes? If the electronic discovery companies don’t have their acts together, what about all the other poor souls who lack their knowledge and expertise?

That last question is a scary one, particularly given today’s environment, and it’s why the situation has stirred up so much chatter out in the electronic discovery blogosphere. Almost without exception, commenters have jumped to one of two conclusions. Either (a) Guidance has not followed proper e-discovery best practices, or (b) Guidance has willfully chosen to hide relevant documents that it could have produced, because they would be detrimental to its case.

Let’s explore each of those conclusions in a little more detail.

First, is there any direct evidence that Guidance did not follow electronic discovery best practices? The answer there is murky. Certainly, from Guidance’s perspective, the answer is a resounding “no”. They continue to claim that the emails that were produced from another source did not exist on the various laptops, desktops, and servers that were part of the initial discovery request, and it is certainly possible that that is true. Perhaps Guidance had a 1-year retention policy for emails, and the emails in question were outside of that policy. Perhaps the individuals involved had legitimately deleted the emails in question prior to receiving a litigation hold notice, without thinking that they would ever be relevant to a legal matter. Certainly an independent observer has grounds for incredulity here, but it does not necessarily follow that Guidance did not follow electronic discovery best practices for a company of their size and resources. Certainly, from the reports, they did not exactly act in a way that earned much confidence from or favor with the arbitrator. However, that’s a completely different issue, and one which may be a legitimate tactical decision by Guidance (to avoid, for example, the high cost of recovering the corrupt backup tapes).

Second, what if Guidance willfully chose to hide relevant documents? At this point, there is no evidence that this is the case. And, you would think that of all of the companies out there, Guidance would be keenly aware of the extremely high level of risk associated with this strategy. A company well-versed in computer forensics understands keenly that the odds of any potentially negative emails not being out there, somewhere, in cyberspace are incredibly small. Thus there is little incentive to intentionally hide documents. If, however, a company did make such a perilous and unethical decision, it has nothing to do with a lack of e-discovery best practices or technology: it simply has to do with a lack of ethics.

So, has the coverage of the Guidance situation been nothing more than an electronic discovery witch hunt? Far from it… even if both of the “conventional wisdom” conclusions are in fact wrong.

Why? Because even if Guidance has its electronic discovery house in order and is acting with complete integrity, if there’s one thing that anyone in the electronic discovery business should have taken away from the last 5 years of court rulings, it’s that perception and transparency in electronic discovery is everything. Electronic discovery is technically complex and fraught with challenges, and companies – particularly those who are perceived as having vast expertise in the space, whether as vendors (i.e. Guidance) or institutions (i.e. pick your favorite TARP recipient) – have to act in such a way as to appear spotless before the court of law and the court of public opinion.

Assuming you already have your electronic discovery house in relative order (a baseline, fundamental requirement for doing business today), perhaps the most important take-away from Guidance is how carefully you need to consider how minor electronic discovery slip-ups, whether real or perceived, can bite, big time. The legal and media environment is primed to pounce on any hint of a cover-up or conspiracy, and enterprises must go the extra mile (or two, or three…) to ensure that their e-discovery efforts are, and will be perceived, as upright, ethical, and above reproach – or be ready and willing to pay the price in sanctions or loss of public confidence.

To the casual observer, it is surprising that a small private company (AccessData) could even think of acquiring a larger, public one (Guidance Software). But that’s exactly what AccessData publicly proposed to Guidance’s shareholders on November 6, after Guidance’s board had rejected its offer of $4.50 per share.

Leaving aside the personalities involved, and the history of bitter rivalry between these two companies, it’s easy to see why Guidance’s board rejected the offer. First, it’s only a 19% premium over Guidance’s share price on October 6, the date that the offer was made. Second, given 23 million shares outstanding, AccessData is offering a total price of just over $100 million for a company with $90 million in revenue and about $25 million in cash. Compare that to other e-discovery acquisitions, such as FTI’s $88 million purchase of Attenex or Iron Mountain’s $158 million deal for Stratify, each of which only had about 30% of Guidance’s revenue, and you cannot help feeling that the price is very low. Third, there’s the question of where AccessData will come up with the money. It’s hard to believe they happen to have $100 million in cash lying around and, with the recent market meltdown, debt is much less of an option than it used to be.

Still, this is not necessarily bad news for Guidance Software. Since its IPO in October 2006, the stock has fallen from a high of $17 per share to a low of $2 per share. The public markets are very unforgiving to small software companies. Guidance has recently made some bold moves, announcing usage-based pricing for its e-discovery product and several notable customer wins, but nothing has moved the stock. So an acquisition offer may be just the ticket to boost the share price, especially if it encourages other, more attractive acquirers to throw their hats into the ring.

Stay tuned, this might get interesting.

Earlier today, Gartner published its eDiscovery MarketScope for 2009. Written by Debra Logan, John Bace, and Whit Andrews, it is perhaps the most comprehensive “buyers guide” available for companies interested in using electronic discovery technology to lower costs.

The eDiscovery MarketScope analyzes about 20 software companies focused on electronic data discovery. Based on extensive interviews with end customers and data from the companies themselves, Gartner rates the companies using criteria similar to those used in its famous Magic Quadrant reports. It also identifies market trends, and makes predictions for 2009 and beyond.

This report is required reading for anyone considering an investment in eDiscovery software, and I strongly recommend that you get a copy, either from Gartner or some other authorized source. To give you a flavor for Gartner’s analysis, a few of its main conclusions are as follows:

1. Bringing eDiscovery In-House Dramatically Reduces Cost

This is a claim that electronic discovery software vendors often make, and prospective customers rightly question. Gartner investigates and finds that many of its corporate clients are saving large amounts of money by using eDiscovery software to reduce the amount they spend on lawyers and legal service providers. It reports that customers typically recover their money from buying eDiscovery software within 3-6 months of implementation.

2. There’s No Single, End-To-End Solution For eDiscovery

Gartner addresses what is probably the most common question I get asked by corporate counsels and litigation support managers – namely, “Isn’t there a single product I can buy that will do end-to-end eDiscovery, covering all aspects of the EDRM?” The answer, of course, is “no” and Gartner goes further by predicting that the answer will remain “no” until at least 2011. So, for the foreseeable future, customers will need to buy best-of-breed products from different vendors for different stages of the EDRM model, and ensure they integrate smoothly.

3. There Are 4 Leading eDiscovery Software Companies

List of vendors achieving highest rating of “strong positive” (from Figure 2, page 10)

Of all the companies it analyzed, Gartner only gives 4 its highest rating of “strong positive”. Each of the four has different strengths. For processing, analysis and review, Clearwell is “fast-to-install and easy-to-use” (page 12) , while FTI’s ability to offer Attenex / RingTail either hosted or on-premise “positions it well for the future” (page 13) . Symantec’s leadership in email archiving makes Discovery Accelerator a good option for its customers who need to search and export data from Enterprise Vault. Finally, Zylab is well-known within law-enforcement circles and has a strong presence in Europe and Asia.

4. There Will Be Consolidation In The Next 12 Months

As the market matures, Gartner predicts that as many as 25% of eDiscovery software providers will either merge, be acquired, or exit the business. Access Data’s ambitious bid for Guidance has publicly put Guidance in play. Beyond that, Gartner suggests that Kazeon and several other players are all likely acquisition targets for larger companies wishing to enter the eDiscovery space.

Of course, Gartner is not the only influential voice in eDiscovery. Earlier this year, George Socha and Tom Gelbmann published their Socha-Gelbmann Survey, which also provides a valuable perspective on the market. How do the two reports compare? That will be the subject of my next post.

Yesterday’s post about the top e-discovery software vendors prompted a couple of interesting comments. George Socha posted a response here, disagreeing with my conclusions; and someone else (“top8”, whoever that is) asked whether one should “always listen to the top 5-10 songs on the list…[or] use the top 5 software products, regardless of one’s situation.”

To clarify, I whole-heartedly agree with George that there is no such thing as a “best” e-discovery service provider – as George says, it really does depend on your situation and I can think of many cases where a smaller, less well-known firm is a better choice than a national brand.

But e-discovery software is different for 2 reasons. First, and most importantly, in software there are increasing returns to scale which do not exist for service providers. The more companies that use a particular software product, the better that product becomes. Speaking from personal experience, when you have a large number of demanding customers, they force you to make your product better – and give you the money to do it. That’s why most technology markets are incredibly concentrated: everything from databases (Oracle) to search engines (Google) have a single dominant player. We are still in the early days of the e-discovery software market, but ultimately I expect it will follow suit and consolidate around a very small number of players.

The second difference between e-discovery software and service providers is that enterprises cannot change their software vendors as easily as they can change their service providers. Once software is deployed behind the firewall, it is fiendishly difficult to get it out, requiring enterprises to pick a single product for all cases. By contrast, it is easy to change service providers, so enterprises can pick the most relevant expertise on a case-by-case basis.

To answer the question posed by “top8”, I am not suggesting that everyone should only read Harry Potter, watch American Idol, and (Heaven forbid!) listen to Britney Spears. Those are matters of personal taste where diversity is what makes for a rich, vibrant society. But there are very good reasons why so many corporations rely on Veritas for backup software, Oracle for databases, Symantec/McAfee for anti-virus, IBM for developer tools, and so on. In software, the best products only get better. That’s why, 5 years from now, the list of top e-discovery software vendors will be even shorter.

There are two independent analyst reports identifying the top e-discovery software vendors.

The first, published in June 2007, is the Socha-Gelbmann Annual Electronic Discovery Survey. The authors, George Socha and Tom Gelbmann, probably know more about e-discovery than anyone else you are likely to meet. As someone who has filled out their 178-page survey, I can tell you it is excruciating in its detail and incredibly rigorous. According to the report, George and Tom contacted nearly 1,000 individuals and collected detailed data from 115 organizations.

The second analyst report is Gartner’s MarketScope, which is published today (December 2007). Its author, Debra Logan, is fast emerging as one of the leading lights of e-discovery and has great instincts about the market. For her report, Debra tells me that surveyed 30 vendors and checked over 90 customer references.

The results from the two reports are as follows:

(1) Companies listed as “Top Electronic Discovery Software Providers Based on 7 Criteria” (Table 19 and 20), listed in alphabetical order. (2) Companies awarded ratings of “Positive” or “Strong Positive” (Figure 1), listed in alphabetical order.

Why are the lists so different? Primarily because of two main factors:

1. Gartner’s list mixes service providers and software companies whereas Socha breaks them out separately. The Socha report has an entirely separate list for service providers.
2. Socha’s report was completed 6 months earlier than Gartner’s. In that intervening period, several new players entered the e-discovery market. For example, Kazeon was ranked by Gartner earlier this year a “niche player” (lower left quadrant) in the enterprise search market, and has not been in e-discovery long enough to participate in the Socha study (or, if they did participate, they did not have enough e-discovery customers to gain a high ranking).

Conclusions

The first conclusion to draw from these lists is that any vendor not in them is probably not worth considering for e-discovery. If neither Socha nor Gartner ranked them highly, then the vendor either could not provide compelling customer references or has lost competitive bake-offs to someone who is on the list. Either way, they are best avoided.

The second thing that stands out is how different these lists are. Of the 21 vendors identified by Socha and Gartner, only 5 are ranked as top e-discovery software vendors by both of them. Those 5 are Attenex, Clearwell, FTI, Guidance, and LexisNexis. So, if you are an enterprise looking for an e-discovery solution, it is clear who you should call first.

Finally, it is worth noting that both these analyst reports are relatively new. This is the third annual survey for Socha, and the first MarketScope for Gartner. That speaks to the fact that e-discovery software is a new, fast-growing product area. More and more enterprises are adopting e-discovery software solutions, and asking analysts about them, because they offer such a compelling ROI.

Thursday, December 6 was a big day for several e-discovery companies. Starting on the home front, it was Clearwell’s 3^rd birthday and we celebrated by announcing our deployment at Bear Stearns. Looking at where we are now, it’s hard to believe that 3 years ago the company consisted of a few guys with an idea. Today, we have over 100 customers who rely on Clearwell for e-discovery, and we are thrilled to count Ruben, Christoph and the team at Bear Stearns among them.

That same day, MetaLINCS announced its acquisition by Seagate (more on that to follow shortly), and Guidance announced the appointment of a new CEO. Victor Limongelli was promoted from President to the top job after what sounds like a comprehensive evaluation of both internal and external candidates by Guidance’s board. I always enjoy my conversations with Victor and applaud Guidance’s decision.

All in all, an eventful day.

Technology companies are notorious for aggressive marketing, whereby they either announce products that do not exist or wildly exaggerate their capabilities. So when ZANTAZ announced its new Desktop Legal Hold solution alongside an image claiming that it can help you “become your company’s superhero”, I was naturally wary. Reading the press release only heightened my suspicion that ZANTAZ’s marketing department may be running ahead of its product development team. For example:

• The release cannot name a single customer using Desktop Legal Hold. The best ZANTAZ could do was quote a retired executive from BASF, who spoke about the potential value from this type of solution (not the actual value realized from this specific solution by a current customer);
• ZANTAZ makes a series of wild claims about the solution. My personal favorite: “Desktop Legal Hold automatically overcomes spoliation, obfuscation, misclassification and non-classification of important data” Need I say more?
• Desktop Legal Hold is not listed in the “Solutions” or “Products” sections of ZANTAZ’s website. Perhaps I’m missing something, but I can only find it mentioned in the press release.

All of this will be re-assuring in the short-term to Guidance, whose Encase product is the leading desktop collection and preservation tool. I doubt customers will be rushing to entrust something as important as their legal holds to ZANTAZ until the product looks more proven, and its capabilities are more clear.

That said, ZANTAZ has clearly signaled its intention to attack Guidance’s core market. ZANTAZ wants to make it easier for its customers to get data into its archives. And it wants a piece of the revenue in this market: from Guidance’s quarterly financials, if you deduct revenue from services and its e-discovery product, it looks like the Encase business is worth $30-35M per year in license revenue. That’s a meaningful prize for ZANTAZ.

It will be interesting to watch how this develops.

For those in regulated industries like financial services, where data retention policies are mandated, every keystroke is tracked and every phone call recorded, the question of how long you should keep data is moot: you keep it for as long as regulations demand.

But for the rest of us in manufacturing, media, technology, government, and elsewhere, it remains an open question. The answer to “what should our email and document retention policy be?” is often a political hot potato, pitting legal and IT’s goal of lower costs against the broader population’s desire to hang on to all their email, just in case they need it later. In fact, the only thing harder than agreeing a retention policy is enforcing it afterwards, as corporate users habitually keep more data than allowed, unless physically prevented from doing so.The reason this matters is that many people believe creating a data retention policy is a key part of implementing an e-discovery solution. I too used to think this way, viewing retention-policy-creation as a necessary rite of passage for legal, IT, and information security people who want to lower e-discovery costs. After all, if the #1 cause of higher e-discovery costs is too much data, then a policy reducing the amount of data looks like a low cost, no-brainer solution.

But life just does not work that way. Outside of the command-and-control environment of regulated industries, retention policies simply do not work. You cannot fight human nature and force people to delete information they want to keep – especially when Gmail, Yahoo Mail, Hotmail and others are training them to do precisely the opposite (i.e., never delete, keep everything) in their personal email accounts.

So, I have changed my mind: to anyone engaged in implementing an e-discovery solution in a non-regulated industry, I say: forget data retention policies, it is a red herring. Too much data is a fact of life that will only get worse. You can no more get people to delete email and documents than you can stop someone writing them in the first place. Instead, focus on the battle you can win by putting in an e-discovery solution that enables you to do two things:

1. Collect data efficiently, so that you have a reliable (defensible) way of getting the data you need. Implementing an email archive from HP, Symantec or others is a great way of approaching this, as is leveraging forensics tools from Guidance or Access Data.

2. Analyze the data up front, so that you can cull it down to only those documents relevant to the case before a human being has to review them. Clearwell’s e-discovery solution is one approach which has worked for a large number of enterprises.

If your experiences, or conclusions, differ from mine, then feel free to post a comment. I am particularly interested to hear about successful examples of data retention policies at non-regulated companies, since I have yet to see one.