e-discovery 2.0

Almost two years ago I wrote one of my first blog posts entitled “Review-less E-Discovery Review.”  Despite the tongue twister of a title, the post posited that “there is a very real possibility that we’re on the cusp of computers taking over a significant e-discovery task for attorneys.” I’d like to take a look and see how much (if at all) my prognostications have materialized.

A cynic might think that this is the moment where E-Discovery 2.0 jumps the shark.  But no, this isn’t one of those sitcom episodes where they flashback to previous shows as an easy way to recycle content.  Instead, it seems useful to see how the legal market has evolved from a litigation workflow perspective, particularly with some vendors touting the benefits of review-less technologies like predictive coding.

In the original blog, I noted that there was a “scenario where a non-manual review methodology may make sense” (while importantly noting that “this approach is not without risk”).  Since my last post there has been the successful adoption of Evidence Rule 502,which makes this methodology (at least conceptually) safer.

But again (imagine dreamy flashback mode), here were the guidelines I previously proffered:

1. Large data set.  This may sound a bit obvious, but a non-manual approach is best suited for large, unwieldy data sets.  The corpus doesn’t need to be in the terabytes, but the data set should be evaluated in term of discovery processing costs and attorney review estimates.
2. Short Production Timelines.  Once the above calculations are conducted, the next step is to determine if a human based review could even conceivably be conducted in the given time frame.  In many instances, an eyes-on review process just won’t be feasible since there won’t be enough bodies to throw at the problem.
3. Next Gen “PAR” Tools.  In order to pull this “review-less” review process off, both safely and quickly, the responding party needs to have access to fast, robust processing, analysis and review (“PAR”) tools.  Certainly, it’s possible to have this scenario work with an e-discovery service provider, if they have the capability.
4. Relatively Small Amount in Controversy.  For the time being, this approach should not be considered for any “bet the company” litigation, nor anything with significant downside risk (governmental inquiries, punitive damages, class actions, 2nd requests, etc.).  Yet, for many standard commercial lawsuits, corporate investigations, HR claims, etc. this review-less approach may be worth considering.
5. Ability to Use a Clawback Provision.  Entering into a clawback provision with the opposition is mandatory in this methodology since the chances of an inadvertent production are statistically ever-present.  Yet, until Evidence Rule 502 is resolved, there will always be a risk that the clawback won’t be enforceable against 3rd parties.
6. Non-governmental Production.  Most information in governmental productions becomes part of the public record, meaning that a clawback isn’t going to be feasible.  Here, trade secret information, personally identifiably data and the like would be disastrous if pushed out into the public domain.

The goal of this post is to see if this dog is any more ready to hunt than it was two years ago.  The short answer (right now) appears to be: No.

We all know that litigators are both risk adverse and generally slow to adopt new technology approaches.  This is particularly true when there’s a perception that they won’t have insight into the technological black box behind automated coding/tagging decisions.  Litigators are understandably sensitive about the ability to prove up the reasonability of their search and review processes.  This “reasonableness” requirement lines up both with the Victor Stanley requirements and FRE 50(b), which eliminates the chance of a waiver only “if the holder of the privilege or work product protection took reasonable precautions to prevent disclosure.”

Given this ongoing hesitancy, the question remains shouldn’t we be seeing more movement in automated review than the glacial progress that’s been achieved to date, particularly with the known shortcomings of the eyes-on review process?  Most are familiar with the 1985 STAIRS study by Blair and Marion where the percentage of relevant documents lawyers thought they had found using Boolean Keyword searches was 75% – when the percentage they actually found was 20%.

But, despite the known deficiencies of eyes-on review it follows into the “go with the devil you know” mindset that often makes sense when dealing with judges and juries who aren’t likely to grok newer-fangled approaches.

In addition to these high-level, almost dogmatic challenges, there is one other tactical element I’d add to my previous list (of 6 factors).

7. All documents processed up-front (no rolling collection). I’ve heard some in the trenches e-discovery experts claim that they’ve never had a case that didn’t involve at least some level of incremental data collections.  Whether this is an overstatement is immaterial.  The fact is that a large number of e-discovery projects involve ESI that is collected (and then processed) in dribs and drabs.  This if often a good thing, largely attributable to the incremental (start slowly) nature of a well thought out e-discovery project where a smaller number of initial custodians are processed, then ECA is conducted and only then is the additional ESI added to the corpus.  This common methodology causes some significant heartburn for a review-less methodology since the ever changing nature of the corpus makes it difficult/impossible for a sample to be truly extensible to what will eventually be the entire data set.  For this reason, the review-less approach should be limited to where the entire corpus is collected and processed at once.

In sum, the seven foregoing factors appear to still be largely valid and create an environment where an automated, review-less methodology will only make sense in a relatively rare set of circumstances.  This may change in the future, but given the risk adverse DNA of most litigators I can’t imagine this tipping point happening any time soon.

It’s been a little over a month since the news first broke that Guidance Software was the frog in an electronic discovery kettle whose water had just reached the boiling point, with the arbitrator in an employment case demanding, “I want this game-playing stopped.” We thought that, with a little time between the initial story and now, it would be worth taking a step back and looking at possible lessons learned — not so much for Guidance specifically, but for enterprises who find themselves in similar situations, as well as the electronic discovery community that serves them.

First, a quick summary. Based on published accounts, it seems like a classic discovery situation (that’s just plain old discovery, without the “e”): a party is sued and fails to produce a document that, lo and behold, surfaces via some other source, throwing the integrity of the sued party into question. After all, if one potentially incriminating document wasn’t discovered, then who knows what else could be out there?

Guidance contended that it did everything that was required of it, and that it didn’t have (or couldn’t find, despite good faith efforts) the emails in question. But, of course, that didn’t stop the litigation support community (via forums such as the Litigation Support List) from pouncing on the perceived hypocrisy.

After all, how could a leading, publically-traded electronic discovery company get caught up in such a mess? How could their cutting-edge electronic discovery technology not have saved them? Or their (hopefully) best-in-class internal electronic discovery processes? If the electronic discovery companies don’t have their acts together, what about all the other poor souls who lack their knowledge and expertise?

That last question is a scary one, particularly given today’s environment, and it’s why the situation has stirred up so much chatter out in the electronic discovery blogosphere. Almost without exception, commenters have jumped to one of two conclusions. Either (a) Guidance has not followed proper e-discovery best practices, or (b) Guidance has willfully chosen to hide relevant documents that it could have produced, because they would be detrimental to its case.

Let’s explore each of those conclusions in a little more detail.

First, is there any direct evidence that Guidance did not follow electronic discovery best practices? The answer there is murky. Certainly, from Guidance’s perspective, the answer is a resounding “no”. They continue to claim that the emails that were produced from another source did not exist on the various laptops, desktops, and servers that were part of the initial discovery request, and it is certainly possible that that is true. Perhaps Guidance had a 1-year retention policy for emails, and the emails in question were outside of that policy. Perhaps the individuals involved had legitimately deleted the emails in question prior to receiving a litigation hold notice, without thinking that they would ever be relevant to a legal matter. Certainly an independent observer has grounds for incredulity here, but it does not necessarily follow that Guidance did not follow electronic discovery best practices for a company of their size and resources. Certainly, from the reports, they did not exactly act in a way that earned much confidence from or favor with the arbitrator. However, that’s a completely different issue, and one which may be a legitimate tactical decision by Guidance (to avoid, for example, the high cost of recovering the corrupt backup tapes).

Second, what if Guidance willfully chose to hide relevant documents? At this point, there is no evidence that this is the case. And, you would think that of all of the companies out there, Guidance would be keenly aware of the extremely high level of risk associated with this strategy. A company well-versed in computer forensics understands keenly that the odds of any potentially negative emails not being out there, somewhere, in cyberspace are incredibly small. Thus there is little incentive to intentionally hide documents. If, however, a company did make such a perilous and unethical decision, it has nothing to do with a lack of e-discovery best practices or technology: it simply has to do with a lack of ethics.

So, has the coverage of the Guidance situation been nothing more than an electronic discovery witch hunt? Far from it… even if both of the “conventional wisdom” conclusions are in fact wrong.

Why? Because even if Guidance has its electronic discovery house in order and is acting with complete integrity, if there’s one thing that anyone in the electronic discovery business should have taken away from the last 5 years of court rulings, it’s that perception and transparency in electronic discovery is everything. Electronic discovery is technically complex and fraught with challenges, and companies – particularly those who are perceived as having vast expertise in the space, whether as vendors (i.e. Guidance) or institutions (i.e. pick your favorite TARP recipient) – have to act in such a way as to appear spotless before the court of law and the court of public opinion.

Assuming you already have your electronic discovery house in relative order (a baseline, fundamental requirement for doing business today), perhaps the most important take-away from Guidance is how carefully you need to consider how minor electronic discovery slip-ups, whether real or perceived, can bite, big time. The legal and media environment is primed to pounce on any hint of a cover-up or conspiracy, and enterprises must go the extra mile (or two, or three…) to ensure that their e-discovery efforts are, and will be perceived, as upright, ethical, and above reproach – or be ready and willing to pay the price in sanctions or loss of public confidence.

Ralph Losey, never one to mince words, recently analyzed a recent litigation survey from the elite Fellows of the American College of Trial Lawyers. The survey highlights the fact that one of the main problems facing the U.S. legal system today is (surprise!) e-discovery. Also (not) a surprise is that the study “places the blame squarely on poor rules, bad law, and judges”, while overlooking the role that lawyers play in the problem.

In his analysis, Ralph makes a number of insightful observations that should help lawyers move from being e-discovery troublemakers to being part of the solution. However, one of his key critiques is targeted not at lawyers but rather at the vendor community: “[E-discovery] is too expensive because lawyers and judges do not know what they are doing, and do not know how to properly cull and review email, and because clients are disorganized pack-rats. Many of the e-discovery vendors are also misinformed, but often they do know better; they just have no pecuniary interest in aggressive culling. Some may even seek to line their own pockets in inflated discoveries.”

As Ralph bluntly points out, pecuniary interests (translation: money) plays a big role here, but so does risk reduction. Imagine you’re given the opportunity to process a 2 terabyte case all the way through to review. With the “funnel” of e-discovery costs placing the highest dollar per gigabyte value on the end of the process (i.e. review), what’s your incentive to cull aggressively at the beginning? Not much from a revenue perspective, certainly, but also not much from a risk perspective: particularly when you have sanctions and lawsuits on your mind and are thinking about the potential liability that you incur by excluding potentially relevant documents by using too broad a brush (or pair of garden clippers) in your pruning.

How do we move forward? As document volumes continue to grow, it’s clear that aggressive culling (with a few caveats which we’ll get to in a minute) is a critical tool for managing costs and improving case outcomes (let’s go out on a limb and define “improving” as producing fairer and more equitable rulings). However, in order to adopt more aggressive culling as a standard part of the electronic discovery process, the community has to come to terms with three things:

• The Myth of Perfection: There may be perfect abs, but there is no perfect e-discovery. Organizations like the E-Discovery Institute are doing fantastic work to measure and improve the accuracy of electronic discovery efforts, but in the end it’s tough to make the argument that having 100 contract attorneys manually reviewing 10 million documents will necessarily produce a better overall e-discovery outcome than  10 specialized attorneys reviewing 200,000 documents that were aggressively (but thoughtfully) culled from initial 10 million document set. There simply is no black and white set of rules that will lead to a perfect process.
• The Benefit of Cost Control: Given that, it is in the best interest of everyone involved (yes, even vendors) to choose the most cost-effective process that provides a high likelihood of producing the information relevant to the case.  This means “saving your bullets” by not spending all of your e-discovery dollars up front in a case pursing the perfection myth, but instead approaching discovery in an incremental fashion which can adapt to changing facts and circumstances as the matter unfolds. How, you may ask, do vendors benefit? They can become more strategic e-discovery advisors by working with counsel over the full lifecycle of a case, providing higher-value (and, by the way, more interesting and intellectually challenging) consulting services to help incrementally adjust and adapt the course of e-discovery. As Ralph puts it: “…Trial lawyers should accept that specialists in the field of e-discovery are a necessary evil. If an e-discovery specialist knows the field, they can save you money and take you out of the e-discovery morass faster and more reliably than a dozen new rules. The world today is too complex for one man or woman to do it all.”
• The Value of Defensibility: Many of you likely winced at the term “high likelihood” in the previous point. “Sacrilege!” you cried. “I demand certainty!” First, go back and re-read the first point about the Myth of Perfection. Then, consider that a better way forward may be an approach to e-discovery that involves more aggressive culling early in the process to focus on the most important documents first, more iterations to adapt to changing facts and circumstances, and, all along the way, a complete audit trail that provides defensibility in the event that any aspect of the process is ever questioned. Such defensibility would include specific documentation about the culling decisions that were made, down to the keyword and “sub-keyword” (i.e. wildcard expansion) level, so all the cards are on the table for everyone to see.  The value of defensibility when performing aggressive culling is enormous, in that it adds an additional measure of safety and trust to the process, minimizing the amount of doubt and second-guessing that so often plagues e-discovery negotiations.

By coming to terms with the fundamental imperfections of the e-discovery process and embracing the promise of lower costs and the agility and responsiveness that can be gained with a more iterative approach, everyone stands to gain from the safe and controlled adoption of aggressive culling – yes, even the vendors (at least the smart ones) and their ever-present pecuniary interests.