e-discovery 2.0

Last month, for the first time, friends of mine who do NOT work in the legal industry starting talking to me about e-discovery. In the past, they had always taken on the glazed look of a bored 8th-grader whenever I spoke about what I do. But suddenly, they were strangely interested and full of questions.

The reason was two articles about e-discovery in the mainstream media which appeared within a week of each other. The first was in the Wall Street Journal, which wrote about how tech firms are at war with lawyers. According to the Journal, the fact that companies are saving money by using e-discovery software is bad news for lawyers, since they are “facing the loss of lucrative client fees.” In response, the lawyers are fighting back: “The attorneys counter that there are pitfalls to replacing them. Early this year, a federal judge required chip maker Qualcomm to pay rival Broadcom more than $8 million after it failed to uncover and share emails relevant to a case.”

I am sure there are lawyers who see technology as a threat, but the firms I deal with are actively embracing e-discovery technology, not fighting it. They see it as another way they can add value to their clients, and would prefer to have their staff focused on practicing law, not mindlessly reading irrelevant documents. So I ended up spending a lot of time explaining to my non-legal friends that there are two sides to the coin. As for my friends who do happen to be lawyers, they focused on the Qualcomm case, pointing out (as we have written before) that the problem was not technology, but rather poor processes and bad judgment on the part of the attorneys concerned.

The second article appeared in the Economist and took a different tack. It argued that the stratospheric cost of e-discovery is gumming up the court system and preventing justice from being served. According to one former justice from Colorado quoted in the article, even mundane landlord-tenant disputes “are now digital wars of attrition”; there are “cases that are settled only because one party cannot afford the costs of e-discovery”; and, many “plaintiffs cannot afford to sue at all, for fear of the e-discovery costs.”

I love the Economist’s tongue-in-cheek style and thought the article made many valid points. My one disappointment was that its spin was unequivocally negative, as though e-discovery is a self-inflicted wound on the American judicial system. Nowhere was there mention of the fact that electronic evidence often helps litigants get at the truth. Rather than incomplete recollections or “he said-she said” claims and counter-claims, there’s no disputing an email that captures a person’s words and actions in black-and-white. Nor was there any mention of how technology is solving the problems that it inadvertently created: today, there are many products that rapidly sift through electronic information, dramatically lowering the cost of e-discovery.

It is great for everyone in the e-discovery community for our domain to get more ink in mainstream, quality publications. I expect that the trend will continue as the industry grows, and especially once the investigations start into our current financial meltdown.

I was recently asked: “what are the first things you do when your client calls you about a case requiring e-discovery?”  So, for the benefit of all, I’ll post my answer.

My first caveat to the advice was context.  Since, while a lot of attorneys have attended CLEs or have read about e-discovery, it’s not the same in the real world.  As the old Spanish Proverb goes:

It’s not the same to talk of bulls as to be in the bullring.

Keeping in mind that reality may differ significantly from academics, here are some things to consider when the next e-discovery case comes up.   Please also keep in mind that these steps (like the EDRM workflow) aren’t linear and may in fact occur cyclically or in parallel:

1. Preserve, preserve, preserve

Nothing is more important than meeting the initial preservation obligation, which begins when litigation is “reasonably likely” – as opposed to just when the complaint is filed.  This first step in the long journey can easily be a trap for the unwary/unprepared.

The challenge once you’re past the trigger issue is to then identify the boundaries of the duty to preserve, i.e., what evidence must be preserved?   This inquiry is often initially comprised of identifying key players, date ranges and data types.

Another significant challenge in this step is to monitor and update the legal hold process.  And, given that litigation more often than not spans years, it’s easy to initially succeed at the preservation effort, but then later fail on execution.  The best way to minimize risk in this step is to move quickly from preservation to collection.  See Is Preservation in E-Discovery Overrated?

2. Work backwards

Once preservation (and ideally collection) is adequately covered, the next step is to start thinking about the end of the process and what success (or lack of failure) looks like.  The exposure and profile of the matter are important to consider when you embark upon an e-discovery project since it’s critical to scale discovery efforts appropriately.

One thing, in particular, that is very important to consider early in the process is the type of production format that will be preferred by reviewing counsel and the opposition.  TIFF-based image productions (which are historically well accepted) are often pitted against native file ESI reviews.  Either format may or may not be acceptable given the situation and the applicability of FRCP Rule 34.

3. Understand the technical landscape

Most attorneys, but for a rare few, aren’t capable of really comprehending technical nuances of the complex and interrelated IT systems found at most Fortune 2,500 enterprises.  Fortunately, they are quite adept at working with experts (either consulting or testifying) to help them get to the bottom of difficult to comprehend and explain issues.  The key is find the right technical people who understand IT systems and who can explain it to judges, juries, and attorneys alike, especially for some of the most common ESI repositories like: email servers, archival systems, shared network drives, instant messaging servers, archival repositories (e.g., tape libraries, real time back-up systems, etc.), records management systems, knowledge management systems, proprietary, but highly leveraged, internal applications, offsite repositories (e.g., hosted IT or email systems) and significant partner or subsidiary data stores.  In many instances it will make sense to leverage or create a map of the data universe so that nothing is missed and inaccessibility arguments can be cogently detailed.

4. Get your lingo straight

Assumptions, whether in e-discovery or not, are often dangerous.  In the complex undertaking where multiple parties are handling ESI it’s critical to make sure that everyone is on the same page especially since every company handles IT, records management, ILM and information security differently.  So, when working with these disparate constituents the outset of an engagement is the right time to make sure everyone is on the same page.  Therefore, standardize on a set of commonly used terms. Examples of potentially ambiguous topics include “imaging” ,“archive”, and “records.”

5. Don’t assume your client will really be helpful

I’ve been involved with hundreds of e-discovery engagements and I’ve found that almost universally the end client professes a profound willingness to help out.  And yet, actual “help” is relatively rare.  To qualify this, it may be prudent to ask several additional questions:

• Does the Client have the time to actually help?  Everyone at the client’s site has a day job that they’re tasked with above and beyond transient e-discovery needs.  So, while bandwidth generally is important, what’s more critical is the ability to comply with aggressive judicial deadlines.
• Are the people helping the ones you’d want to see on the stand?  It’s often not realistic to have internal folks (especially IT and Records Managers) stay isolated during the various pre-trial events - meet & confer conferences and potentially 30(b)(6) depositions so it’s important to evaluate how a given witness will fare when providing testimony.
• How likely is it that you client would throw you under the bus if things went wrong?  In my opinion, there is now more reason for outside counsel to manage the risks of an e-discovery project going awry.  See, Sullivan and Cromwell’s suit against EED.  Some will wisely bring in 3rd party consultants/experts to have a neutral, unbiased constituent in the process.

6. Build a budget and team (internal/external)

Everyone is probably now aware of how expensive e-discovery can be if managed improperly.  This makes it all that more imperative to work quickly to get a rough sense of the scope (which will lead to a budget) and the client’s willingness to absorb associated charges.  The most important step is to right-size the e-discovery effort with the risks inherent in the corresponding litigation/investigation.  Otherwise, there’s a high likelihood that e-discovery process will be over-engineered (too expensive) or under-scoped (cutting dangerous corners).

7. Figure out your risk profile

Similar to right-sizing the budget, it also makes sense to adopt a “horses for courses” approach to e-discovery since there is no singular way to handle a given matter.  For example, in one case you make take forensic images, restore backup tapes, capture instant messaging data, harness metadata, or decide to do an automated review with a with a “clawback” provision. In either case, the only mistake is to assume that an approach from another, dissimilar matter is warranted in the instant case.

8. Assume the opposition is better informed than you are

While this actually may not be the case, it’s a safer bet that assuming a level of naiveté that may not exist.  What is certain is that the Plaintiff’s bar is increasingly well informed and can be very aggressive.  They’ve seen the playbook that calls for baiting the opposition into a discovery misstep that can result in significant, case altering sanctions.  According to a recent survey, 63% of the polled attorneys said that e-discovery is being abused by counsel, so it’s important to be wary initially.

It’s also important to consider the potential reciprocity of a given matter and adjust your position accordingly.  In many instances it’s easy to consider your role only as a producing party, but with cross/counter claims it may be possible to simultaneously be propounding discovery and in the opposition’s shoes.

9. Prepare for an early case assessment

A recent industry survey found that effective early case assessment (ECA) approaches reduced overall litigation in half of the cases evaluated, and resulted in favorable outcomes for 76 percent of the cases.   The key to this methodology is to use the available next generation case analysis solutions earlier in the process, not just to review data for relevancy and privilege, but to:

• Identify the key players. This is critical in order to have a defensible legal hold process
• Evaluate the posture of the case to determine how it looks on the merits
• Diagnose potential outliers in the e-discovery process to facilitate meet and confer discussions and help create “inaccessibility” arguments
• Conduct a search term analysis for keyword negotiations during meet and confer discussions.  Objectively demonstrating the results of proposed search queries can go a long way in speeding up keyword negotiations

10. Don’t take search for granted

For many attorneys, e-discovery search is just like Lexis or Google.  Unfortunately, that isn’t the case.  Instead, it’s become highly complex and is now receiving significant judicial scrutiny.  In Victor Stanley v. Creative Pipe Judge Grimm suggested that attorneys need to rethink how they’ve traditionally managed the search process:  “[F]or lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread.”  It’s now important to devise (and share at early meet & confer conferences) a defensible search strategy that can withstand judicial scrutiny.

In my last post, I discussed how the “black box” design of enterprise search engines makes it challenging to defensibly use keyword search in e-discovery and follow Judge Grimm’s guidance in Victor Stanley, Inc. v. Creative Pipe, Inc., 2008 WL 2221841 (D. Md. May 29, 2008).  In Victor Stanley, Judge Grimm notes that because keyword search technology is prone to producing over- and under-inclusive results, attorneys using keyword search should adopt one of two approaches: either collaborate with the opposing party to agree on keyword search methodology, or utilize best practices that demonstrate they have taken reasonable measures to reduce over- and under-inclusiveness.  However, the black box search technologies that are used in e-discovery today make following this guidance difficult.  They can’t reduce under-inclusiveness without increasing over-inclusiveness.  And they make it expensive to utilize collaborative or best practices methodologies including testing, sampling, refining and documenting searches.  All of which begs an obvious question: what can be done to improve search for e-discovery?

In my opinion, the answer is simple: e-discovery search needs to become more transparent.  Instead of being forced to feed one search query at a time into a “black box” search engine and then getting results  with no idea how those results were generated, lawyers and litigation support professionals need technology that provides them with greater visibility into the search process. They need to understand how the results were obtained, so they can reduce both the over- and under-inclusiveness of keyword search, and easily follow Judge Grimm’s advice to improve the defensibility of their search methodology.

A transparent search solution should have four key elements:

1. Transparent query expansion.  Query expansion is the process by which search engines take the query that the user submitted and expand or convert it into a new and improved form.  Wildcard, stemming, concept and fuzzy searches all follow this query expansion process.  For example, the search “divers*,” would be expanded to search for all the words that start with “divers” in the data set, such as “diverse,” “diversity,” “diversion,” “diversification,” etc.  In transparent search, query expansion would be exposed to users, allowing them to include or exclude expanded keywords. To continue with the previous example, a user that is searching for documents related to diversity would then have the ability to exclude false positive expanded terms, such as “divers”, “diversion,” and “diversification” from the search.  Making query expansion transparent can significantly reduce the over-inclusiveness of keyword search.  It also makes it practical to use technologies, such as concept and fuzzy search, that have not been used to date because of their complexity and tendency to produce massively over-inclusive results.
2. Multiple query support. When a search contains multiple keyword queries, such as “hiring” and “interview,” transparent search should provide visibility into the results for each individual query as well as the combination of all the queries. For example, with the search “hiring OR interview,” users should have separate visibility into the results for “hiring” and “interview” as well as “hiring OR interview.”  They should know that out of the 100 documents that match “hiring OR interview”, only 5 match interview and 95 match hiring.  This kind of visibility is critical if you want to either collaborate or follow search testing, sampling, and refinement best practices when there are a large number of queries.
3. Rapid sampling. Transparent search should support the ability to rapidly sample the results from all of the individual queries, such as “hiring” and “interview”, contained within a search. It should also be easy to take a random sample of non-matching documents in order to assess whether one or more searches have identified as many of the relevant documents as possible.  As Judge Grimm states in Victor Stanley when assessing keyword searches used to find privileged documents, “The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive.”
4. Automated documentation. Transparent search technology needs to document all aspects of the search process including (but not limited to) any keyword that has been excluded during transparent query expansion, the combined results of a search containing multiple individual queries, and the results for each of the individual queries within that search.  Automatically documenting the search methodology used and the results obtained is critical so that users can “show their work” if their search methodology is ever called into question.

Benefits of Transparent Search

By addressing the main technology challenges of keyword search, transparent search provides significant benefits to attorneys and litigation support professionals using search for e-discovery. First, parties that adopt transparent search can improve the defensibility of their e-discovery search practices. By enabling iterative testing, sampling and refinement, transparent search allows users to adopt the approaches recommended by Judge Grimm when it was previously impractical to do so.  At the end of the day, this means less risk.

Second, the use of transparent search can substantially reduce downstream production and review costs by removing false positives. For example, it is not uncommon for certain wildcard searches to generate results where 20-40% of the included documents are false positives that can be removed by transparent query expansion.  This can result in thousands of dollars of savings on a single search query.

Finally, transparent search can dramatically reduce the time and cost required to complete the search and culling stage of e-discovery. Currently, it can take hundreds of hours to run a significant number of searches one at a time, document the results of each search, and sample and refine each individual query. With transparent search, running multiple queries and documenting each of the individual results takes minutes. Sampling each of the individual queries takes seconds.

When it comes to e-discovery search, it’s important to recognize that there are no “silver bullets.”  Search will remain an imperfect science with the possibility of over- and under-inclusive results.  But equally, there is no doubt that search remains the best solution for reducing the vast quantities of electronic information that are a part of every e-discovery process down to a reasonable level for human review. While attorneys and litigation support professionals can’t completely remove the imperfections of keyword search, they can, with transparent search, take action to minimize the impact of these imperfections and defensibly meet the requirements of new case law.  In doing so, they will be able to turn their attention to where it should be: the substance of the case.

In the spirit of the popular New York Times magazine feature, with this post we inaugurate what we hope to be a long-running series of interviews with e-discovery luminaries to get their take on emerging ideas and trends (and hopefully have some fun as well).

Today’s questionee is e-discovery and forensics expert (and popular Law Technology News columnist) Craig Ball.  Craig’s combination of wit and insight speaks for itself, so let’s just get right to the questions.

1) The cases that are on everyone’s mind are O’Keefe/Lundin and Victor Stanley. What’s the practical impact of these rulings to the e-discovery practitioner?

Certainly these decisions have captured my enthusiastic attention.  Lawyers now have to devote greater care and thought to electronic search, and wake to the empirical evidence establishing the shocking shortfalls of keyword search in unstructured ESI collections.  The days of “let’s try these search terms and see what happens” are numbered.  Queries that will be run across mushrooming collections must pass muster in terms of noisiness, ambiguity, potential for misspelling, affinity to stemming, synonyms, slang, acronyms, IM-speak and other criteria unfamiliar to a profession that prides itself on precise expression.  Lawyers need to embrace concepts of “precision,” “recall” and “sampling” with the same fervor we once brought to the Statute of Frauds and the Rule Against Perpetuities.

Currently, lawyers on both the north and south sides of the docket are the unjust beneficiaries of slipshod search.  Requesting parties benefit from the economic leverage attendant to costly-yet-unavailing fishing expeditions while counsel for producing parties mint obscene pyramidal profits reviewing mountains of electrochaff.  Despite all the vitriol, rarely does either side’s counsel set out to exploit flawed searches.  It’s mostly blissful ignorance at work, coupled with little incentive to fix what’s broken.  Accordingly, Judges like Facciola and Grimm are picking up the baton and running with it.  It’ll be a long, tough race—and not every jurist will head for the tape—but I applaud those who’ve left the blocks!

Search demands nuance, discipline and scientific method.  Prepare to routinely test queries against sample collections, as soon that practice will be as commonplace as DNA testing in paternity cases.

2) What can e-discovery technology providers do to help?

At the risk of appearing ungracious, I can’t help but note that vendors eat at the same gluttonous table as lawyers, and vendor marketing is often so much snake oil.  Until the EDD vendor community takes a longer view of the market, stops building businesses for acquisition and starts building them to last, I don’t think they can be of much help.  The industry should stop pretending their processes and software are “proprietary” and touting their secret sauces.  Instead, how about delivering consistent, predictable service and pricing delivered by experienced, reliable and unflinchingly honest, genuinely knowledgeable personnel who welcome the chance to help lawyers understand this stuff.  If employees stayed around more than six months, that would be nice, too.

3) You recently participated in a new track at LegalTech West called FutureTech.  For those who missed it or the follow-up podcasts, what’s an emerging e-discovery trend that you think might take people by surprise?

Several come to mind.  Mediated meet-and-confer, for example.  The cost of a failed EDD effort can dwarf the amount in controversy, so it makes sense to turn to neutral, technically adept intermediaries to help resolve nettlesome questions, of scope, search, forms of production and cost sharing.  Folks just behave better when company comes.  I also foresee divergence between discovery and the other traditional phases of litigation.  We may see entirely different teams handle discovery in a zealous but non-confrontational manner, leaving the scorched earth stuff to others.

Another development that will sneak up on most lawyers is the growing marginalization of text.  As natural interfaces emerge—where you will talk or gesture to your computers—and as communication gets more real time and visual, words will manifest conduct less frequently.  Take YouTube.  I don’t get it—to me, it’s silly and boring—but it’s rich and exciting to my kids…and text is tertiary.

Something else that will change is where we look for evidence.  If you were pursuing discovery against a teenager, where would you go to locate their most revealing ESI?   Social networking (virtualized storage)?   Cell phones and laptops (portable devices)?   Gaming devices (alternate platforms)?  In ten years, don’t imagine they won’t favor and extend the tools they grew up with.

Data is the ultimate portable commodity, so it’s odd we don’t take our computing environments with us. We will. If desktop machines survive, they will be little more than screens with network connectivity temporarily hosting the virtual identities we carry in our pockets or store online. Local hard drives will be an increasingly irrelevant place to search for files as EDD turns to personal storage devices and online storage.

Other trends lawyers may not foresee: People will retain much more data as there will be little incentive and less time to make it go away. “Cheaper to keep her” will be how most of us deal with data.  Location data will be routinely tracked by many devices with GPS functionality on and about our person, so this will become a new and useful evidence stream.  Virtual machines will be used as forms of production.  Local storage will give way to cloud storage.  Hey, I could do this one all day!

4) You have an extensive background in both e-discovery and computer forensics. Do you see a convergence, or will they remain largely separate worlds from a process and technology perspective?

I see convergence already.  “Forensically sound” practices are creeping into EDD harvest and traditionally rigid approaches to disk forensics are being challenged by the practical realities of immense volume and mission-critical operations.   We see the growth of “live” forensics, hash values displacing Bates numbers and operating systems allowing more and more deleted information to be easily resurrected.

The tools and techniques of each discipline are also converging.  But there will remain a distinction between the two flowing from the unique ability of a skilled forensics examiner to distill the bits and bytes into a compelling tale of human strength or frailty.  It’s painfully easy to misread the significance of digital footprints.  There’s a component of science and art to computer forensics that will insure its distinction and growth.

We face convergent challenges, too.  In both forensics and EDD, the lure of lucre pulls in people who really ought to be doing something less harmful.  Lives, liberty, fortunes, and careers hinge on some computer forensic examinations; yet, some schools and tool sellers promote the notion that you can learn what you need to know over a long weekend.  Just as many copy shops decided they were e-discovery experts one dark night, a lot of poorly trained, incurious and careless forensic examiners are popping up all over.  I’m frankly appalled by some of what I see out there.   Where I hope we ultimately converge is a high standard of professionalism and proven expertise.

5) Finally, the question on the mind of every loyal “Ball in Your Court” reader: Which court is it — basketball, tennis, or volleyball?

I’ve never been much for team sports, but if I have to choose, I opt for the one played on the beach by fit, bikini-clad women.  I may be a hopeless nerd, but I’m not stupid.

Yesterday, George Socha and Tom Gelbmann published summary results for their 2008 EDD survey. George and Tom gathered self-reported data from 85 e-discovery service providers and 40 e-discovery software companies. To help vendors resist the temptation to “exaggerate” their accomplishments, they then cross-referenced the responses against independent surveys submitted by 29 law firms and 19 corporations, and applied a healthy dose of their own good judgment. The outcome, which they will publish in-full next month, is a great snapshot of the industry, and probably the most objective ranking of e-discovery vendors that you can find.

By comparing this year’s results to the 2007 survey, you get a sense for how much has changed in the e-discovery world over the past 12 months:

Top E-Discovery Software Companies

Note: arrows show change to rankings from last year’s Socha-Gelbmann Survey

Autonomy and Clearwell move up to the Top 5, overtaking Attenex and CT Summation which slip back to the second tier. There are also 3 new names ranked 6 through 10 (Epiq, iConect and Symantec) who displace Cataphora, Doculex, ISYS, and Oracle, none of whom even make it into the top 15. In other words, 70% of the rankings have changed since last year.

If a litigation support manager were to focus only on the Top 5 in making her e-discovery software decision, she would have a choice of some very different solutions. Autonomy positions itself as a high-end (expensive) platform for corporations, while Lexis offers a comprehensive toolset for law firms. Guidance and Clearwell are complementary in that both provide best-of-breed solutions for parts of the EDRM model: Guidance is the leader in collection and preservation, while Clearwell is the leader in processing, analysis and review. Finally, FTI takes a services-based approach which centers around RingTail, its hosted review application.

Looking lower down the list, there were some other interesting results, primarily around which companies were NOT ranked. Kazeon made it into the third tier (ranked 11-15) whereas StoredIQ, its main competitor, did not. Nor did Recommind break into the rankings, despite making a major push into e-discovery from knowledge management over the past year. But the most striking absentees are PSS Systems and Exterro, which have pioneered litigation hold management for Fortune 100 companies. I can only guess that they cover too much of niche market to warrant inclusion in an industry-wide report.

Top E-Discovery Service Providers

In contrast to the world of software, e-discovery services saw much less movement in this year’s rankings:

Note: arrows show change to rankings from last year’s Socha-Gelbmann Survey

There was only one change to the top 5: Fios moved up, displacing Guidance which plummeted 10-20 places down to a 16-25 ranking. In addition, there were two new players in the top 10, Epiq and Huron, who edged out Electronic Evidence Discovery and Ernst & Young.

Conclusion

Changes to the software rankings reflect broader changes in the e-discovery market. As e-discovery has moved in-house, corporations have become a major driver of purchase decisions that were previously left to law firms. Many software companies, such as Attenex, have struggled to make this transition, while others, such as Clearwell, have capitalized on it. There has been no such change in the service provider world and, as a result, the rankings are relatively stable.

It will be interesting to see what happens next year. Every other software space is dominated by a small number of players, like Oracle for databases or VMWare for virtualization. If the same is true for e-discovery, then we can expect many fewer changes to the software rankings in future surveys as the leaders pull away from the pack.