e-discovery 2.0

The other day, Michael Clark of EDDix sent me a fascinating academic paper (thanks, Michael!) about “information inflation” at its impact on the legal system. I had never really thought of it this way, but there have really only been 3 significant events in the evolution of information:

1. Writing (c. 5,000 years ago): Pre-historic man started to etch his markings on clay tablets, stone, wax, papyrus, bark, cloth, wood, paper, cave walls and anything else that came to hand.
2. Printing (c. 1450): Gutenberg’s movable type printing press enabled mass production of information, contributing to (among other things) the Renaissance and the Scientific Revolution.
3. Digitization (c. late 20th Century): The personal computer, wide area networks, internet, email, have all led to a massive explosion of information in the past 50 years. As the article points out, “close to 100 billion emails are sent daily…In a small business, whereas formerly there was usually 1 four-drawer file cabinet full of paper records, now there is the equivalent of 2,000 four-drawer file cabinets full of such records, all contained in a cubic foot or so in the form of electronically stored information.”

How can the legal profession cope, given that a lawyer’s job is often to synthesize this mind-boggling amount of data? Fortunately, the authors have a solution:

“A family of computer technology employing new types of search methods and techniques beyond use of mere keywords should now be considered for use in litigation….Litigators can no longer depend on manual review alone. It is too time-consuming and expensive – with cost often exceeding the amounts in dispute.”

To illustrate its point, the paper tells the story of the White House and the problem of a billion emails. During the Clinton administration, the White House agreed to a form of electronic record keeping called ARMS (Automated Records Management System). At the end of each administration, these records are handed over to the National Archives and Records Administration (NARA). The table below shows the number of stored emails NARA has, or expects to receive at the end of each administration.

Now assume that, like previous administrations, the Next President’s administration is subject to a lawsuit that requires e-discovery. The paper calculates:

“Without employing any automated computer process to generate potentially responsive documents, the review effort for this litigation would take 100 people, working 10 hours a day, 7 days a week, 52 weeks a year, over 54 years to complete. And the cost of such a review, at an assumed billing rate of $100/hour, would be $2 billion. Even, however, if present day search methods are used to initially reduce the email universe to 1% of its size (i.e., 10 million documents out of 1 billion), the case would still cost $20 million for a first pass review conducted by 100 people over 28 weeks, without accounting for any additional privilege review.”

This is a great example of why companies and government agencies are adopting e-discovery 2.0 technologies that go far beyond keyword search. In the face of information inflation, what choice do they have?

I will let others better qualified than me comment on the political implications of the recent furore over the firing of several US Attorneys. But one interesting aspect of the story from my perspective has been seeing email front-and-center in the news cycle.

Everyone from CNN to the Washington Post to the Wall Street Journal has led with “email-driven” stories, with headlines like “Rove, Gonzales discussed firings, e-mails show”. On March 14, the Journal (subscription required) provided this chart and reported:

Emails between White House aides and Attorney General Alberto Gonzales’s chief of staff show an orchestrated effort to fire several U.S. attorneys, counter to Mr. Gonzales’s previous assertions that the firings weren’t instigated by the White House.

Today, the Washington Post led with (bold and underlines are added by me):

The Justice Department advocated in early 2005 removing up to 20 percent of the nation’s U.S. attorneys whom it considered to be “underperforming” but retaining prosecutors who were “loyal Bushies,” according to e-mails released by Justice late yesterday.

The three e-mails also show that presidential adviser Karl Rove asked the White House counsel’s office in early January 2005 whether it planned to proceed with a proposal to fire all 93 federal prosecutors. Officials said yesterday that Rove was opposed to that idea but wanted to know whether Justice planned to carry it out.

The e-mails provide new details about the early decision-making that led to the firings of eight U.S. attorneys last year, indicating that Justice officials endorsed a larger number of firings than has been disclosed and that Rove expressed an early interest in the debate over the removals.

Setting aside the politics of all this, the press is using email to address two questions: who was involved, and are their public statements accurate? This is very similar to how I see email being used every day in the corporate world. Any legal proceeding or corporate investigation centers on understanding who knew what and when – and email is the place lawyers and investigators go to find that out.

Why? Because the beauty of email is that it is the source of truth, the indisputable statement of record. No need to ask people for incomplete recollections, no need to filter out the spin; just analyze their email and you will find out who did what – and perhaps even get a window into how they decided to do it.