e-discovery 2.0

As a long-time reader of The Economist, I was excited to find that this week’s edition writes at length about the exponential growth in electronically stored information (ESI), and how people are using technology to manage it.  I believe this is one of the most significant “mega-trends” impacting our economy, and I was thrilled to see it recognized by a mainstream publication. But when I read the 14-page special report, I was disappointed to find that its analysis of the legal and regulatory implications of “the data deluge” is really weak.

The survey does a good job of teeing up the issue:

The world contains an unimaginably vast amount of digital information which is getting ever vaster ever more rapidly. This makes it possible to do many things that previously could not be done: spot business trends, prevent diseases, combat crime and so on. Managed well, the data can be used to unlock new sources of economic value, provide fresh insights into science and hold governments to account.

It goes on to talk about how companies like Walmart, which has 2.5 petabytes of point-of-sale transaction data, is using business intelligence software to analyze the 1 million transactions it does every hour. By doing so, Walmart is able to improve the efficiency of its supply chain and the effectiveness of its marketing. The article also describes how companies like Amazon and Google use web analytics software on click stream data to improve their services.

What’s missing is an equally intelligent analysis of the legal and regulatory implications of all this data. The move from paper to ESI (email and files) has created a user-generated, written record of everything that happens in a company. That’s incredibly helpful when, after the fact, questions or disputes arise. Rather than relying on incomplete recollections, courts and regulators can now consult a written record – one where every document is time-stamped and very often attached to a person’s name via email. That enables judges and regulators to get better information which, in turn, leads to better decisions. It’s hard to quantify the value of that, but there’s no doubt it’s substantial.

There is, however, a catch. Because the volume of ESI is so great, it’s really expensive to gather, sift through, and then produce information. Add the requirement that the process needs to be defensible (i.e., easily explained in court), and the whole thing gets really expensive really fast. Hence the need for electronic discovery software: it’s the only cost-effective way for companies to manage their ESI from a legal and regulatory perspective.

That’s why I believe e-discovery software will be as big a category as web analytics software or business intelligence software – it’s a different side to the same coin. Or, more specifically, a different dimension to managing digital information stores which, as The Economist points out, are growing tenfold every five years.

Update: Nick Patience at The 451 Group has also posted on this topic, at almost exactly the same time as me.

Yesterday, I explained what I think Iron Mountain’s acquisition of Mimosa says about valuations in the archiving / e-discovery industry. Today, I will address the other questions that people commonly ask about the deal – why did Iron Mountain (Stratify) do it, and what does it mean for the electronic discovery industry?

In their letter to customers announcing the deal, Ramana Venkata (President of Iron Mountain Digital) and TM Ravi (CEO of Mimosa) point to two main benefits from combining the companies. On the archiving side, Iron Mountain can now offer Mimosa as an on-premise solution in addition to its existing hosted service. If it can integrate the two, then it can offer “location-independent” archiving which “will help you transparently and seamlessly move data between the on-premises data center and the cloud.” One additional benefit to Iron Mountain, which is not mentioned in the letter, is that it could even leverage Mimosa’s technology for its hosted offering, and replace Mimecast who it currently pays to provide this service.

On the e-discovery front, Iron Mountain now has a suite of 2 products and 1 service: Mimosa NearPoint for collection and preservation; the Stratify eVantage appliance for ECA (Early Case Assessment); and, Stratify Legal Discovery Services for review and production. This makes Iron Mountain a competitor to Autonomy, Clearwell, EMC/Kazeon, and everyone else listed in Gartner’s recent MarketScope covering e-discovery software companies. I’m sure the hope is that there’s synergy between the different products so that, for example, Mimosa’s experience in on-premise software will help Iron Mountain drive adoption of its new Stratify eVantage appliance behind the firewall.

Will the combination work? As Barry Murphy (a former Mimosa employee) points out in his excellent post on this topic, a lot depends on execution. But there are at least 2 reasons to be doubtful. First, the competition is far ahead, and will be hard to catch. As Barry, points out: “Iron Mountain will have a tough road ahead to compete with the likes of Autonomy, which bought successful archiving company Zantaz and has now had almost two years of development time for its hybrid on-premise/SaaS archiving offering.” The same is true on the e-discovery side, where companies like Clearwell have hundreds of corporate customers for on-premise ECA and review.

The second reason to doubt why the combined company will be any more successful than either were before the acquisition is that Mimosa and Iron Mountain Digital serve very different markets. Most of Mimosa’s customers are small to medium sized companies; most of Iron Mountain Digital (ie., Stratify)’s revenue comes from law firms. So it’s not obvious that by combining them you create a company well-suited to serving large corporations, which is the sweet spot for e-discovery and archiving.

It will be interesting to watch events unfold.

Lately, the electronic discovery blogosphere has been, well, a-twitter about twitter and other social media as they relate to electronic discovery. While twitter struggles to find a business model, enterprises and law firms are racing to understand the implications of this latest boomtown of user-generated content that’s being built in out on the frontier of the World Wide Web (or is that Wild Wild West?).

There’s talk of intellectual property being cast out, irrevocably, onto the Internet for all to see. Or slanderous things being uttered for which your company may be held liable. But, hold on a second: is there really anything new here? Anyone heard of e-mail? Web pages? Peer-to-peer? Google? Instant messaging? As Debra Logan astutely points out in her recent post on the topic, “everything that exists is discoverable (at least pretty much).” If you haven’t already, take a look at the FRCP’s definition of ESI and you’ll get her point. So, yes, it’s obviously important to have a common sense corporate policy around what’s appropriate and what’s not for the public Internet, but it shouldn’t be any different from the policy that you should have already had in place regarding blogs, web pages, and email.

What about the other side of the electronic discovery coin: finding information that’s responsive to a request? If anything, social media are more easily discoverable than just about any other form of user-generated content (though admittedly in some cases they can be more transient, which can post unique challenges). And, while it’s not universally true, the argument can be made that the more easily something can be discovered, the lower the cost and risk of that content to you. Worried if anyone on twitter is stealing your new idea for a router architecture? How about the top-secret approach to making coffee you were thinking about patenting? Well, if anyone twittered about it, tracking it down is a snap. Just keep in mind that because of the public nature of social media, it’s likely that the more important the information is to your company in the context of electronic discovery, the less likely it is to live out on the public Internet. Obviously, there will be exceptions. But when there are those exceptions, tracking down the relevant information will likely be a fairly straightforward and relatively inexpensive process.

However, before we dismiss social media as nothing new and something that can largely be addressed through already-existing policies and discovery techniques, let’s consider one aspect of social media that is on the upswing, but often out of the blogging limelight: enterprise applications.

Increasingly, companies are moving to advanced enterprise social media platforms such as Jive or SocialText as a way of improving internal collaboration and making projects run more smoothly and effectively. Because such enterprise platforms are often used on a company’s most important and strategic projects, having robust e-discovery capabilities to allow internal blog, wiki, and discussion content to be captured and placed into a format that can be seamlessly searched along with other more traditional documents is becoming critical to forward-thinking enterprises.

For example, I recently came across a large financial institution that uses Jive SBS as its wiki and Clearwell as its e-discovery solution. What surprised me is that this company has created its own Jive/Clearwell “adapter” that feeds Jive discussions directly into Clearwell as a conversation thread. This is just one example, but I’m sure more will follow. Over time, it will become a requirement for e-discovery platforms to integrate with enterprise social media products. And, rest assured, as that happens, we’ll be sure to tweet about it!

UPDATE: Whit Andrews of Gartner was kind enough point out his (prescient) research note on the subject of e-discovery and social networking from November, 2007. He points out that there is in fact a very important “new new thing” about social networks, which is that they may be able to be leveraged in an e-discovery context to find out more about the people relevant to an investigation. By tapping these publically-available sources of information, investigators may be able to gain better insight into private (i.e. enterprise) information stores to guide the e-discovery process. More detail on this and other insights can be found at http://www.gartner.com/DisplayDocument?id=543110&ref=g_forward&call=email.

E-Discovery search has become a hot topic of late (in blogs and in the news), and I think it’s pretty clear that the unwashed (attorney) masses still don’t really grok the importance of using a defensible search protocol.  Neither do they seem to understand the enhanced scrutiny that’s being applied by the judiciary.

Kipperman v. Onex Corp., 2008 WL 4372005 (N.D. Ga. Sept. 19, 2008) is another in what will assuredly be a long string of cases that demonstrate how easy it is for litigators to get wrapped around the axel of e-discovery search.  In Kipperman, the defendant (Onex) presented several motions to the court, including attempts to obtain relief from the need to produce email identified after searching several backup tapes.

During a previous hearing the court ordered Onex to search all the mailboxes on two tapes, as well as on an additional tape selected by Plaintiff. The court determined that despite Onex’s objections and representations, the backup tapes were “producing meaningful discoverable information.”  The court was nevertheless sympathetic to Onex’s burden and therefore weighed in with some guidance:

“The court did suggest, … , that Plaintiff be more artful with its search terms and that Plaintiff utilize a list of the people, provided by Defendants, to review whether all mailboxes needed to be searched.”

The court also gave Onex the chance to narrow the search terms.  Unfortunately, they didn’t seize the opportunity to provide a narrower list or a refinement of their search terms.  “As such, they agreed to search and restore all the mailboxes with the search terms provided by Plaintiff.”

Not surprisingly, Onex then sought relief from having to review and produce all of the results from the search because the “broad search terms resulted in thousands and thousands of irrelevant hits.”  For example, the search terms included the word “republic” which used to elicit emails regarding Republic Builders Products, one of the companies involved in this matter.

“Defendants claim that the search captured thousands of irrelevant pages due to one occurrence of the word ‘republic’ often related to Onex business interests having nothing to do with Magnatrax in the ‘Republic of France,’ ‘Republic of Ireland,’ and ‘Czech Republic’.”

Again the court reaffirmed their sympathy with Onex’s burden and yet denied the requested relief, in large part because Onex was warned about not being more “artful”:

“[T]he court is not unsympathetic to the massive amount of discovery involved in this matter, the considerable burden of working with it, and the overproduction that often comes with e-mail production. Therefore, the court gave Defendants numerous tools by which to reduce the burden of e-mail discovery, including an opportunity to limit Plaintiff’s search terms and an opportunity to provide a list by which the number of peoples and the number of boxes being searched could be reduced. Defendants did not take advantage of these opportunities. Defendants must now lie in the bed that they have made. Thus, Defendants’ objections on the basis of relevancy and volume are DENIED.” (emphasis added).

Needless to say, Kipperman is probably not all that atypical.  Attorneys everywhere have historically used blunt e-discovery search instruments and haven’t often run afoul of the judiciary.  Now, post Victor Stanley, et al, the playing field has changed dramatically.  It’s important to leverage best practices (from Sedona and others), craft a defensible search strategy, sample the results and “show your work.”  Missteps along the way, especially ones that the court has tried to help the parties avoid won’t be met with much tolerance

In my experience, e-discovery does not make the radar screen of most corporate General Counsels (GCs). Typically, it is one many issues left to others (e.g., Chief of Litigation, Director of Litigation Support) within the GC’s group. That may change after the recent verdict in the case of Broadcom vs. Qualcomm.

See below for the story, as told by Corporate Counsel in their October issue, with additional commentary from me [added in brackets]:

Collateral Damage

After a string of punishing legal defeats, Qualcomm Incorporated has switched general counsel. On August 13 the company announced that Carol Lam would replace Louis Lupin as its legal chief [Sounds like he got fired]. The move came a week after a federal judge issued a scorching order accusing Qualcomm and its outside lawyers of “gross litigation misconduct.” [Sounds like a pretty good reason why he got fired]

Emily Kilpatrick, Qualcomm’s director of corporate communications, says Lupin is leaving for personal reasons [Isn’t that what they always say?]. “He has been an outstanding leader and contributor to Qualcomm’s success over the past 12 years,” according to Kilpatrick. “However, he has decided to step down as general counsel and take a personal leave.” [a decision most likely made at the request of his boss]

Lam, who was hired in February to supervise Qualcomm’s worldwide litigation, will take over as interim GC, according to a company statement. Lam is one of the U.S. Attorneys fired by the U.S. Department of Justice this past winter. [oh, the irony…]

Based in San Diego, Qualcomm licenses semiconductor technology and system software to cell phone makers. For several years it’s been engaged in a pitched battle with rival Broadcom Corporation over who has infringed whose patents.

Qualcomm’s biggest problems have come in a case in San Diego federal district court. In January a jury ruled that the company had violated Broadcom’s patents. But even before the verdict, Qualcomm suffered a major setback as the trial drew to a close. One of the company’s witnesses revealed the existence of email that Broadcom said should have been produced during discovery. [Yet again, email is the smoking gun]

In April general counsel Lupin and one of Qualcomm’s outside attorneys sent letters of apology to the court, saying they failed to do a detailed enough keyword search of the company’s email. [No big deal, right? After all, we are saying sorry]

But that wasn’t enough for Judge Rudi Brewster, who has been hearing the San Diego case. On August 6 he issued a blistering 54-page ruling. He accused Qualcomm not only of failing to turn over more than 200,000 pages of relevant email and electronic documents during discovery, [i.e., this is a case of a deeply flawed e-discovery process, not of a simple missing email] but of engaging in a years-long campaign to deliberately mislead a technological standards body. Brewster ordered Qualcomm to pay Broadcomm’s litigation costs, and voided two of its patents. (David Rosmann, vice president of intellectual property litigation at Broadcom, estimates that its fees could be around $10 million). [The legal costs alone are several times what it would have cost Qualcomm to purchase an e-discovery solution and avoid this whole situation in the first place]

In a statement, Qualcomm said it “respectfully disagrees” with Brewster’s ruling and intends to appeal. “Qualcomm acknowledges the seriousness of the court’s findings and reiterates its previous apology to the court for the errors made during discovery and for the inaccurate testimony of certain of its witnesses,” the statement read. [We said sorry, isn’t that enough for you guys?]

The company’s problems aren’t over, however. Federal magistrate judge Barbara Major is now considering whether to levy sanctions against Qualcomm’s attorneys. [Don’t think you can hide behind your deep-pocketed employer. If you screw up e-discovery, it will be your neck on the line] Major has given “any and all…attorneys who signed discovery responses, signed pleadings and pretrial motions, and/or appeared at trial on behalf of Qualcomm” until September 21 to file a statement explaining why they shouldn’t be penalized. [For the lawyers in question, it’s guilty unless their arguments convince the judge they are innocent]

The other day, Michael Clark of EDDix sent me a fascinating academic paper (thanks, Michael!) about “information inflation” at its impact on the legal system. I had never really thought of it this way, but there have really only been 3 significant events in the evolution of information:

1. Writing (c. 5,000 years ago): Pre-historic man started to etch his markings on clay tablets, stone, wax, papyrus, bark, cloth, wood, paper, cave walls and anything else that came to hand.
2. Printing (c. 1450): Gutenberg’s movable type printing press enabled mass production of information, contributing to (among other things) the Renaissance and the Scientific Revolution.
3. Digitization (c. late 20th Century): The personal computer, wide area networks, internet, email, have all led to a massive explosion of information in the past 50 years. As the article points out, “close to 100 billion emails are sent daily…In a small business, whereas formerly there was usually 1 four-drawer file cabinet full of paper records, now there is the equivalent of 2,000 four-drawer file cabinets full of such records, all contained in a cubic foot or so in the form of electronically stored information.”

How can the legal profession cope, given that a lawyer’s job is often to synthesize this mind-boggling amount of data? Fortunately, the authors have a solution:

“A family of computer technology employing new types of search methods and techniques beyond use of mere keywords should now be considered for use in litigation….Litigators can no longer depend on manual review alone. It is too time-consuming and expensive – with cost often exceeding the amounts in dispute.”

To illustrate its point, the paper tells the story of the White House and the problem of a billion emails. During the Clinton administration, the White House agreed to a form of electronic record keeping called ARMS (Automated Records Management System). At the end of each administration, these records are handed over to the National Archives and Records Administration (NARA). The table below shows the number of stored emails NARA has, or expects to receive at the end of each administration.

Now assume that, like previous administrations, the Next President’s administration is subject to a lawsuit that requires e-discovery. The paper calculates:

“Without employing any automated computer process to generate potentially responsive documents, the review effort for this litigation would take 100 people, working 10 hours a day, 7 days a week, 52 weeks a year, over 54 years to complete. And the cost of such a review, at an assumed billing rate of $100/hour, would be $2 billion. Even, however, if present day search methods are used to initially reduce the email universe to 1% of its size (i.e., 10 million documents out of 1 billion), the case would still cost $20 million for a first pass review conducted by 100 people over 28 weeks, without accounting for any additional privilege review.”

This is a great example of why companies and government agencies are adopting e-discovery 2.0 technologies that go far beyond keyword search. In the face of information inflation, what choice do they have?

If there’s one idea that has captivated Silicon Valley in the past 3 years, it is Web 2.0. People may debate its meaning and definition, but the gist of it is clear: a handful of powerful forces have coalesced to make the internet of today fundamentally different to what it was 5 years ago. Opinions vary on which of these forces is most important: the growth of broadband to the home; open source, ajax and other technologies which lower the cost and increase the functionality of web applications; the power of community in a world where more people are on the web. Whichever you choose, there is no doubt that collectively these forces have had a huge impact, powering the growth of now-household names such as Google, MySpace, and YouTube.

I believe that an analogous set of changes is transforming the way companies do e-discovery. Ten years ago, e-discovery was an after-thought – a necessary, but incidental, part of corporate legal expenses. Today, it is a huge line-item in the legal budget, a headache for corporate IT, and the foundation upon which many cases are built.

E-discovery 1.0 was an ad hoc activity; e-discovery 2.0 is a core business process. E-discovery 1.0 was barely noticed; e-discovery 2.0 is driving the news cycle, affecting everyone from Intel to the US Attorney General. In the legal world, e-discovery 2.0 has had every bit as big an impact on enterprises as Web 2.0 has had on the dating lives of teenagers.

What happened? A series of fundamental changes have made e-discovery far more important, expensive, and complex than it was in the 1990s. Chief among these changes are:

1. Email, Not Voicemail: In the past 10 years, companies have switched from voicemail to email as the primary way they communicate. This has created a written record where none previously existed. Just as oral histories eventually die out, every voicemail eventually gets deleted; but emails and the written word live forever. Whatsmore, the convenience and time-efficiency of email makes it addictive, with the result that every meaningful conversation is captured, time-stamped, and attached to a person’s name. Given that many legal cases turn on intent, and proving who knew what when, this makes email a virtual treasure trove for anyone building a case.

2. Electronic Files, Not Paper: Electronic files are fundamentally different to paper documents: they reproduce like rabbits and are far cheaper to store. For example, one laptop is the equivalent of 2,000 boxes of paper; one server corresponds to 8,000-40,000 boxes of paper. The number of servers and laptops holding vast quantities of email is only increasing as the cost of hard disk storage falls, down from $2.04 per GB in 2004 to $0.77 per GB in 2006. Net net: going electronic has vastly increased the amount of data that must be analyzed as part of the discovery process.

3. Sooner, Not Later: Recent changes to the FRCP guidelines have moved e-discovery up in the process, forcing companies to have an e-discovery plan within 99 days of a suit being filed. Since disputes rarely settle that quickly, that means enterprises must now incur the expense of e-discovery on every case, not just the small number that actually make it to court. The result is a massive increase in e-discovery expenses and workload.

Anecdotal evidence of e-discovery 2.0 is everywhere. A few years back, no one would have guessed that every major analyst firm would have people dedicated to tracking e-discovery. Nor would you have expected to find a litigation support manager at every major enterprise.

So what exactly is e-discovery 2.0? Well, I will talk about that in a future post.

It is a truism to say that it is easier to dispense advice than to follow it, and with good reason. How many venture capital firms practice the financial discipline they preach to their portfolio companies? How many management consulting companies employ the innovative management theories they advocate to their clients? And how many technology companies actually leverage leading-edge technology to solve their own business problems?

The answer, at least based on my experience, is “not very many”. For example, if you look at Silicon Valley’s leading technology companies, the vast majority do not have an e-discovery solution in place. Yes, there are some exceptions but for the most part, when it comes to e-discovery, the likes of eBay, Google, Yahoo, and (until recently) Intel have preferred to muddle through with manual, error-prone, expensive processes.

The justifications are typically the same. Some technology companies argue that they don’t need a legal discovery solution because theirs is not a litigious industry; others say they delete everything off their Exchange servers within three weeks and so don’t have any email to discover; all agree that things like email and document retention policies are needlessly bureaucratic.

The danger of this “we- don’t- need- car- insurance- because- we- will- never- have- an- accident” approach has been brutally exposed in the past few weeks by the painful experience of Intel. In case you missed the press coverage: AMD sued Intel for anti-trust violations. Like any company on the receiving end of a subpoena, Intel was obliged to provide opposing counsel with all email and documents relevant to the case.

If Intel had an e-discovery solution, that would have been a straightforward process. Intel’s IT group would simply identify a group of messages by date range, person, and perhaps keyword within their larger email archive. The legal group would then use an analysis product to cull down the messages to only those relevant to the case. The whole thing would take a few days. But that’s not what happened. Since Intel did not have an e-discovery solution, the company had no simple way to preserve and analyze the relevant data. Intel’s legal department was obliged to inform over a thousand employees that they could no longer delete data at will. Somewhere along the line, the message did not get through and employees kept on deleting. As a result, Intel was forced to go back to the judge with the proverbial “the dog ate my homework” defense, while AMD cried foul.

How much this costs Intel is yet to be determined. But my guess is that they will end up spending more on lawyers to fix the mess than they would have spent on an e-discovery solution that would have avoided the problem to begin with.

While I have given up on venture capitalists and management consultants, I remain optimistic that the technology industry will practice what it preaches and leverage technology to solve its own business problems in e-discovery. As Intel discovered, it is not enough to have smart lawyers on staff. You also need to equip them with an e-discovery solution that allows them to preserve and analyze information relevant to the case.

To do otherwise is an open invitation to your competitors to sue you. Just ask Larry Ellison – or better yet, SAP.

I will let others better qualified than me comment on the political implications of the recent furore over the firing of several US Attorneys. But one interesting aspect of the story from my perspective has been seeing email front-and-center in the news cycle.

Everyone from CNN to the Washington Post to the Wall Street Journal has led with “email-driven” stories, with headlines like “Rove, Gonzales discussed firings, e-mails show”. On March 14, the Journal (subscription required) provided this chart and reported:

Emails between White House aides and Attorney General Alberto Gonzales’s chief of staff show an orchestrated effort to fire several U.S. attorneys, counter to Mr. Gonzales’s previous assertions that the firings weren’t instigated by the White House.

Today, the Washington Post led with (bold and underlines are added by me):

The Justice Department advocated in early 2005 removing up to 20 percent of the nation’s U.S. attorneys whom it considered to be “underperforming” but retaining prosecutors who were “loyal Bushies,” according to e-mails released by Justice late yesterday.

The three e-mails also show that presidential adviser Karl Rove asked the White House counsel’s office in early January 2005 whether it planned to proceed with a proposal to fire all 93 federal prosecutors. Officials said yesterday that Rove was opposed to that idea but wanted to know whether Justice planned to carry it out.

The e-mails provide new details about the early decision-making that led to the firings of eight U.S. attorneys last year, indicating that Justice officials endorsed a larger number of firings than has been disclosed and that Rove expressed an early interest in the debate over the removals.

Setting aside the politics of all this, the press is using email to address two questions: who was involved, and are their public statements accurate? This is very similar to how I see email being used every day in the corporate world. Any legal proceeding or corporate investigation centers on understanding who knew what and when – and email is the place lawyers and investigators go to find that out.

Why? Because the beauty of email is that it is the source of truth, the indisputable statement of record. No need to ask people for incomplete recollections, no need to filter out the spin; just analyze their email and you will find out who did what – and perhaps even get a window into how they decided to do it.