e-discovery 2.0

Almost two years ago I wrote one of my first blog posts entitled “Review-less E-Discovery Review.”  Despite the tongue twister of a title, the post posited that “there is a very real possibility that we’re on the cusp of computers taking over a significant e-discovery task for attorneys.” I’d like to take a look and see how much (if at all) my prognostications have materialized.

A cynic might think that this is the moment where E-Discovery 2.0 jumps the shark.  But no, this isn’t one of those sitcom episodes where they flashback to previous shows as an easy way to recycle content.  Instead, it seems useful to see how the legal market has evolved from a litigation workflow perspective, particularly with some vendors touting the benefits of review-less technologies like predictive coding.

In the original blog, I noted that there was a “scenario where a non-manual review methodology may make sense” (while importantly noting that “this approach is not without risk”).  Since my last post there has been the successful adoption of Evidence Rule 502,which makes this methodology (at least conceptually) safer.

But again (imagine dreamy flashback mode), here were the guidelines I previously proffered:

1. Large data set.  This may sound a bit obvious, but a non-manual approach is best suited for large, unwieldy data sets.  The corpus doesn’t need to be in the terabytes, but the data set should be evaluated in term of discovery processing costs and attorney review estimates.
2. Short Production Timelines.  Once the above calculations are conducted, the next step is to determine if a human based review could even conceivably be conducted in the given time frame.  In many instances, an eyes-on review process just won’t be feasible since there won’t be enough bodies to throw at the problem.
3. Next Gen “PAR” Tools.  In order to pull this “review-less” review process off, both safely and quickly, the responding party needs to have access to fast, robust processing, analysis and review (“PAR”) tools.  Certainly, it’s possible to have this scenario work with an e-discovery service provider, if they have the capability.
4. Relatively Small Amount in Controversy.  For the time being, this approach should not be considered for any “bet the company” litigation, nor anything with significant downside risk (governmental inquiries, punitive damages, class actions, 2nd requests, etc.).  Yet, for many standard commercial lawsuits, corporate investigations, HR claims, etc. this review-less approach may be worth considering.
5. Ability to Use a Clawback Provision.  Entering into a clawback provision with the opposition is mandatory in this methodology since the chances of an inadvertent production are statistically ever-present.  Yet, until Evidence Rule 502 is resolved, there will always be a risk that the clawback won’t be enforceable against 3rd parties.
6. Non-governmental Production.  Most information in governmental productions becomes part of the public record, meaning that a clawback isn’t going to be feasible.  Here, trade secret information, personally identifiably data and the like would be disastrous if pushed out into the public domain.

The goal of this post is to see if this dog is any more ready to hunt than it was two years ago.  The short answer (right now) appears to be: No.

We all know that litigators are both risk adverse and generally slow to adopt new technology approaches.  This is particularly true when there’s a perception that they won’t have insight into the technological black box behind automated coding/tagging decisions.  Litigators are understandably sensitive about the ability to prove up the reasonability of their search and review processes.  This “reasonableness” requirement lines up both with the Victor Stanley requirements and FRE 50(b), which eliminates the chance of a waiver only “if the holder of the privilege or work product protection took reasonable precautions to prevent disclosure.”

Given this ongoing hesitancy, the question remains shouldn’t we be seeing more movement in automated review than the glacial progress that’s been achieved to date, particularly with the known shortcomings of the eyes-on review process?  Most are familiar with the 1985 STAIRS study by Blair and Marion where the percentage of relevant documents lawyers thought they had found using Boolean Keyword searches was 75% – when the percentage they actually found was 20%.

But, despite the known deficiencies of eyes-on review it follows into the “go with the devil you know” mindset that often makes sense when dealing with judges and juries who aren’t likely to grok newer-fangled approaches.

In addition to these high-level, almost dogmatic challenges, there is one other tactical element I’d add to my previous list (of 6 factors).

7. All documents processed up-front (no rolling collection). I’ve heard some in the trenches e-discovery experts claim that they’ve never had a case that didn’t involve at least some level of incremental data collections.  Whether this is an overstatement is immaterial.  The fact is that a large number of e-discovery projects involve ESI that is collected (and then processed) in dribs and drabs.  This if often a good thing, largely attributable to the incremental (start slowly) nature of a well thought out e-discovery project where a smaller number of initial custodians are processed, then ECA is conducted and only then is the additional ESI added to the corpus.  This common methodology causes some significant heartburn for a review-less methodology since the ever changing nature of the corpus makes it difficult/impossible for a sample to be truly extensible to what will eventually be the entire data set.  For this reason, the review-less approach should be limited to where the entire corpus is collected and processed at once.

In sum, the seven foregoing factors appear to still be largely valid and create an environment where an automated, review-less methodology will only make sense in a relatively rare set of circumstances.  This may change in the future, but given the risk adverse DNA of most litigators I can’t imagine this tipping point happening any time soon.

Recently, I attended a conference in Washington DC with a large number of government agencies, including (I must confess) many Clearwell customers like the Department of Health and Human Services, the Department of Homeland Security, and the Veterans Administration. It will probably come as no surprise that, during our conversations, it became abundantly clear that they had substantial electronic discovery technology needs. Many were still reviewing PST files manually in Outlook; others were TIFFing millions of pages of documents prior to directly loading into a traditional review application for eyes-on review. That’s right, nary a trace of early case assessment, transparent search, or culling to be found.

Sadly, no news there. What was fascinating for us was the reaction to the latest release of the Clearwell E-Discovery Platform, Version 5.5. Version 5.5 contains significant new functionality, including dramatically increased performance and scalability along with a number of substantial processing, analysis, review, and production enhancements. But, in addition to these features, we have rolled out a set of e-discovery best practices templates designed to make it vastly easier for organizations to implement a formal e-discovery methodology that builds on the integrated nature of our platform. And it was the prospect of such a methodology, even more than the technology, that people were buzzing about at the summit.

Why? With all of the activity going on in the e-discovery space around product and technology innovation, there was some strong feedback that process and methodology may have gotten lost in the shuffle. And, if you think about it, it’s process and methodology that are likely to be most carefully assessed when the courts are considering the reasonableness (or lack thereof) of e-discovery for a case.

The importance of putting process and methodology front and center (along with a commitment to making the necessary organizational changes to make it happen) is not exactly a new concept. Ralph Losey has been talking about it for years over on his groundbreaking and irreverent e-discovery team blog, and it’s a frequent topic of keynote speakers on the e-discovery lecture circuit. However, like eating your vegetables or exercising, putting in place the right e-discovery process in an organization is something that people realize the benefit of, but still ignore.

This cannot continue, as the stakes are escalating. Take the recent case of Mt. Hawley Ins. Co. v. Felman Prod., Inc. Dean will dive into this case in much greater detail in an upcoming post, but it is very relevant to the methodology versus technology discussion in that it highlights how a methodology problem can cause a fateful technology problem to be overlooked. In this case, a lack of sufficient quality control processes caused the plaintiff to inadvertently produce a number of privileged emails. The court found the inadvertent production was not “solely attributable” to a problem with a Concordance index, and that the plaintiff “failed to perform critical quality control sampling” to determine whether the production was appropriate. Privilege was waived.

What’s the solution? We believe that we’re on to something with Clearwell 5.5, in that we can, uniquely among e-discovery products, marry together methodology and technology in a single platform that allows for the entire e-discovery process to be documented and defended, end-to-end. We have particularly focused on the most critical part of the process which seems to come up over and over again in sanction and privilege waiver decisions, which is the way that an organization moves from an initial pool of documents to a set of defensibly-culled, potentially responsive documents, on through to tagging and production. Our unique workflow capabilities allow the entire process to be documented and instantly recalled with the click of a mouse, letting you see each decision that was made during the course of the case in a step-by-step fashion, and then to structure additional quality control audits on top of those decisions to ensure that every “i” is dotted and every “t” crossed.

It’s a good thing for everyone involved in litigation that e-discovery technology is maturing rapidly to the point where it can start to help solve these sorts of process problems rather than being the cause of them (as was the unfortunately case in Mt. Hawley). This is a major focus for us at Clearwell and you’ll see a lot more exciting news from us on this front over the next few months, so stay tuned!

Yesterday, Marsh & McLennan (M&M) announced the sale of Kroll, its investigative services division which last year generated $678 million in revenue. Kroll is being acquired by Altegrity, another investigative services company which is owned by Providence Equity. The acquisition price is $1.13 billion, below the $1.3 billion M&M was rumored to be asking, and the deal is financed by Apollo Investment Services and Goldman Sachs.

There are many aspects to this transaction, but I want to focus on just one: what does this mean for Kroll Ontrack, Kroll’s largest division with $250 million in revenue and a staggering 1,500 employees, making it by far the world’s largest e-discovery service provider?

To answer this question, I will first outline the strategic challenge facing Kroll Ontrack, before outlining two alternative strategies its new owners may adopt for addressing it.

Strategic Challenge: Kroll Ontrack Is The “Yahoo! Of E-Discovery”

Just as Yahoo was an internet pioneer in the 1990s, Kroll Ontrack was the pioneer of electronic discovery services. Like all pioneers, as the first to market, Kroll had to build everything itself. So Kroll Ontrack invested not only in recruiting and training its staff of skilled consultants, the company also developed its own suite of e-discovery tools and software. It offered this integrated package of services and software to the market and, justifiably, charged a price premium.

But as the industry matured, it disaggregated with more savvy customers and new companies focused on specific parts of the value chain. Customers became better educated and more confident making decisions, diminishing the value of Kroll’s “we-are-the-safe-choice” value proposition. These customers today have many more options for e-discovery than was the case in years gone by, primarily because of a generation of e-discovery software companies, such as Clearwell, Guidance, Exterro, and kCura/Relativity, which offer capabilities like collection, ECA (Early Case Assessment), litigation hold management, and linear review. These have been widely adopted by Kroll Ontrack’s competitors, negating Kroll’s technological advantage. Even worse, because Kroll Ontrack’s competitors do not need to invest in R&D, they have a substantially lower cost structure. As a result, they have undercut Kroll Ontrack on price, which has halted its growth and squeezed its margins.

In a directly analogous way, Yahoo! has seen its broad internet service to consumers eroded by a host of more focused competitors such as Google, Facebook, and Skype. Consumers today are much more familiar with the internet, and feel comfortable making separate choices for search, social networking, and messaging, without the need for an umbrella brand. That has left Yahoo! without a reason for being: even today, its CEO struggles to answer the fundamental question “what is Yahoo!?”

Solution: Sell It Or Fix It

As Kroll Ontrack’s new owner, Altegrity has a simple choice. It could sell Kroll Ontrack, making the strategic challenge someone else’s problem; or Altegrity could fix it, by adopting a fundamentally different strategy.

Let’s consider each in turn:

Sell It: Most sensible people would find it funny to think about selling something right after you bought it. But in this case, it could make a lot of sense. Altegrity is a leading provider of investigative services, not e-discovery, making the “non-Ontrack” part of Kroll’s business a much better fit. So why not sell Kroll Ontrack, pay down debt, and focus on the services business which it understands? This would be especially attractive if, as Vivian Tero at IDC suggests, there are willing buyers such as ECM or storage software companies which like Kroll Ontrack but do not want the services business.

Fix It: Mike Cherkasky, Altegrity’s CEO, is a former head of Kroll, and so is perhaps uniquely well placed to bring about a change in direction. To do so, he must decide what Kroll Ontrack wants to be. If its goal is to be the leading e-discovery service provider, then it should kill its internal software development efforts and focus on providing customers the absolute best service using industry leading tools. If it wants to be an e-discovery software company, which would be a much harder transition, then it needs to exit the services business and make its technology available every litigation support company.

Either way, it will take time and a lot of painful decisions for Kroll Ontrack to recover its momentum. But if any encouragement is needed, the Altegrity and the Kroll Ontrack teams need only look at what’s happening to Fios, another of the industry’s early pioneers. So far, Fios has refused to decide what it wants to be, abandoning its internal review platform for Relativity but keeping its proprietary processing software. The result? It’s had three different CEOs in the past 12 months, and competitors continue to steal market share.

Every car dealer knows he should focus customers on the monthly payment, not the total cost of the car. Every credit card solicitation (or sub-prime mortgage, for that matter) starts with the offer of 0% interest, not the actual interest rate or fees the customer will pay after the first 6 months. The reason is simple: once you lease the car or put a balance on the credit card, it’s very hard to switch away when – as often happens – you find yourself paying much more than you should later on.

I was reminded of these examples when reading about Kroll Ontrack’s offer of “free ECA” and Stratify’s recent press release announcing “free early stage filtering” for electronic discovery. Taking each in turn:

Kroll Ontrack Advanceview

Based on feedback from several customers in Washington DC, New York, and the Mid-West, Kroll Ontrack often provides Advanceview at no charge. That means customers can get “custodian de-duplication” and “1 keyword and date filter pass” for free, although Kroll still charges $200-250/hour for doing the work. The resulting data set is then processed and loaded into its review platform for $1,500-$1,800 per gigabyte.

Is this a good deal? For the vast majority of customers, the answer is “no” for three reasons.

First, customers typically end up paying more than they would using alternative products. For example, in the chart below, we compare the cost of using Kroll Ontrack to that of Clearwell for a 100 gigabyte project. In both cases, we assume customers are doing de-duplication, filtering, keyword searching, first pass review, and load file creation. As with any comparison of this sort, you have to make some simplifying assumptions. For example, we excluded data hosting fees and professional services fees from the analysis.

Whether customers are better off with Kroll depends entirely on how much data is culled out for free before customers incur the high, back-end charges. Given that all Kroll is doing for free is custodian de-duplication and running one set of keywords and date filters, the typical cull rate is likely be anywhere from 20% to 50% — nowhere near the 80% cull rate required for Kroll to be more cost effective than Clearwell.

The second reason why this is not a good deal is that it gives customers no certainty about costs. Culling rates from de-duplication and blind keyword searches are unpredictable and vary widely, meaning that some projects will cost more than expected while others will cost less. But every project has budget that’s determined up front and, as any litigation support manager will tell you, you get much less credit for being under budget than you get pain for going over budget. That’s why cost certainty is one of the leading requests from anyone involved in electronic discovery.

Finally, excluding data based on a single round of keyword searches and date filters is not in line with The Sedona Conference best practices. Rather, Sedona recommends that customers iterate their keywords and culling strategies to hone them appropriately.

Iron Mountain Stratify OnPoint

It is not yet possible to do the same detailed analysis on Stratify’s OnPoint which offers “free early stage filtering”, because it’s impossible to tell exactly what that means. In its artfully-worded press release and data sheet, Stratify promises to provide “free processing and loading of unlimited data for early stage filtering”. Does that include de-duplication? Does that include any keyword searching? My guess is “no”, in which case all they are really doing for free is offering to load data into their review platform so that they can then charge you – not a very compelling offer. But if anyone does know the answer to these questions, or if Stratify would like to clarify exactly what’s being offered for free, then please let me know and I’ll post an update.

Once data is in Stratify’s system, it charges a “one-time fee starting at $500 per gigabyte” for “reviewable data”. But it does not say if that’s the only fee. What about monthly hosting charges? Fees for additional reviewers? Again, it’s not yet clear what the downstream cost of review really is using Stratify, so it’s impossible to know whether this is a good deal.

If there’s one lesson from all of this, it’s “buyer beware”. Just as when you buy a car, sign up for a credit card, or click on that offer to get more corn on Farmville, you need to look beyond the “free offer” and understand what it’s really going to cost you.

To me, litigation holds in electronic discovery are like Federal sentencing.  Most Federal criminal cases end in a plea agreement, so very few are ever tried in front of a jury, but sentencing must occur in 100% of the cases.  Preservation is very similar – although you are likely to collect and produce in only a subset of cases, you must preserve in 100% of the cases.  So, just as sentencing law is extremely important to the Federal criminal bar, preservation is one of the most important phases of e-discovery to corporate or litigation counsel.  Naturally, I am surprised when I talk to corporate law departments that do not have a documented, repeatable process for legal holds that also includes standard procedures for IT.  Simply emailing a hold notice to a custodian and recording the hold on a spreadsheet is not enough given Judge Shira Sheindlin’s recent opinion in Pension Committee v. Banc of America Securities.

So, what is a defensible litigation hold strategy, and how do you get it done?  It is important to remember that the ultimate goal is preservation of any and all evidence that may be relevant to the matter at hand.  There are no hard and fast rules or checklists – only opinions from the bench of what is not enough.  The keys to defensibility are consistency, standardization, documentation, and diligence.  The place to start is with standardized policies and procedures as well as a person (or a portion of a resource) designated within the enterprise as the single point of contact for all issues related to preservation of evidence (read: project management).  Technology can help, but a great preservation strategy can be developed without an expensive or vast technology investment.  So, how do you get this done?  Here are four steps you should take now:

I. Define the Process (You cannot execute a plan that does not exist):

The first step to creating a great litigation hold strategy is defining a process or set of procedures that can be followed repeatedly.  This process could include setting up tiers for ESI sources, with tier one sources always being preserved (these might include frequently requested sources like email, file servers, and perhaps computers of key custodians), tier two sources being preserved on a case by case basis (including sources that are more industry-specific to the enterprise and therefore not always relevant in routine employment litigation or contract disputes), and finally tier three sources which include everything else.  By tiering ESI sources, the enterprise can focus on developing detailed preservation protocols for the tier one and tier two sources while developing a less detailed protocol for tier three sources which would include simply naming IT subject matter experts who would be consulted in the event that those sources were ever subject to preservation requirements.  This approach would allow the enterprise to prioritize data mapping and cataloging efforts on the tier one and tier two sources first.  Data mapping can be an enormous undertaking, so the enterprise might do well to know as much as it can about the email system rather than spending critical time focusing on ESI that may never be subject to any preservation requirement.

II. Dedicate Resources Appropriately and Engage Them Early:

Personnel are a key concern for preservation.  Allowing too many people to initiate and manage litigation holds could lead to confusion for IT and will certainly yield different results across multiple matters.  A single person or team should be appointed to manage litigation holds for the enterprise, and that person or team should develop a sound relationship with IT, paying particular attention to the subject matter experts who manage tier one data sources.  By forging the relationship early, communication and expectations will align before mistakes are made or spoliation occurs.  Of course, consistency is a key component of defensibility, and this cannot be achieved with a reactive, undocumented approach.  IT must understand the scope and consequences of spoliation, and they must be alerted early enough to halt routine destruction of ESI from the company’s systems.  Most corporate law departments have policies in place for “opening files” when new matters occur; it would be relatively easy to include a step in this procedure for engaging the appropriate personnel to begin preserving ESI at this early stage.  Leaving it up to individual attorneys or paralegals could cause preservation procedures to get implemented at various stages in the litigation, exposing the corporation to unpredictable and varying degrees of liability.

III. Audit, Log, and then Audit the Log:

Auditing the process is also critical to defensibility.  When ESI is moved, the process must be documented (even retaining shipping receipts for ESI that must be transported), and all conversions or alterations of the ESI must be documented (preferably with a procedure for commonly-encountered issues and then in a log as performed).  Log every step along the preservation path – including correspondence with custodians, when litigation hold reminders are sent, and any action performed on the preserved ESI (throughout the duration of the litigation).

IV. Talk to Your Custodians!

Interviews with custodians can go a long way toward achieving the diligence element of defensibility.  Acknowledgment by the custodian that they understand the preservation requirement being placed upon them and that they agree to preserve ESI is critical.  Often, custodians can also provide information about additional custodians who should be contacted or ESI sources that may not have been considered.  These and other matter-specific questions should be asked of the key custodians at a minimum.  This information may also prove useful when developing keywords for early case assessment or meet and confer preparation.

As you may have noticed, I have not mentioned any technology yet.  That is because a sound preservation strategy begins with sound policy, not simply purchasing a technology solution that purports to solve all of your problems.  Invest time in analyzing your litigation history, open matters, ESI sources, and personnel.  From there, processes can be developed, and where there are processes, there are then opportunities for automation and optimization through technology.  Simply defining the process is not enough – it must be repeatable and implemented consistently.  One of the best ways to ensure that a process is repeated is to automate it, removing the potentially error-prone manual work  where it is not required.

Following these steps should help jumpstart the development of a defensible preservation strategy, and once developed, you may be surprised to find that existing technology can help automate many of the procedures you have created.  Where existing technology investments fall short, venture into the market place informed and with the keys to defensibility in mind: consistency, standardization, documentation, and diligence.

Since I’ve finished off the last of the Halloween candy and tossed out the moldy, squirrel ravaged pumpkins, it occurred to me that now might be a good time to think about what 2010 will hold for the electronic discovery industry.  My 2009 list seems to have been fairly prescient and many of those notions still hold true since the legal industry (as we know) doesn’t move at the most blistering pace.

Again, doing my best Nostradamus impersonation, here are my top ten trends for 2010:

1. Early case assessment (ECA) moves from a “nice to have” to a “must have” requirement for any matter involving electronically stored information (ESI).  In 2009, we saw ECA move into the mainstream as a methodology to quickly understand case facts, assess risk and lower both review and data processing costs.  But, in 2010, with the advancement of the tools and the increased socialization within the bar and the litigation support community, ECA will graduate into a core methodology for savvy litigators regardless of matter type or size.
2. Appetites for broad information lifecycle management initiatives diminish as organizations realize these programs are far too complex to solve specific pain points, and they often take too much time (measured in years) to execute.  The economic reality is that these holistic, cross data, cross enterprise pipe dreams really can’t demonstrate the ROI that’s needed in today’s challenging economy.
3. Staffing roles continue to evolve with a newfound focus on project management. The role of an in-house e-discovery coordinator will emerge as more of a project management and analyst versus pure legal or IT. This shift will become increasingly necessary as e-discovery evolves from an ad-hoc fire drill to a standard business process that is repeatable, measurable, and defensible.
4. Data analytics and statistical methodologies gain traction to augment the type of subjective decision making approaches that have historically formed the backbone of the e-discovery search and review processes.  These objective methodologies have long been called on as best practices by the likes of the Sedona Working Group. In 2010, they now will start to move from theoretical to practical task as e-discovery tools increasingly move in-house and departments enhance defensibility and add elements such as sampling into the workflow.
5. Platform e-discovery solutions finally become a reality as customers finally graduate from painfully stitching point solutions together, thus requiring less physical document hand-offs (i.e., exports and imports) between applications, cutting costs and lowering the risk of data loss.
6. Associate-based review gradually goes extinct, as both clients and law firms tire of expensive, linear review processes.  More review work becomes either insourced or is managed with specialized contract attorneys, who are both cheaper and better trained for this type of work.
7. Similarly, FRE 502 and “clawback” agreements will be increasingly used to reduce the need for any manual, eyes-on review, although many litigators will resist this trend because of the fears of “un-ringing the bell” when privileged information is disclosed in any context.
8. While perhaps anathema, alternatives to the much lauded EDRM model will gain traction, as practitioners strive to find an even better, and perhaps more practical, project management framework, in many cases acknowledging the role that the EDRM has taken in forming *the* lingua franca of the e-discovery industry.
9. The push for cooperation in the e-discovery process, will make incremental progress despite reticence by old school litigators.  Increasingly, this type of cooperation, as strongly advocated by the Sedona Working Group, will be ironically forced by judges and local rules.
10. “Cloud” computing starts to really impact how e-discovery data preservation/collection is done, both in terms of social media and traditional ESI.  More and more companies block social media applications and file types in the workplace because of fears surrounding the inability to preserve and collect.

Recently, I moderated a panel at the Masters Conference with John Loveland, Sonya Thornton, and Bruce Markowitz entitled: How Defensible is Your E-Discovery Process? (Click here to read a summary of the panel.) It was well attended, and I think that the draw (aside from the esteemed panel) was that this topic still remains very vexing for most practitioners.

Initially, we started at ground zero with the notion that defensibility is in most instances equated with the “reasonableness” standard, which is pervasive across many areas of the EDRM spectrum… from preservation to production.  Instances include:

• Preservation — “[a]s soon as a potential claim is . . . identified, a party is under a duty to preserve evidence which it knows, or reasonably should know, is relevant to the future litigation.”
• FRE 502 (b) – the disclosure does not operate as a waiver in a Federal or State proceeding if the (2) the holder of the privilege or protection took reasonable steps to prevent disclosure;
• General Privilege Waiver — In SEC v. Badian, 2009 WL 222783 (S.D.N.Y. Jan. 26, 2009)(link), “there is no basis … to conclude that there were precautions [to prevent the disclosure], let alone whether they were reasonable.”
• FRCP 37(e) — Absent exceptional circumstances, a court may not impose sanctions under these rules on a party for failing to provide electronically stored information lost as a result of the routine, good-faith operation of an electronic information system.

While the foregoing isn’t exhaustive it does highlight the persistent nature of the reasonableness standard as practitioners seek a defensibility sanctuary.  The good news is that the law doesn’t require perfection and there are also a number of ways to obtain reasonable defensibility:

• Demonstrable acceptance by the opposition – here the notion is that collaboration with the opposition allows the parties to comfortably move ahead with their discovery process and even if it’s not objectively reasonable, the parties consent to the protocol will in most instances carry an imprimatur of reasonableness.
• Auditing / process transparency.  Similar to the first bullet, auditing the process and giving the opposition visibility into the process steps will often make it hard for them to lodge successful downstream challenges.
• Adherence to Local Rules (See 7th Circuit Pilot Program) or judicial order.  Another avenue than can provide some degree of safety is compliance with a discovery protocol mandated by local rules, although that compliance may ultimately be challenged.
• Statistical confidence intervals / sampling – the use of statistics as a way to bolster process defensibility is starting to come to maturity and in the future I think that detailed precision, recall and other statistical indicates will play a large role in e-discovery defensibility.

None of these steps can be guaranteed to really get you off the hook from a rapid opposing party calling foul, but using them in a “belt and suspenders” fashion will certainly help buttress any discovery process.

For more illumination on the topic please see the following video of my interview with John Loveland, who’s waxing poetically about discovery defensibility.

As most electronic discovery veterans are aware, the EDRM Project is an effort founded five years ago by George Socha and Tom Gelbmann to bring together a community of e-discovery practitioners for the purpose of solving some of the industry’s most challenging problems.

It may be hard to believe, but there was time in the very recent past where the iconic EDRM model did not yet exist. No multicolored boxes, no arrows, no sloping volume and relevance lines — nothing. Coming up with a standard way of talking about electronic discovery was the first problem that the group set about solving, and I think it would be hard to argue with the fact that they came up with the gold standard: a simple, clear, concise model that, at least so far, is standing the test of time as a way of thinking about the flow of the e-discovery process.

With each passing year, the group has started to address a broader set of problems, all with a practical bent.  Currently, there are eight:

This year’s annual EDRM conference took place back in May. After years of meeting in the same chilly and wind-swept location in downtown St. Paul, Minnesota, George and Tom had the brilliant idea of spicing up the meeting a bit by moving it to a more exotic locale: Bora Bora! Plans were set in motion, but quickly the overwhelming feedback came back from EDRM members: E-discovery is so fascinating, so heart-warming, that adding Bora Bora to the mix would simply be too much for the vast majority of the participants to bear. So St. Paul it was!

This was Clearwell’s third EDRM conference, and location aside, it’s been fascinating to see how it has changed over the last few years. Here are several notable trends from this year’s kickoff:

• More participation from end-users: There was a definite increase in the number of end-user/consumer participants (that is, those not from the vendor community), particularly from law firms. This could be taken as further evidence that e-discovery is indeed moving in-house.
• Increased enthusiasm to take on new challenges: One of the great things about EDRM is its willingness to try to tackle new areas that aren’t being directly addressed by some of the other (fantastic) organizations out there like Sedona. This was in evidence several years ago, when Clearwell was fortunate to get involved in the early stages of the EDRM XML project, which has proven to be a huge time, cost, and risk reducer for many in the industry by providing a common standard that can be used to move data within the e-discovery process. It was in evidence last year when Clearwell’s CTO was able to help launch a new effort around Search that is seeking to develop standards and best practices in an increasingly complex and contentious area. And, finally, it was in evidence this year with the launch of the Information Management project, a cutting-edge group that is exploring how to solve the challenges that e-discovery poses for information management – certainly a complex area in need of thought leadership.

• Improved collaboration: One thing that has amazed us from day one is how collaborative EDRM is, and continues to become. There are a lot of e-discovery vendors involved who, outside of the confines of the St. Paul Hotel, aggressively compete in the marketplace. However, George and Tom have been able to create an environment at EDRM where competitive spirits are set aside and ideas can be cultivated which provide huge value across the e-discovery landscape (both vendor and consumer).

One final note: If you’re an e-discovery practitioner in a law firm or corporate setting, I’d encourage you to get connected, either informally (through the EDRM web site) or formally (by signing up for one or more of the projects). While end-user involvement continues to grow, there is definitely still a need for more non-vendor involvement. It is critical in ensuring real and relevant problems get solved, and to pushing the state of the art in e-discovery forward. Please join us!

Over the past 4 years, I have had hundreds of conversations with corporate counsel and “legal IT”, meaning technical folks charged with supporting the legal team. More and more of them are looking to lower their costs by bringing e-discovery in-house. But as they work through that process, there’s one question that consistently comes up, even today – namely, “When [insert name of software company] says they “do” e-discovery, what exactly does that mean?”

There has been progress towards answering this question, thanks mainly to the analyst community. George Socha and Tom Gelbmann’s EDRM framework has been immensely helpful in breaking down electronic discovery into its component steps. Other analysts, like Debra Logan at Gartner, were quick to embrace the framework, prompting every software provider to follow suit. As a result, there is today a common language that everyone uses to describe the e-discovery process.

The Electronic Discovery Reference Model (EDRM) breaks down the e-discovery process into a series of steps. Companies looking to buy e-discovery software to lower costs typically map different software products to each of these steps, to make sure that they cover the entire process.

But having a universally-agreed framework is only half the answer. To eliminate customer confusion, there also needs to be agreement on how different software products fit into the framework. This is especially important since there is no single, end-to-end solution for e-discovery which covers all aspects of EDRM. So customers are forced to think about how different software solutions fit together. And that is where things begin to fall apart.

Many software vendors feel it is advantageous to claim that they do everything, even though they do not. Customers are rightly suspicious of those claims, and so press vendors to provide more detailed information – hence the question, “when you say you do e-discovery, what exactly does that mean?”

In light of that, how can litigation support teams, corporate counsel, or legal IT people figure out which e-discovery solution best meets their needs? From observing this decision-making process hundreds of times, I have found 3 simple steps are incredibly helpful.

Step 1: Read the analyst reports

Two reports in particular make for required reading. One is Gartner’s MarketScope Report, which is available for free at certain sites; the other is the 451Group’s recent e-discovery report, which is summarized in a publicly available presentation. The helpful thing about the 451 Group’s report is that it tells you which software companies do which parts of the EDRM process. You do have to buy the report to get the full picture (it’s well worth it!), but the publicly available presentation will give you a flavor for their analyis, and I have drawn from that presentation in the figure below:

Analyst firms like the 451 Group map software vendors to the EDRM framework according to what they actually do, which is often different from what software vendors claim they do.

The 451 Group’s analysis highlights several important points. First, it shows that there is no single end-to-end solution. Even the products of giants like EMC (SourceOne), HP (IAP), and IBM (CommonStore) only solve one piece of the puzzle, information management. Second, it shows that customers have choices at each stage of the EDRM process. For example, to solve the problem of identification, collection, and preservation of electronic information, customers can choose from solutions as diverse as Guidance EnCase (forensic collection), Index Engines (back-up tapes) and Mimosa NearPoint (email archive). Third, it provides an independent assessment of what vendors do, as opposed to what they may claim. For example, Kazeon claims analysis and review capabilities, whereas the report shows its product does identification, collection, and preservation; Recommind claims its Axcelerate eDiscovery and MindServer products do processing, whereas the report finds that they do not.

Step 2: Evaluate the products prior to purchase

Just as anyone would test-drive a car prior to purchase, it’s critical to test-drive e-discovery software. Any vendor should be willing to provide their software free of charge for an evaluation on-premise. The most effective evaluations are when the customer uses the product themselves, either on a live case or test data. This is far preferable to just sending the data to the vendor who then loads it into their system, as in that scenario there are too many opportunities for the vendor to hide their product’s shortcomings.

Step 3: Check references carefully

The trick with references is to insist on relevant references. It’s not good enough for the vendor to dredge up some random person who says nice things; or even a credible knowledgeable person who is using the product in a completely different way. For example, if a company is happy with Autonomy’s IDOL for enterprise search, that does not tell you much about what Autonomy might be like for e-discovery. What really counts are references from other customers who are using the product for the same application that you are.

All this can sound like a lot of work, but I have seen people go through the process in as little as a month, and be much happier for it. A little work up front can save a lot of time (and heart-ache!) later on.

When we first started brainstorming about our “Five Questions” feature a few months back, Tom Gelbmann was on our short list of “must-have” interviewees. As most readers are probably aware, Tom Gelbmann, together with George Socha, founded the Electronic Discovery Reference Model (EDRM) project and, also with George, directs the annual Socha-Gelbmann Electronic Discovery Survey. Since 1993 he has been managing director of Gelbmann and Associates, a consultancy based in Saint Paul, Minnesota.

1. There’s been a lot of buzz (particularly within the vendor community!) about the changes to the Socha-Gelbmann Electronic Discovery Survey. You and George have written about this a bit before, but we’ll try to take a slightly different angle: Beyond concerns about people taking rankings out of context or as an excuse for not doing due diligence, were there specific trends and drivers among end-users of e-discovery products and services that motivated the change?

The most important reason for killing the rankings was that they were too often interpreted by end-users as being a “one size fits all” evaluation of top product and vendors, and we thought this type of decision-making was becoming increasingly dangerous as electronic discovery was increasingly in its visibility and importance. It was similar to a prospective car buyer focusing in on the cars that earned the Car & Driver Car of the Year award. They may not need a BMW 3-Series or a Porsche Boxster if they need something to haul lumber or transport the kids to hockey practice. We also saw the opportunity to deliver something that could be useful in identifying software and service providers that meet a set of requirements based on the EDRM model. The result of the overhaul of the Survey we are currently working on will be the capability to specify requirements, dial up/down weighting of criteria and see which providers fit those parameters. Our objective is to deliver something more useful than the rankings.

2. Your other main project is, of course, the Electronic Discovery Reference Model (EDRM). How did EDRM get started? Was it you and George scribbling on the back of a napkin in a coffee shop?

It wasn’t quite on the back of a napkin, but close – it was an Etch-a-Sketch®! Actually, the idea came out of the 2004 Socha-Gelbmann Electronic Discovery Survey. After the dust settled in the fall of 2004, we took a step back to look at what we learned. It wasn’t difficult to spot the #1 issue concerning consumers and providers: confusion and frustration over the lack of standards in the electronic discovery industry. Many people told us this was the Wild West and something needed to be done. That something was a standards initiative. The reference model approach came from looking at the tremendous value the Open Systems Interconnection Reference Model had on computer communications. Developed in the late 1970s, this model provided a common structure for development of products. We thought a similar approach would work for electronic discovery. We started talking to providers and consumers about this idea and initially thought we may get 15 or so organizations interested… 20 tops. We now list the EDRM alumni of 500+ individuals from 140+ organizations.

3. EDRM will be celebrating its 5th year at the upcoming May meeting in Saint Paul. What’s surprised you most about the first five years of the Project?

The biggest surprise by far is that there was a second year, to say nothing of a fifth year that we are now planning. We started out with the objective of keeping the project to a single year, thinking that having a hard deadline would motivate everyone to complete the project on time. We also knew that placing all content in the public domain would attract interest and participation in this initiative that would help the entire industry.

4. The global recession is having a tremendous impact on the electronic discovery community. Any thoughts on how the landscape will be reshaped coming out of the storm?

Potential influencers keep surfacing, making it difficult to anticipate what is around the corner. Certainly, we have seen and will continue to see consolidation of providers. Acquisitions will likely continue as will dissolutions. It is clear that everyone is tightening the belt. Some corporations and law firms who had begun to expand internal electronic discovery operations will sharply curtail or stop the expansion. Experienced, highly capable people will be out of a job and looking for new opportunities, which means short term turmoil for these folks, but hopefully long term success within well managed organizations. The volume of work doesn’t seem to be ebbing yet, so there is great opportunity for the survivors.

5. You and George Socha are the “Dynamic Duo” of electronic discovery and have been working together for a long time.  Anything you want to reveal about George that might not be generally known?  Hidden talents? Secret ambitions?

Interesting question. George is a man of many talents. One pleasant surprise to me was learning of his expertise in baking. I thought I was good at making good pie crusts, after a dozen years making Thanksgiving pies with my daughters. My crusts have gotten rave reviews from the family. But George is the real expert. His creations are a work of art. When it comes to secret ambitions, I think he would like to assemble a massively parallel computer complex in his home office. He has a good start with a half dozen monitors, several computers, servers, etc. and shows no signs of slowing down.