e-discovery 2.0

Recently, I moderated a panel at the Masters Conference with John Loveland, Sonya Thornton, and Bruce Markowitz entitled: How Defensible is Your E-Discovery Process? (Click here to read a summary of the panel.) It was well attended, and I think that the draw (aside from the esteemed panel) was that this topic still remains very vexing for most practitioners.

Initially, we started at ground zero with the notion that defensibility is in most instances equated with the “reasonableness” standard, which is pervasive across many areas of the EDRM spectrum… from preservation to production.  Instances include:

• Preservation — “[a]s soon as a potential claim is . . . identified, a party is under a duty to preserve evidence which it knows, or reasonably should know, is relevant to the future litigation.”
• FRE 502 (b) – the disclosure does not operate as a waiver in a Federal or State proceeding if the (2) the holder of the privilege or protection took reasonable steps to prevent disclosure;
• General Privilege Waiver — In SEC v. Badian, 2009 WL 222783 (S.D.N.Y. Jan. 26, 2009)(link), “there is no basis … to conclude that there were precautions [to prevent the disclosure], let alone whether they were reasonable.”
• FRCP 37(e) — Absent exceptional circumstances, a court may not impose sanctions under these rules on a party for failing to provide electronically stored information lost as a result of the routine, good-faith operation of an electronic information system.

While the foregoing isn’t exhaustive it does highlight the persistent nature of the reasonableness standard as practitioners seek a defensibility sanctuary.  The good news is that the law doesn’t require perfection and there are also a number of ways to obtain reasonable defensibility:

• Demonstrable acceptance by the opposition – here the notion is that collaboration with the opposition allows the parties to comfortably move ahead with their discovery process and even if it’s not objectively reasonable, the parties consent to the protocol will in most instances carry an imprimatur of reasonableness.
• Auditing / process transparency.  Similar to the first bullet, auditing the process and giving the opposition visibility into the process steps will often make it hard for them to lodge successful downstream challenges.
• Adherence to Local Rules (See 7th Circuit Pilot Program) or judicial order.  Another avenue than can provide some degree of safety is compliance with a discovery protocol mandated by local rules, although that compliance may ultimately be challenged.
• Statistical confidence intervals / sampling – the use of statistics as a way to bolster process defensibility is starting to come to maturity and in the future I think that detailed precision, recall and other statistical indicates will play a large role in e-discovery defensibility.

None of these steps can be guaranteed to really get you off the hook from a rapid opposing party calling foul, but using them in a “belt and suspenders” fashion will certainly help buttress any discovery process.

For more illumination on the topic please see the following video of my interview with John Loveland, who’s waxing poetically about discovery defensibility.

On of August 14, 2009, the California Judicial Counsel amended their Rules of Court to augment discussion of electronic discovery issues during the meet and confer process.

Rule of Court 3.724 was amended to require discussion of “Any issues relating to the discovery of electronically stored information” no later than 30 calendar days before the date set for the initial case management conference.  The broad language (i.e., “any”) was augmented by eight specific categories that must be expressly discussed:

(A) Issues relating to the preservation of discoverable electronically stored information;

(B) The form or forms in which information will be produced;

(C) The time within which the information will be produced;

(D) The scope of discovery of the information;

(E) The method for asserting or preserving claims of privilege or attorney work product, including whether such claims may be asserted after production;

(F) The method for asserting or preserving the confidentiality, privacy, trade secrets, or proprietary status of information relating to a party or person not a party to the civil proceedings;

(G) How the cost of production of electronically stored information is to be allocated among the parties;

(H) Any other issues relating to the discovery of electronically stored information, including developing a proposed plan relating to the discovery of the information;

Many of these issues track FRCP language (including forms of production, preservation, privilege issues, etc.).  However, section G seems somewhat novel given the historical “American Rule” where the producing party is required to bear all necessary costs of production.

Curiously missing, in comparison with FRCP 26 B(2)(b), is the need to discuss the handling of “inaccessible” ESI, although this could easily be subsumed in the “any other issues” language of section H.  Also missing is a discussion about proposed searching and/culling protocols (aka “keyword negotiations”) which are often part of the core meet and confer topics in Federal court.

Nevertheless, the scope is broad enough to require *a* discussion of all likely relevant electronic discovery issues, which was often lacking historically.  Once that discussion starts, reasonably savvy counsel should be able to flesh out most of the significant issues.  And, given this broad language a judge would presumably give them a hard time for any material omissions.

In the post, E-Discovery 911: Reducing E-Discovery Costs in a Recession, we analyzed the question: which electronic discovery activities are the most costly today and thus have the greatest room for cost reductions? An analysis of a typical, hypothetical case demonstrated that the bulk of e-discovery costs reside in the processing and review stages. In this post, we want to look at the different ways of reducing e-discovery costs and which are likely to be the most effective, especially given processing and review costs are the largest sources of expense.

Corporations have the following options for reducing e-discovery costs. Some of these approaches are aimed at changing the overall way e-discovery is performed. And some of these are aimed at improving the results of a particular step within a typical e-discovery process. None of the options are mutually exclusive.

• Retain less data through information management: one of the methods that corporations can undertake to reduce e-discovery costs even before e-discovery has begun is to adopt a data or document retention policy. Such a policy can, for example, stipulate that the corporation deletes all documents not required for specific business, legal or compliance reasons after a fixed period of time, such as 90 days. As a result, a properly implemented document retention policy has the potential to significantly reduce the amount of data that is identified and collected during electronic discovery.

• Better assess your case and your discovery issues: another approach to reducing the overall costs of litigation including discovery is to perform an early case assessment. Pioneered by Dupont and others, the objective of this approach is to understand all the key case facts within a short period of time so that the litigation team can make better decisions quicker. Because costs always rise over time, quicker resolution of litigation reduces costs. While early case assessment was originally an overall approach to litigation, there is now an equivalent in electronic discovery. The goal is to identify all the potentially discoverable data, but only collect, process, and analyze a prioritized portion of this data in order to inform an understanding of the case AND calculate an estimate of the ultimate potential e-discovery costs.

• Bring e-discovery in-house: another holistic method for reducing electronic discovery costs is to manage all or a portion of the e-discovery process in some or all matters inside the Enterprise as opposed to outsourcing it to law firms or litigation service providers. While bringing e-discovery in-house has other benefits, such as improved security and control, the principal benefit is to convert variable service costs, typically priced on a per Gigabyte basis, into fixed software costs thus producing a return on the investment to manage e-discovery in-house.

• Preserve and collect less data: in addition to holistic approaches, e-discovery costs can be reduced at each step in the e-discovery process. One way to reduce e-discovery costs would be to preserve and collect less data. Reducing the amount of preserved and collected data not only reduces the cost of each of these steps but also reduces the cost of each downstream step. There are pros and cons to this approach which I will discuss in a later post.

• Process less data: more data is frequently preserved and collected than needs to be processed for analysis and review. This excess data can be filtered out prior to processing thus reducing processing and all other downstream costs. The techniques used to do this are often referred to as pre-filtering, pre-processing or early data analysis.

• Process differently and review native: historically, most electronic data was converted to an image format, such as TIFF, prior to review. This process is computationally intensive and expensive. In recent years, e-discovery practitioners have been processing and reviewing more documents in a native or near-native format and avoiding the cost of converting documents to an image format until later in the process.

• Review less data: data can also be reduced after processing and prior to review and production. Much has been written in the e-discovery community about this process, often called “cull-down,” and the different search and analysis techniques that can be used as part of this process, such as keyword search, concept search, de-duplication, and others. The fewer documents requiring processing and review, which as we have seen is a substantial portion of the overall costs, the lower the overall costs.

• Review data faster: in addition to reducing less data, the electronic discovery community has pioneered new methods of reviewing data faster including data clustering, near de-duplication, and other more automated review techniques. The faster documents are reviewed, the lower the attorney review costs.

While all of these approaches have the potential to reduce the costs of electronic discovery, some are going to be more effective than others. Each approach can be implemented using a multitude of techniques or practices and each of these techniques has their pros and cons. For example, some techniques may have a greater risk of raising defensibility issues from the court or opposing side than others. Other practices may be less expensive initially, but, over the course of a changing and iterative e-discovery, may prove to be more costly overall. In a series of future posts, we’ll review the different practices used as part of these approaches and analyze the pros and cons of each to understand which may be the most effective for your organization.

Over the past 4 years, I have had hundreds of conversations with corporate counsel and “legal IT”, meaning technical folks charged with supporting the legal team. More and more of them are looking to lower their costs by bringing e-discovery in-house. But as they work through that process, there’s one question that consistently comes up, even today – namely, “When [insert name of software company] says they “do” e-discovery, what exactly does that mean?”

There has been progress towards answering this question, thanks mainly to the analyst community. George Socha and Tom Gelbmann’s EDRM framework has been immensely helpful in breaking down electronic discovery into its component steps. Other analysts, like Debra Logan at Gartner, were quick to embrace the framework, prompting every software provider to follow suit. As a result, there is today a common language that everyone uses to describe the e-discovery process.

The Electronic Discovery Reference Model (EDRM) breaks down the e-discovery process into a series of steps. Companies looking to buy e-discovery software to lower costs typically map different software products to each of these steps, to make sure that they cover the entire process.

But having a universally-agreed framework is only half the answer. To eliminate customer confusion, there also needs to be agreement on how different software products fit into the framework. This is especially important since there is no single, end-to-end solution for e-discovery which covers all aspects of EDRM. So customers are forced to think about how different software solutions fit together. And that is where things begin to fall apart.

Many software vendors feel it is advantageous to claim that they do everything, even though they do not. Customers are rightly suspicious of those claims, and so press vendors to provide more detailed information – hence the question, “when you say you do e-discovery, what exactly does that mean?”

In light of that, how can litigation support teams, corporate counsel, or legal IT people figure out which e-discovery solution best meets their needs? From observing this decision-making process hundreds of times, I have found 3 simple steps are incredibly helpful.

Step 1: Read the analyst reports

Two reports in particular make for required reading. One is Gartner’s MarketScope Report, which is available for free at certain sites; the other is the 451Group’s recent e-discovery report, which is summarized in a publicly available presentation. The helpful thing about the 451 Group’s report is that it tells you which software companies do which parts of the EDRM process. You do have to buy the report to get the full picture (it’s well worth it!), but the publicly available presentation will give you a flavor for their analyis, and I have drawn from that presentation in the figure below:

Analyst firms like the 451 Group map software vendors to the EDRM framework according to what they actually do, which is often different from what software vendors claim they do.

The 451 Group’s analysis highlights several important points. First, it shows that there is no single end-to-end solution. Even the products of giants like EMC (SourceOne), HP (IAP), and IBM (CommonStore) only solve one piece of the puzzle, information management. Second, it shows that customers have choices at each stage of the EDRM process. For example, to solve the problem of identification, collection, and preservation of electronic information, customers can choose from solutions as diverse as Guidance EnCase (forensic collection), Index Engines (back-up tapes) and Mimosa NearPoint (email archive). Third, it provides an independent assessment of what vendors do, as opposed to what they may claim. For example, Kazeon claims analysis and review capabilities, whereas the report shows its product does identification, collection, and preservation; Recommind claims its Axcelerate eDiscovery and MindServer products do processing, whereas the report finds that they do not.

Step 2: Evaluate the products prior to purchase

Just as anyone would test-drive a car prior to purchase, it’s critical to test-drive e-discovery software. Any vendor should be willing to provide their software free of charge for an evaluation on-premise. The most effective evaluations are when the customer uses the product themselves, either on a live case or test data. This is far preferable to just sending the data to the vendor who then loads it into their system, as in that scenario there are too many opportunities for the vendor to hide their product’s shortcomings.

Step 3: Check references carefully

The trick with references is to insist on relevant references. It’s not good enough for the vendor to dredge up some random person who says nice things; or even a credible knowledgeable person who is using the product in a completely different way. For example, if a company is happy with Autonomy’s IDOL for enterprise search, that does not tell you much about what Autonomy might be like for e-discovery. What really counts are references from other customers who are using the product for the same application that you are.

All this can sound like a lot of work, but I have seen people go through the process in as little as a month, and be much happier for it. A little work up front can save a lot of time (and heart-ache!) later on.

Cheesy Successories posters aside (for an alternative take, go here), the need to work together is much more than just a cliché in today’s environment.

In its recent brief on the five major trends that will shape business technology in 2009, leading management consultancy McKinsey and Company noted one trend in particular which highlights the urgent need for an organization’s IT and legal groups to forge better, faster, and more efficient ways of collaborating on electronic discovery issues:

Regulators demand more from IT

Government scrutiny of business will intensify in many developed countries. Already, in the United States, the Office of the Comptroller of the Currency weighs in on the resiliency of banking systems, the Food and Drug Administration (FDA) requires that many pharmaceutical systems be “validated,” and Sarbanes-Oxley drives decisions about accounting systems in every industry. In the future, policy makers and regulators will probably demand that IT systems capture more and better data in order to gain greater insight into and control over how banks manage risk, pharma companies manage drugs, and industrial companies affect the environment. Government officials also will monitor many legal and business rules more closely to ensure compliance with mandates. Successful CIOs should enhance their relationships with internal legal and corporate-affairs teams and be prepared to engage productively with regulators. They will need to seek solutions that meet government mandates at manageable cost and with minimal disruption.

- McKinsey Quarterly, February 2009

The current economic environment is creating a “Double Whammy” within almost every enterprise that has ongoing or pending electronic discovery issues (and are there many organizations left out there that don’t?):

• As the McKinsey article notes, regulators will increasingly be demanding more from IT as government scrutiny of business intensifies. Just look at the just-launched recovery.gov site to see the level of transparency and accountability that the government is aiming for with regard to the stimulus package. The bailout will not directly affect every business, but there is a new sheriff in town who will likely set the tone across the entire business landscape.
• At the same time, there is relentless pressure on controlling costs. When times are tough, dollars that can be saved on the expense side are much more valuable that top-line revenue, since 100% of every dollar of cost savings goes directly to the bottom line.

The net-net: Enterprises will be forced to do more, with less.

How? With regard to electronic discovery, there is a lot of low-hanging fruit to be picked in the area of IT and legal cooperation:

• In-house legal teams should meet with IT (if they aren’t already) to help them better understand the nature of electronic discovery, particularly as it applies to the more “upstream” parts of the process (specifically, identification, preservation, and collection) which IT tends to be more responsible for. Through a better understanding of the nature of electronic discovery, IT can improve its ability find the right documents, avoiding over-collection and reducing downstream processing costs. In addition, new electronic discovery technologies are making it increasingly easy for legal to own more of the process, reducing the electronic discovery burden on IT.
• Conversely, IT should coordinate with in-house legal teams to provide advice and mentoring as legal seeks to bring e-discovery platforms in-house to assist with early case assessment, search, culling, and analysis. To many legal teams, bringing e-discovery in-house may seem like a daunting proposition, but enterprise software has been around for a long time, and learning from IT’s experiences can make the process far less intimidating.

Yes, regulators are going to be far more demanding in the future than they have been in the past. But some simple collaboration and coordination between IT and legal will go a long way toward lightening the regulatory burden, especially as it pertains to electronic discovery.

I was recently asked: “what are the first things you do when your client calls you about a case requiring e-discovery?”  So, for the benefit of all, I’ll post my answer.

My first caveat to the advice was context.  Since, while a lot of attorneys have attended CLEs or have read about e-discovery, it’s not the same in the real world.  As the old Spanish Proverb goes:

It’s not the same to talk of bulls as to be in the bullring.

Keeping in mind that reality may differ significantly from academics, here are some things to consider when the next e-discovery case comes up.   Please also keep in mind that these steps (like the EDRM workflow) aren’t linear and may in fact occur cyclically or in parallel:

1. Preserve, preserve, preserve

Nothing is more important than meeting the initial preservation obligation, which begins when litigation is “reasonably likely” – as opposed to just when the complaint is filed.  This first step in the long journey can easily be a trap for the unwary/unprepared.

The challenge once you’re past the trigger issue is to then identify the boundaries of the duty to preserve, i.e., what evidence must be preserved?   This inquiry is often initially comprised of identifying key players, date ranges and data types.

Another significant challenge in this step is to monitor and update the legal hold process.  And, given that litigation more often than not spans years, it’s easy to initially succeed at the preservation effort, but then later fail on execution.  The best way to minimize risk in this step is to move quickly from preservation to collection.  See Is Preservation in E-Discovery Overrated?

2. Work backwards

Once preservation (and ideally collection) is adequately covered, the next step is to start thinking about the end of the process and what success (or lack of failure) looks like.  The exposure and profile of the matter are important to consider when you embark upon an e-discovery project since it’s critical to scale discovery efforts appropriately.

One thing, in particular, that is very important to consider early in the process is the type of production format that will be preferred by reviewing counsel and the opposition.  TIFF-based image productions (which are historically well accepted) are often pitted against native file ESI reviews.  Either format may or may not be acceptable given the situation and the applicability of FRCP Rule 34.

3. Understand the technical landscape

Most attorneys, but for a rare few, aren’t capable of really comprehending technical nuances of the complex and interrelated IT systems found at most Fortune 2,500 enterprises.  Fortunately, they are quite adept at working with experts (either consulting or testifying) to help them get to the bottom of difficult to comprehend and explain issues.  The key is find the right technical people who understand IT systems and who can explain it to judges, juries, and attorneys alike, especially for some of the most common ESI repositories like: email servers, archival systems, shared network drives, instant messaging servers, archival repositories (e.g., tape libraries, real time back-up systems, etc.), records management systems, knowledge management systems, proprietary, but highly leveraged, internal applications, offsite repositories (e.g., hosted IT or email systems) and significant partner or subsidiary data stores.  In many instances it will make sense to leverage or create a map of the data universe so that nothing is missed and inaccessibility arguments can be cogently detailed.

4. Get your lingo straight

Assumptions, whether in e-discovery or not, are often dangerous.  In the complex undertaking where multiple parties are handling ESI it’s critical to make sure that everyone is on the same page especially since every company handles IT, records management, ILM and information security differently.  So, when working with these disparate constituents the outset of an engagement is the right time to make sure everyone is on the same page.  Therefore, standardize on a set of commonly used terms. Examples of potentially ambiguous topics include “imaging” ,“archive”, and “records.”

5. Don’t assume your client will really be helpful

I’ve been involved with hundreds of e-discovery engagements and I’ve found that almost universally the end client professes a profound willingness to help out.  And yet, actual “help” is relatively rare.  To qualify this, it may be prudent to ask several additional questions:

• Does the Client have the time to actually help?  Everyone at the client’s site has a day job that they’re tasked with above and beyond transient e-discovery needs.  So, while bandwidth generally is important, what’s more critical is the ability to comply with aggressive judicial deadlines.
• Are the people helping the ones you’d want to see on the stand?  It’s often not realistic to have internal folks (especially IT and Records Managers) stay isolated during the various pre-trial events – meet & confer conferences and potentially 30(b)(6) depositions so it’s important to evaluate how a given witness will fare when providing testimony.
• How likely is it that you client would throw you under the bus if things went wrong?  In my opinion, there is now more reason for outside counsel to manage the risks of an e-discovery project going awry.  See, Sullivan and Cromwell’s suit against EED.  Some will wisely bring in 3rd party consultants/experts to have a neutral, unbiased constituent in the process.

6. Build a budget and team (internal/external)

Everyone is probably now aware of how expensive e-discovery can be if managed improperly.  This makes it all that more imperative to work quickly to get a rough sense of the scope (which will lead to a budget) and the client’s willingness to absorb associated charges.  The most important step is to right-size the e-discovery effort with the risks inherent in the corresponding litigation/investigation.  Otherwise, there’s a high likelihood that e-discovery process will be over-engineered (too expensive) or under-scoped (cutting dangerous corners).

7. Figure out your risk profile

Similar to right-sizing the budget, it also makes sense to adopt a “horses for courses” approach to e-discovery since there is no singular way to handle a given matter.  For example, in one case you make take forensic images, restore backup tapes, capture instant messaging data, harness metadata, or decide to do an automated review with a with a “clawback” provision. In either case, the only mistake is to assume that an approach from another, dissimilar matter is warranted in the instant case.

8. Assume the opposition is better informed than you are

While this actually may not be the case, it’s a safer bet that assuming a level of naiveté that may not exist.  What is certain is that the Plaintiff’s bar is increasingly well informed and can be very aggressive.  They’ve seen the playbook that calls for baiting the opposition into a discovery misstep that can result in significant, case altering sanctions.  According to a recent survey, 63% of the polled attorneys said that e-discovery is being abused by counsel, so it’s important to be wary initially.

It’s also important to consider the potential reciprocity of a given matter and adjust your position accordingly.  In many instances it’s easy to consider your role only as a producing party, but with cross/counter claims it may be possible to simultaneously be propounding discovery and in the opposition’s shoes.

9. Prepare for an early case assessment

A recent industry survey found that effective early case assessment (ECA) approaches reduced overall litigation in half of the cases evaluated, and resulted in favorable outcomes for 76 percent of the cases.   The key to this methodology is to use the available next generation case analysis solutions earlier in the process, not just to review data for relevancy and privilege, but to:

• Identify the key players. This is critical in order to have a defensible legal hold process
• Evaluate the posture of the case to determine how it looks on the merits
• Diagnose potential outliers in the e-discovery process to facilitate meet and confer discussions and help create “inaccessibility” arguments
• Conduct a search term analysis for keyword negotiations during meet and confer discussions.  Objectively demonstrating the results of proposed search queries can go a long way in speeding up keyword negotiations

10. Don’t take search for granted

For many attorneys, e-discovery search is just like Lexis or Google.  Unfortunately, that isn’t the case.  Instead, it’s become highly complex and is now receiving significant judicial scrutiny.  In Victor Stanley v. Creative Pipe Judge Grimm suggested that attorneys need to rethink how they’ve traditionally managed the search process:  “[F]or lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread.”  It’s now important to devise (and share at early meet & confer conferences) a defensible search strategy that can withstand judicial scrutiny.

The recent announcement of $18 million in financing for PSS Systems got me thinking about preservation.  PSS is a provider of enterprise-class preservation and litigation hold management systems with solutions starting in, from what I can tell, six figures.  Nevertheless, this begs the question, why would a Fortune 500 company need such an expensive enterprise class software application to manage legal holds?

So, let’s start from the top…

With the advent of e-discovery during the last decade an entirely new class of evidence spoliation came into existence – i.e., situations where electronically stored information (ESI), particularly back-up tapes, could inadvertently become overwritten, lost, erased, etc.  In the good old days of paper-based discovery, there was certainly an opportunity for spoliation, but paper documents didn’t routinely become lost or otherwise unavailable, unless in extreme instances of intentional spoliation.  For a particularly comprehensive tome on this type of negligent spoliation, please see this excellent piece written by Judge Scheindlin (of Zubulake fame).

Accordingly, in the past several years litigators have had to learn and then re-learn the notion that the duty to preserve ESI begins once litigation is “reasonably likely.”  Unfortunately, this duty to preserve is fraught with a number of practical challenges, including:

• When is the duty triggered?  For example, the duty is in most instances certainly in place prior to a complaint being actually served.  But, as you move upstream from that crystalline moment reasonable minds certainly can differ about when litigation is “reasonably likely.”  EEOC claims, in the HR context, are a good example of potentially early trigger points.
• Then, assuming that the duty is triggered what must then be preserved?  Is it just the ubiquitous email?  Or, as is more likely, will an increasingly broad and voluminous set of ESI be implicated, such as loose files, instant messaging, blog posts (maybe this one?), mobile or PDA/handheld data, deleted but forensically recoverable files, etc.?

Those two thorny problems aren’t the only issues that counsel needs to deal with when they embark upon issuing a legal “hold” – the decree that instructs custodians of their obligation to preserve all relevant information related to the matter at hand.  But, the duty to preserve is only the start of the challenge.  This is where folks like PSS come in, meaning that they manage the potentially complex logistical tasks associated with hold notification, monitoring, and compliance.

Here’s where I start to have a problems with large scale, complex preservation efforts.  Let’s take a somewhat common example:  a multi-national enterprise is sued for misappropriation of trade secrets.  Even prior to the complaint being filed, plaintiff’s counsel issued a demand letter, which in some cases could be held as a triggering event.  But, in either case, once the complaint hits the GC’s desk the duty to preserve is clearly in force.   Let’s then say that in consultation with outside counsel they wisely embark on a set of interviews to determine the scope of departments/locations/custodians that may be reasonably implicated.  Then, following the synthesis of this information they issue a legal hold notice to 2,500 people located throughout numerous domestic and international offices.

Now, here’s where the risk comes in…   One thing is statistically certain with that number of custodians: the legal hold will not be followed to perfection.  If I were more mathematically inclined I’d say it could be reduced to a formula along these lines:

Legal hold compliance *decreases* exponentially as you multiply:

• The number of custodians
• The length of time the legal hold is in effect
• The types and volumes of potential ESI that may be relevant
• The presence of individuals who don’t want data to be preserved due to their own perceived errors/foibles/omissions

The answer, in my mind, doesn’t lie in a better mouse trap to manage the vagaries of the legal hold process.  No, the best way to take the risk out of the legal hold process is to move very rapidly from preservation to collection.

Once ESI is collected two main things start to happen:

1. Subjective notions about the universe of data (allegedly) covered by the preservation process can be changed into objective observations that the custodians really are the right ones.  For example, in the above example the 2,500 custodian list is again almost certainly not correct.  Since the decision process was made subjectively (likely without insight into the data) the custodian list is inherently either under or over-inclusive.  However, with the advent of early case assessment solutions, the preserving party can now quickly collect and assess an initial corpus of data to ensure that exactly the right folks are in the collection/preservation process.
2. Once the ESI is collected, the risk of loss, deletion, etc. will largely have been taken out of the equation meaning that the danger of spoliation is greatly reduced.

My belief is that the larger the preservation effort the more likely there will be gaps that the opposition can use as leverage.  Scaling up the preservation effort is only one way to skin the cat.  Instead, the better practice is to start small, collect quickly, and then expand collection efforts once your legal team has objective insights into the case data.

Yes, preservation is still important. But, biting off more that you can chew simply means a statistically greater chance of failure.