Ralph Losey, an attorney for Jackson Lewis, reported last week that a Delaware judge took matters into his own hands by proactively requiring both parties to show cause as to why they should not use predictive coding technology to manage electronic discovery. Predictive coding advocates around the globe will eagerly trumpet Judge Laster’s move as another judicial stamp of approval for predictive coding much the same way proponents lauded Judge Peck’s order in in Da Silva Moore, et. al. v. Publicis Groupe, et. al. In Da Silva Moore, Judge Peck stated that computer-assisted review is “acceptable in appropriate cases.” In stark contrast to Da Silva Moore, the parties in EORHB, Inc., et al v. HOA Holdings, LLC, not only never agreed to use predictive coding technology, there is no indication they ever initiated the discussion with one another let alone with Judge Laster. In addition to attempting to dictate the technology tool to be used, Judge Laster also directed the parties to use the same vendor. Apparently, Judge Laster not only has the looks of Agent 007, he shares James Bonds’ bold demeanor as well.
Although many proponents of predictive coding technology will see Judge Laster’s approach as an important step forward toward broader acceptance of predictive coding technology, the directive may sound alarm bells for others. The approach contradicts the apparent judicial philosophy applied in Kleen Products, LLC, et. al. v. Packaging Corporation of America, et. al. — a 7th Circuit case also addressing the use of predictive coding technology. During one of many hearings between the parties in Kleen, Judge Nan Nolan stated that “the defendant under Sedona 6 has the right to pick the [eDiscovery] method.” Judge Nolan’s statement is a nod to Principle 6 of the Sedona Best Practices Recommendations & Principles for Addressing Electronic Document Production which states:
“[r]esponding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.”
Many attorneys shudder at the notion that the judiciary should choose (or at least strongly urge) the specific technology tools parties must use during discovery. The concern is based largely on the belief that many judges lack familiarity with the wide range of eDiscovery technology tools that exist today. For example, keyword search, concept search, and email threading represent only a few of the many technology tools in the litigator’s tool belt that can be used in conjunction with predictive coding tools to accelerate document review and analysis. The current challenge is that predictive coding technology is relatively new to the legal industry so the technology is much more complex than some of the older tools in the litigator’s tool belt. Not surprisingly, this complexity combined with an onslaught of new entrants to the predictive coding market has generated a lot of confusion about how to use predictive coding tools properly.
Current market confusion is precisely what Judge Laster and the parties in EORHB must overcome in order to successfully advance the adoption of predictive coding tools within the legal community. Key to the success of this mission is the recognition that predictive coding pitfalls are not always easy to identify– let alone avoid. However, if these pitfalls are properly identified and navigated, then Judge Laster’s mission may be possible.
Identifying pitfalls is challenging because industry momentum has led many to erroneously assume that all predictive coding tools work the same way. The momentum has been driven by the potential for organizations to save millions in document review costs with predictive coding technology. As a result, vendors are racing to market at breakneck speed to offer their own brand of predictive coding technology. Those without their own solutions are rapidly forming partnerships with those who have offerings so they too can capitalize on the predictive coding financial bonanza that many believe is around the corner. This rush to market has left the legal and academic communities with little time to build consensus about the best way to properly vet a wide range of new technology offerings. More specifically, the predictive coding craze has fostered an environment where there is often a lack of scrutiny related to individual predictive coding tools.
The harsh reality is that all predictive coding tools are not created equally. For example, some providers erroneously call their solution “predictive coding technology” when the solution they offer is merely a type of clustering and/or concept searching technology that has been commonly used for over a decade. Even among predictive coding tools that are perceived as legitimate, pricing varies so widely that using some tools may not even be economically feasible considering the value of the case at hand. Some solution providers charge a premium to use their predictive coding tools and require additional expenditures in the form of consulting fees, while others tools are integrated within easy-to-use eDiscovery platforms at no additional cost.
If the court and parties decide that using predictive coding technology in EORHB makes economic sense, they must understand the importance of statistics and transparency to insure a fair playing field. The widespread belief that all predictive coding technologies surpass the accuracy of human review is a pervasive misperception that continues to drive confusion in the industry. The assumption is false not only because these tools must be used correctly to yield reliable results, but because the underlying statistical methodology applied by the tools must also be sound for the tools to work properly and exceed the accuracy of human review. (See Predictive Coding for Dummies for a more comprehensive explanation of predictive coding and statistics).
The underlying statistical methodology utilized by most tools today is almost always unclear which should automatically raise red flags for Judge Laster. In fact, this lack of transparency has led many to characterize most predictive coding tools as “black box” technologies – meaning that inadequate information about how the tools apply statistics makes it difficult to trust the results. There are differing schools of thought about the proper application of statistics in predictive coding that have largely been ignored to date. Hopefully Judge Laster and the parties will use the present case as an opportunity to clarify some of this confusion so that the adoption of predictive coding technology within the legal community is accelerated in a way that involves sufficient scrutiny of the processes and tools used.
Judge Laster and the parties in EORHB are presented with a unique opportunity to address many important issues related to the use of predictive coding technology that are often misunderstood and overlooked. Hopefully the parties use predictive coding technology and engage in a dialogue that highlights the importance of selecting the right predictive coding tool, using that tool correctly, and the proper application of statistics. If the court and the parties shed light on these three areas, Judge Laster’s predictive coding mission may be possible.