e-discovery 2.0

Last fall, when Transparent Concept Search was still in development, we showed an early version of it to a group of our customers. Their excitement was palpable, and they spent most of our session together comparing notes about the varied ways they will use it. But at the end of the discussion, one of them asked the question which was on everyone’s mind: “how much will you charge for it?”, or as someone else immediately said “I get charged $200/GB for plain vanilla concept search, so how much of a premium do you think you will get for this?”

Our answer surprised them: there’s no charge. Transparent Concept Search is included in Clearwell for free. Here’s why doing that makes sense:

There are two business strategies in the technology industry which are proven to work. One is to be the low-cost provider and compete on price. These companies, such as Chinese PC manufacturers, do not spend anything on R&D or marketing. Instead, they ruthlessly squeeze out cost savings and pass them on to their customers. The other proven strategy is to be the innovation-leader, whereby you continually delight customers by giving them more and more functionality at the existing price. Players following this strategy are never the cheapest, since they charge a little extra to fund new product development. For example, iPhone is by no means the cheapest smart phone, but its price did not go up when, with the iPhone 4, Apple added video, a forward-facing camera, better battery life, and a retina display.

It is worth noting that either strategy can work, and companies sometimes move between the two, although making that transition is incredibly hard. Staying in the PC industry, Dell started as the low cost provider, but has more recently tried to move up the value chain by investing more in the design of its products. The results, so far, have been mixed.

At Clearwell, our strategy is to be the innovation leader in e-discovery software. We tackle really hard technical problems, solve them in innovative ways, and then seek to delight our users by providing them with breakout, new capabilities at no incremental cost. Transparent Concept Search is a perfect example of this.

Rather than just integrate with concept analysis plug-ins, as pretty much every review platform does, we asked ourselves: if we were to create concept search from scratch specifically for e-discovery, what would we build? As part of that process, we tapped into the latest academic research in semantic analysis coming out of UCLA, University of Pittsburgh, and other universities, and discovered that it offers a solution to the biggest single problem users have with concept search: the heavy computational burden traditional approaches require. By using a variation of the semantic space model which is explained in that new research rather than, say, latent semantic indexing, we can deliver concept searching to much larger legal matters.

Beyond the core technology, we also wanted to change the user experience, by bringing the same level of visibility and control that our users enjoy in keyword search to this domain. Our goal is to enable users to balance both precision and recall in a way that was not previously possible. The result – Transparent Concept Search – is completely seamless within Clearwell in a way that simply cannot be matched by concept search plug-ins to a review platform, which are essentially two separate products from two separate vendors. In summary, it’s a vastly superior user experience – at no incremental cost.

This is the first of many things you will see from us this year. Our team could not be more excited about the new products and ideas that we have in the pipeline.

For years, concept search in electronic discovery has been like concept cars at auto shows: Cool. Slick. The thing that everyone is talking about.

But not ready to move to the assembly line and be put into production.

Like a concept car, concept search has been based on a lot of good ideas and shown a lot of promise. However, it has failed to move beyond a few edge use cases and reach mass adoption in the e-discovery market.  Why is this the case?

It’s not been because it’s an unproven idea or that the basic technology hasn’t been available. In fact, the core algorithm that underlies most existing concept search technologies has actually been around since 1988, when latent semantic analysis (LSA) was first patented by a team from Bell Labs. Over the last 20 years, dozens if not hundreds of companies have sprung up to apply concept search to the broad area of enterprise search and to e-discovery in particular.

To understand why concept search has never taken off, it’s always interesting to look for parallels, and the parallel du jour is social networking. Readers of David Kirkpatrick’s excellent book The Facebook Effect and (perhaps to a lesser, more fictionalized extent) viewers of the movie The Social Network understand that Facebook was far from the first social networking site (remember MySpace? You won’t admit it, but I know you do). But, despite being several years late to the party, Facebook somehow took the core of the social networking idea and presented it to users in a way that really allowed it to “cross the chasm” to the mainstream market.

In introducing Transparent Concept Search, Clearwell plans to help conceptual search cross that same chasm in e-discovery.  In talking to customers over the last couple of years, we have found that there are unmet customer needs with existing concept search products that, once addressed, will really allow its use in e-discovery to flourish – and not just in a way that makes concept search marginally more useful, but, a la Facebook, makes it orders of magnitude more useful.

What are these unmet customer needs?

Ease of use: Historically, concept search has been relatively easy to use in the strictest sense of the word – you type in some terms that represent your concept, and you get a set of search results back, along with some related terms and/or clusters of related documents. Simple, right? The issue is that in most cases that’s not what the user really wants to do. Because concept search is inherently “fuzzy”, users want to be able to refine their concept based on the feedback that they got from their initial search. Concept search, just like keyword search, is an iterative process, and prior-generation technologies have not allowed for that form of iteration. In contrast, Clearwell’s Transparent Concept Search allows concepts to be defined and refined in a way that is intuitive, visual, and (don’t take my word for it, but try it for yourself) fun.

Precision: Traditional concept search increased recall when compared to just keyword search, but it came at the cost of precision. The refinement process facilitated by Clearwell’s Transparent Search addresses this issue by allowing intelligent human input to guide the concept search process. You get the best of both the recall and precision worlds with vastly diminished time and effort.

Defensibility: Even more important than ease of use and precision is defensibility. Defensibility, for those new to the term, isn’t so much about whether the way the algorithms work is known and able to be understood. They are, and aren’t that complicated. Rather, defensibility is about reasonableness: was the concept search a reasonable way of determining which documents are responsive? Without the ability to define your concept in an interactive manner, we believe that the answer has historically been “no”, making concept search nice in theory but unusable in actual legal practice. Transparent Concept Search promises to change that. The end result is a more defensible search process that yields both greater recall and greater precision, enabling users to more quickly analyze case facts, rapidly identify key documents that may have been missed, eliminate irrelevant documents, and prioritize the most relevant documents for review. Clearwell also provides a reporting and auditing feature to document your search, allowing you to improve defensibility by “proving up” what was done.

Low cost: Finally, never underestimate the value of “free” in helping meet the ever-important unmet need of cost predictability and control. Historically, vendors have charged price premiums (often substantial) for concept search. Trying to charge a premium in e-discovery for something that doesn’t fully meet the customer use case and isn’t defensible, and it’s a recipe for low adoption. However, provide a highly useable, effective, and defensible capability as part of the core functionality of today’s leading e-discovery platform, and it starts to look very attractive indeed.

Hopefully you can tell that we’re incredible excited about the promise that this technology holds for the market, and this initial version is really just the beginning. Want to see it for yourself? Check out the video below, visit our web site or, if you are in New York this week, please visit us at LegalTech New York – we would love to see you.

I’ve been doing this long enough that predictions about the future (for good or ill) seem to be a useful convention to talk about emerging trends in the electronic discovery space.  My recent post about the top 5 trends for the upcoming year received way more attention than I would’ve imagined.  So, in attempt to replicate that and do my best Carnac impersonation, here are a few predictions about ALM’s 2011 Legaltech New York:

• “All in one” e-discovery will be the vendor message de jure, even if the beginning phase is first pass review and the last phase is granular review.  The mantra “don’t let the truth get in the way of a good story” (not surprisingly) holds more water at LTNY than in most places.
• For the first time, it will actually take longer to get to your room in the Hilton (never mind the annoying ads) than it would at another hotel across the street.
• At least one person will have an Inception moment and think that they’re having a dream (or perhaps a dream within a dream – if they’re really sedated) about being on the Legaltech show floor, while they’re really still in their hotel room, waiting for their Starbucks Trenta (a whopping 30 ounces) to kick in.
• The b-discovery group will hold a massive all chapter party without any official affiliation with the LTNY conference, showing again that the off-the-floor meetings, parties, sessions, interviews and the like are continuing to eclipse the officially sanctioned events.  As another prime example, check out these stellar Supersessions.
• Members of the European Cockpit Association (yes there really is one) will accidentally show up for day one of the conference, after getting a pamphlet at LaGuardia airport advertising free wine and cheese with a discussion of ECA.
• DTI will announce that it’s stepping up and acquiring FTI (since the acronyms are pretty similar) after a torrent of other purchases in the space, including Unlimited Discovery and Daticon/EED.
• Given his prominence in NYC, I think it’s likely that Donald Trump makes an appearance (probably to look for a lit support professional as an all new Apprentice: the e-discovery edition).

As the old saw goes, “you don’t need to be a weatherman to know which way the wind blows.”  I recently read the recent Duke Law Journal article “Sanction for E-Discovery Violations: By the Numbers.” And, my initial reaction (as someone who tries to read all the electronic discovery cases as they come off the presses) wasn’t one of surprise.  For me, most of the interesting e-discovery cases all seem to involve sanctions as they usually provide lessons about “worst” practices, from which we hopefully can infer “best” or at least “better” practices.

But, after a deeper dive into the piece I was fascinated by the metrics and granular trending analysis.  The article summarized 401 cases involving motions for sanctions related to discovery of electronically stored information (ESI) in federal courts prior to January 1, 2010:

“We analyzed these cases for a variety of factors, including sanctioning court, sanctioning authority, sanctioned party, sanction type, and sanctioned misconduct. Our analysis indicates that although the annual number of e-discovery sanction cases is generally increasing, there has been a significant increase in both motions and awards since 2004. Motions for sanctions have been filed in all types of cases and all types of courts. The sanctions imposed against parties in many cases are severe, including dismissals, adverse jury instructions, and significant monetary awards. Sanctions against counsel, although uncommon, are on the rise as well. All the while, the safe harbor provisions of Rule 37(e) of the Federal Rules of Civil Procedure2 have provided little protection to parties or counsel.”

Full analysis of the entire article isn’t the goal of this blog post.  Instead, I wanted to focus on one particular area that hasn’t received a lot of attention.  In an age where counsel seems particularly obsessed with conducting e-discovery in the cheapest way possible, it seems that many have lost sight of the risks associated with cutting corners.  The Duke article significantly noted that “[c]onsistent with the overall increase in sanction cases,…counsel sanctions for e-discovery have steadily increased since 2004.”  The authors identified various levels of misconduct as the basis for counsel sanctions — “four cases involved negligence, seven cases involved gross negligence, nine cases involved reckless disregard, and ten cases involved intentional conduct or bad faith.”

What should be eye-opening for counsel are the negligent sanctions cases because most will (probably correctly) assume that intentional, bad acts are relatively rare.  Fortunately, the article analyzes the three different categories of sanction-able conduct by counsel that have been deemed grossly negligent.

“First, the failure to advise the client to issue litigation holds or to otherwise take steps to preserve potentially relevant information has been found to be gross negligence. Second, the failure to supervise a client search for responsive information by accepting client representations as to the adequacy of the client’s search, in light of clear information to the contrary, has been held to constitute gross negligence. Finally, the failure to produce a critical document in the possession of counsel for several years has also been held to constitute grossly negligent conduct.”

Sanctions can also be based on the “counsel’s personal execution of discovery tasks or on the counsel’s role in coordinating and overseeing the client’s discovery.” As discussed in Metropolitan Opera, although “counsel need not supervise every step of the document production process and may rely on their clients in some respects, the rule expressly requires counsel’s responses to be made upon reasonable inquiry.”

What’s clear, in my mind at least, is that the standard of care for counsel conducting e-discovery is quickly rising.  In the past, any number of obnoxious behaviors was seemingly tolerated by the bench, including everything ranging from the blissfully ignorant to downright obstreperous conduct.  Now, the bench and the plaintiffs bar have both been through enough wars to know “best” e-discovery practices from its lesser cousins.  For counsel that have been running and gunning, hoping to skate by on a quick and cheap MO, these increasing sanctions awards should be a clarion call to what the future likely holds – and that’s an increase in malpractice insurance premiums.