(Editor’s note: This special guest post was written by Chitran

As we’ve previously shared, our product strategy throughout 2009 and 2010 was to expand the product footprint across the EDRM as customers were demanding a single, end-to-end eDiscovery product. During this period we successfully expanded from our roots in processing, search and analysis to review and production (August 2009), identification and collection (September 2010) and legal hold workflow (March 2011). Over the last several months, our focus has been to go deep in each of these modules and provide features that deliver even greater return on investment to our customers.

Today, I am excited to announce significant new features and feature enhancements to the Clearwell Review and Production Module and say a few words about what motivated us to build these features and how they enable our customers to further streamline their legal review workflow.

There are several exciting features in this release, but I would to like to highlight three in particular:

1. Ability to seamlessly import production load files

Most matters require reviewing relevant documents alongside the documents received from third parties, opposing parties, and even previous litigations. With the new load file import feature, users can now streamline the process of importing load files with three simple steps.

In Step 1, a step-by-step wizard-like interface guides users though the selection of formatting information such as field delimiters and nested value delimiters, metadata information such as bates numbers, family relationships, tags, folders and any number of custom attributes, and content information such as images, extracted text and native files. When the load file has both extracted texts and native files, the wizard gives users an option to specify which content should be used for searching.

In Step 2, the system performs a deep validation of the load file and generates a report documenting any inconsistencies such as missing bates numbers or missing values for required fields found in the load file. As a result, customers have the ability to quickly find and fix any issues with the load file before the import begins.

In Step 3, the system imports the documents and builds analytics. Once this step completes, the imported documents, including all metadata and content, are available for viewing and searching.

All the analytics capabilities customers are familiar with, such as discussion threads and concept search, are also available for documents imported from load files. This allows users to quickly discover documents in the load file that are conceptually similar to natively processed documents, for example.

2. Support for large scale reviews and productions

As the volume of electronically stored information (ESI) continues to grow, our customers find themselves reviewing and exporting more and more documents, and they need a solution that can cope with the massive growth in data. At the same time, they don’t want to spend large sums of money building a server farm in anticipation of the growth. They want the flexibility to add capacity when needed and remove it when not needed.

Clearwell’s scale-out architecture enables administrators to easily add appliances and allocate them to a particular matter and to a specific task using a point-and-click interface.

For example, if an administrator needs to increase the number of reviewers from 200 to 400 in order to meet a tight deadline, he or she can easily add 2 appliances to the cluster and assign them for review. Once the review completes, the administrator can now easily re-assign these appliances for production, allowing users to easily meet deadlines while reducing their overall hardware costs.

This flexibility allows our customers to maximize the use of their hardware resources while providing infinite review, export and production scalability.

3. Streamlined management of exports and productions

Clearwell provides powerful export options, and while our customers use them extensively for creating a variety of different production formats, they typically standardize on a few. Clearwell’s new case export and production templates provide a quick and easy way for case administrators to define the export format once and use it across multiple cases. When exporting documents, users can simply select a template from the list of visible templates in that case. This capability significantly reduces the overhead associated with managing export formats and allows our customers to produce documents in a consistent format across multiple matters.

Additionally, new production pre-mediation reports automatically identify problem documents and group them by issue type for quick resolution. This enables users to preemptively identify and resolve document production issues without delaying entire productions.

Says Wendy Butler Curtis, chair of Orrick, Herrington & Sutcliffe’s eDiscovery Working Group, “Legal review is one of the most challenging phases of the eDiscovery process. As electronic data volumes continue to grow, it is increasingly important to leverage technologies that can streamline and improve legal review, ensure defensibility and reduce costs. Solutions like the Clearwell eDiscovery Platform enable legal teams to create an iterative eDiscovery workflow that allows for more efficient and effective large-scale review.”

We will be showcasing the new features at ILTA (Booth 816) this week in Nashville, so come see us and let us know what you think.

(Chitrang Shah is a Principal Product Manager at Clearwell Systems, now a part of Symantec, and the lead Product Manager for Clearwell’s Processing & Analysis and Review & Production Modules)

But, in 2011 we’ve now reached the tipping point – with 58 percent of respondents of the ESG survey expecting to manage social media applications as part of eDiscovery, more than double the 27 percent who did so in 2010.  That’s not only a massive increase in one year, but it also moves social media from a fringe element to a mainstream source of ESI.  When asked what types of social media applications would be the most relevant for eDiscovery, 79 percent of survey respondents named Facebook, followed by Twitter (64 percent) and LinkedIn (55 percent).

Similarly (and coincidentally), Applied Research and Symantec (who just acquired Clearwell) queried 1,225 senior enterprise IT professionals around the world in a Social Media Flash Poll.  In one of the main findings, the Flash Poll found that social media is extremely ubiquitous in the enterprise environment, with 45 percent of respondents using it for personal uses and 42 percent using it for business reasons.  Rating highly were a number of disparate social media devices including blogs, multimedia sharing, business forums and, of course, social networking – both personal (e.g., Facebook) and business (e.g., LinkedIn).

The impact on eDiscovery, while somewhat obvious, is nevertheless a significant challenge for many enterprises.

Initially, the increased use of social media intrinsically means that email isn’t likely to be the sole source of responsive information pertaining to a lawsuit (or governmental inquiry).  While this hasn’t really been the case for a while, it’s time for the attorneys scoping eDiscovery matters to face facts and abandon old school notions that email axiomatically equals eDiscovery.  For good or ill, our world of potentially responsive ESI simply isn’t that homogenous.

The Flash Poll also honed in on how this increased use of social media is impacting IT professionals.  While information governance concepts (compliance with regulations and retention polices – both at 45 percent) rated higher on their risk index, the management of eDiscovery was still a significant (and growing) concern at 37 percent.  And, while IT folks are increasingly concerned, it’s safe to say that their attorney counterparts (who have a heightened sense of risk profiling) are even more worried about the impact of social media on the already complex eDiscovery process.

So, what can be done in the face of this changing eDiscovery landscape that used to be dominated by email?  First and foremost, it’s imperative to understand your unique regulatory and legal requirements.  This facilitates the mapping of new social media technologies and content to the requisite policies that address data mapping and the retention of social media content, either in a proactive sense (i.e., archiving) or in a reactive sense (i.e., litigation hold).

As Glenn Close frighteningly said in her 1987 thriller, Fatal Attraction, “I will not be ignored.”  That warning fits the entire social media genre as it relates to eDiscovery in 2011.  And, just like ignoring Glenn Close, failing to pay proper attention to social media is done at significant peril to both IT professionals and attorneys alike.

There are already a large number of joint customers using the Clearwell and Symantec solutions as part of an integrated eDiscovery and archiving workflow, and we are well underway towards building more robust integration between Clearwell and Symantec Enterprise Vault. In updating our product roadmaps, all our decisions are guided by feedback from customers who have told us over and over again that they want to:

• Reduce costs across all phases represented in the Electronic Discovery Reference Model, from information management through review and production
• Reduce risk by improving the defensibility and repeatability of their archiving and eDiscovery processes
• Streamline their end to end archiving and eDiscovery lifecycle to meet legal and regulatory deadlines
• Start managing information and conducting eDiscovery in as little as one day; whether on-premise, as a hosted solution or in the cloud
• Meet their enterprise-wide archiving and eDiscovery needs, whether they have less than 25 to more than one million users

As we’ve discussed before, our plan as part of Symantec is to deliver a seamless, integrated archiving and eDiscovery management workflow that benefits all our customers. To keep everyone in the loop, we will continue to post updates and answer questions on the integrated product portfolio here and on the Symantec eDiscovery blog.

For more on the acquisition, and the response from our customers, partners and the industry at large, visit: http://www.symantec.com/clearwell.

For 6 of Clearwell’s 6 ½ years in existence, it has been my privilege to lead the company as its CEO. These have been, by far, the most rewarding, stressful, exhausting, and exhilarating years of my career. So in this, my final blog post, I would like to reflect on how we got here, and take this opportunity to thank some of the many people who made it possible.

***

In my view, there’s no single thing that makes a company successful. Rather, it’s a distinctive mixture of the right idea at the right time, executed the right way, by the right team, which gets the right lucky breaks and is propelled forward ahead of the competition by surging customer demand. That, in summary, is the story of Clearwell.

Right idea at the right time:

In the early days of a company’s life, when there’s no product and no hint of a customer, the only thing that you have is the idea. This is not the specific idea of what the company will do (that comes later); it’s the idea that there’s a huge change, a shift in the tectonic plates, that creates the opportunity to build a substantial new company. Much of this is about timing. Many changes are obvious over a 10-year timeframe, but it’s very hard to gauge which of them will occur in the 2-4 years that investors are willing to fund a startup venture.

The founding team at Clearwell was attracted by two big trends which combined to produce a profound change. One trend was that, by the mid-2000s, almost all communication within an organization had started to flow through email, as opposed to voicemail, memos, or hallway conversations. The other was that storage costs had fallen to the point where it was almost free to store all the email that people were generating. We realized that these two trends in combination had resulted in the creation of a user-generated written record of everything happening within an organization – something which had never existed before. Our hunch was that there had to be some way of unlocking value from this written record, while still respecting privacy.

Executed the right way:

We came to Clearwell with very specific ideas about how to build a world-class software company. These are too numerous and varied to capture here, but I will give you a few examples. In product development, we have always sought to build our enterprise products as if they were consumer products, so we made sure that they are intuitive and easy to use without any training. We designed them with the sales process in mind, by making them very easy to install and evaluate, so that prospects can try them out for free prior to purchase.  When it comes to marketing, we sought to promote a better way of doing e-discovery, rather than just pitch features, by championing the importance of early case assessments (ECA). With respect to pricing, we made the entry-point price as low as possible to encourage adoption, and pegged it to a metric that scales in line with value.  Strategically, we chose processing, analysis, and review as our entry point into the e-discovery market, because that’s where software provides the biggest, most immediate ROI.

In every area of the business, we brought a distinctive approach, all centered around our view of the ideal customer experience – the experience we would want to have, if we were our customers.

Right team:

The standard playbook for recruiting is to hire people who have done it before, ideally in the same domain. We took a different approach, and instead hired primarily based on personal qualities. Some of our team had no prior experience in enterprise software; many (including me) had never worked in e-discovery before coming to Clearwell. But we all share one thing in common: a relentless drive to win in the marketplace by building better products and providing better service than anyone else.

That hunger to win will trump experience every time. It’s the reason why engineers work through the weekend to resolve customer issues without being asked, or why a salesperson will travel 4 days out of every week to call on customers. It’s something that gets built into the company culture and then self-perpetuates. Our team tripled in size in the space of 18 months, and I never cease to be impressed by the fresh ideas and boundless energy coming from the new generations of “first-timers”.

Right lucky breaks:

Every successful company needs the rub of the green, and there have been many occasions when I’ve marveled at our good fortune. But perhaps our biggest break was that the Federal Rules of Civil Procedure (FRCP) changed for the first time in 38 years in December 2006, defining rules for the treatment of electronic information in the courts. This accelerated the movement from paper to electronically stored information and coincided perfectly with our entry into the market, drawing us into the electronic discovery domain.

Surging customer demand:

It’s an amazing feeling when you achieve “product/market fit”, as we did at the beginning of 2009. The user community among law firms and litigation support firms embraced our technology for ECA, taking our user base from hundreds to thousands. Enterprises woke up to the money that could be saved by bringing electronic discovery in-house, proactively issuing RFPs and creating new positions specifically responsible for e-discovery. Federal agencies began to adopt e-discovery solutions to sift through the vast quantities of data coming to them as part of their regulatory and investigative duties. Essentially, e-discovery became a core business process, just like finance, sales or HR – it became something that every organization had to do. And just as other departments use applications like salesforce.com (sales), Success Factors (HR), or NetSuite (finance) to manage those business processes, so it was that legal departments realized that they needed an application like Clearwell to manage the e-discovery process.

All of a sudden, the business accelerated, sales took off, and we felt ourselves being pulled in every direction at once. In response, we expanded our platform, moving from 1 product to an integrated platform of 4 products; and, we increased our geographic coverage by building out the sales team across North America and establishing beachheads in Europe and Asia. The Clearwell team worked around the clock to respond to customer demand, while at the same time recruiting and training as we added people at a furious pace. We learned that hyper-growth can be painful, but in a good way.

***

When things go well, the CEO often takes a disproportionate share of the credit. I must confess, it would be nice to think that the company’s success is due to some kind of brilliance or magic touch on my part, but the reality is quite different. This has been a team effort from beginning to end and there is a very long list of people who deserve recognition. It’s impossible to capture them all, but I’m going to do my best, by saying a heart-felt “thank you” to:

• Venkat Rangan and Charu Rudrakshi who started the company, raised the first round of funding, and set the DNA of the engineering team;
• Jim Goetz at Sequoia Capital who acted more as co-founder than investor in the company’s first year, and has since been incredibly supportive of the management team;
• Tom Dyal at Redpoint Ventures for his support and insightful advice on strategy; Bill Coughran at Google for helping us think through how best to scale engineering; John Dillon at EngineYard for teaching me what it means to sell software; and, Scott Dettmer at Gunderson Dettmer for his finesse and deft touch in managing the most delicate negotiations;
• Andy Byrne, Anup Singh, Kamal Shah, Ryan Snyder, Soumitro Tagore, Trevor Eddy, and Venkat Rangan for creating a truly outstanding management team built on trust and mutual understanding – it is quite remarkable that in 6 years, the company has only ever had 1 VP Business Development, 1 CFO, 1 VP Marketing, 1 VP Sales, and 1 CTO;
• Amar Laud, Amy Johnson, Andy Kashyap, Aruna Mantripragada, Bill Duffy, Brandon Cook, Cat Lee, Chitrang Shah, Cris Barrett, Dave Fraleigh, David Speicher, Dean Gonsowski, Donna Hui , Doug Kaminski, Ed Hinton, Jason Montgomery, Jason Reeve, Joe Schwartz, Krista Jones, Kurt Leafstrand, Malay Desai, Manish Sampat, Mark Wentworth, Mike Lee, Peter McLaughlin, Sangeeta Relan, Sean Wilcox, Steve Rapp, Subbu Gooty, Teddy Cha, Tom Kennedy, Tom Wells and Umair Hamid for being the leaders who have really defined the company, and without whom we would never have got anything done;
• Clearwell “Class of 2005” for their super-human efforts in shipping Version 1 and launching the company; Clearwell “Class of 2006, 2007 and 2008” for tirelessly iterating until we cracked the code for a profitable business model; and, Clearwell “Class of 2009, 2010, and 2011” for driving the huge expansion of our operations, both in the US and overseas;
• John Petruzzi from Constellation Energy, Joe Tawasha from Charles Schwab, Don McLaughlin from Qwest, Pallab Chakraborty at Oracle, Jesse Hartman at the Department of Health and Human Services, and Ron Best at MTO for being bleeding edge customers who took a chance on a fledgling technology;
• Jeff Fehrman from Onsite; Greg Mazares, Keith Lieberman and the infamous Taylor brothers at Encore; and Paul Tombleson at KPMG UK – for being the first service providers to embrace Clearwell’s technology;
• Debra Logan and John Bace at Gartner; Barry Murphy and Greg Buckles at eDiscovery Journal; Brian Babineau and Katey Wood at ESG; Brian Hill at Forrester; Chris Dale of the eDisclosure Information Project; George Socha and Tom Gelbmann; Nick Patience at 451Group; and Vivian Tero at IDC – for doing so much to help define e-discovery software as a space and make it intelligible to end-customers;
• Deepak Mohan and Brian Dye at Symantec for sponsoring an acquisition that will massively accelerate the adoption of Clearwell’s technology; and,
• Finally, Enrique Salem and the entire Symantec M&A and Integration Teams for giving us such a warm welcome into the Symantec family.

***

It has been a remarkable journey. I feel proud, and humbled, to have been a part of it.

Before Clearwell, vendors could only rely on building their own indexes when attempting to collect content by keyword from unstructured document sources. They did this in one of two ways.

The first method was to build one-off indexes with each collection, indexing content and then discarding the index after collection is complete. This minimized the amount of infrastructure required to maintain the index, but was painfully slow and wasteful of computing and network resources. These sorts of solutions came from vendors who originally focused on the forensic investigation side of the world, whose tools had been designed around small-scale collection from individual devices and hard drives. Unfortunately, they simply don’t scale to meet the demands of today’s large enterprises with their ever-increasing data volumes.

The second method was to attempt to create an uber-index of all of the information in an enterprise and keep it continually updated so that it would be ready at a moment’s notice for your collection needs. This approach proved to be incredibly challenging to implement, required a huge amount of infrastructure to maintain, and, worst of all, didn’t really work: creating the uber-index, as it turns out, was uber-difficult.

In talking with hundreds of customers over the last couple of years, we realized that there was a better “third way,” which combined the lightweight nature of the first method with the comprehensiveness of the second. How? By leveraging the indexes that enterprises already have in place. From comprehensive, robust archiving solutions like Symantec Enterprise Vault to the fully-searchable indexes found on Microsoft SharePoint, Exchange, and file servers, the way of finding the information you need quickly for e-discovery is, by and large, already out there. It’s simply a matter of building an e-discovery platform sophisticated enough to leverage those indexes and, when necessary, be intelligent enough to build its own when not available from another source. That’s exactly what we’ve done with Clearwell’s targeted keyword collection feature.

One of the most exciting things about this approach is that, while it works great for today’s enterprise information infrastructure, it is perhaps even more powerful in tomorrow’s. As your company’s information stores gradually shift toward the cloud, leveraging the indexes in the cloud becomes essential to being able to access the information that lives there in a fast and efficient manner. It’s simply not feasible to be able to use the “one-off” or “uber-index” approaches when data is living in a cloud infrastructure, since data access rates are often slower because they are occurring over a wider-area network.  Last year, Clearwell was the first e-discovery platform to support direct access of cloud Exchange and SharePoint environments, and now with keyword collection we have made another great stride forward in achieving our customer’s vision for next generation e-discovery. And there’s still more to come as we accelerate our product development by integrating with Symantec’s world-class information management team. Stay tuned!