Comments on: Judge Grimm, Victor Stanley, And The Problem Of “Black-Box” E-Discovery Search http://www.clearwellsystems.com/e-discovery-blog/2008/08/22/judge-grimm-victor-stanley-and-the-problem-of-%e2%80%9cblack-box%e2%80%9d-e-discovery-search/ thoughts about the evolution of e-discovery Fri, 12 Feb 2010 19:00:42 +0000 http://wordpress.org/?v=2.8.4 hourly 1 By: Will Uppington http://www.clearwellsystems.com/e-discovery-blog/2008/08/22/judge-grimm-victor-stanley-and-the-problem-of-%e2%80%9cblack-box%e2%80%9d-e-discovery-search/comment-page-1/#comment-1105 Will Uppington Thu, 28 Aug 2008 00:27:47 +0000 http://www.clearwellsystems.com/e-discovery-blog/?p=115#comment-1105 Daniel, thanks for your comment and for the additional information on Endeca's approach to search. I knew that Endeca always provided some visibility into search results ever since I first used your product in 2004.  It's good to see that you are emphasizing visibility in query expansion as well.  I agree with you that enterprise search can also benefit from moving away from a black box approach to a more transparent approach. As the research you point to demonstrates, transparency in search is valuable in helping to improve the relevancy of results by, for instance, suggesting additional keywords to the user.  However, critical differences remain.  For example, in e-discovery, transparency is also valuable in reducing false positives.  This is much less of an issue in enterprise search because false positives are simply not as costly as they are in e-discovery.  In addition, there is less of a need to provide visibility into multiple queries run as a single search.  Enterprise search users still typically want to run one query at a time and not the dozens of queries that e-discovery searchers must run.  Finally, there are still limits as to how much transparency enterprise users will want or use.  Many of these users are often unwilling to spend much additional time to reduce over- and under-inclusive results -- it depends on how important their search is and what their alternative is for finding the information. When the alternative is to call someone who knows, their patience can be limited.  E-Discovery users, on the other hand, are much more interested in improving their results because the payback is so significant.  It's not every day that you can save thousands of dollars by simply improving the results of a keyword search. Regards, Will Daniel, thanks for your comment and for the additional information on Endeca’s approach to search. I knew that Endeca always provided some visibility into search results ever since I first used your product in 2004.  It’s good to see that you are emphasizing visibility in query expansion as well.  I agree with you that enterprise search can also benefit from moving away from a black box approach to a more transparent approach. As the research you point to demonstrates, transparency in search is valuable in helping to improve the relevancy of results by, for instance, suggesting additional keywords to the user.  However, critical differences remain.  For example, in e-discovery, transparency is also valuable in reducing false positives.  This is much less of an issue in enterprise search because false positives are simply not as costly as they are in e-discovery.  In addition, there is less of a need to provide visibility into multiple queries run as a single search.  Enterprise search users still typically want to run one query at a time and not the dozens of queries that e-discovery searchers must run.  Finally, there are still limits as to how much transparency enterprise users will want or use.  Many of these users are often unwilling to spend much additional time to reduce over- and under-inclusive results — it depends on how important their search is and what their alternative is for finding the information. When the alternative is to call someone who knows, their patience can be limited.  E-Discovery users, on the other hand, are much more interested in improving their results because the payback is so significant.  It’s not every day that you can save thousands of dollars by simply improving the results of a keyword search.

Regards,
Will

]]> By: Daniel Tunkelang http://www.clearwellsystems.com/e-discovery-blog/2008/08/22/judge-grimm-victor-stanley-and-the-problem-of-%e2%80%9cblack-box%e2%80%9d-e-discovery-search/comment-page-1/#comment-1096 Daniel Tunkelang Fri, 22 Aug 2008 14:16:13 +0000 http://www.clearwellsystems.com/e-discovery-blog/?p=115#comment-1096 Will, great post! But, as Endeca's Chief Scientist, I'd like to correct a misunderstanding. Unlike most search engines, and actually in contrast to most of the approaches described in the information retrieval literature, we emphasize the transparency of query processing. We do so because our emphasis is on supporting exploratory search, where we expect that a user will progressively refine as he or she learns more about the available information and even about his or her information needs. As Nicholas Belkin and other library and information science researchers have demonstrated through user studies, transparency is key to supporting effective interaction. If you'd like to learn more about this research, you might start with this classic paper: http://home.cc.gatech.edu/nance/uploads/5/p205-koenemann.pdf In the spirit of transparency, I encourage you to check out my blog, The Noisy Channel: http://thenoisychannel.blogspot.com/ While I can't reveal all of Endeca's secrets there, I do try to communicate the vision of transparent, interactive information retrieval that drives everything we do. Will, great post! But, as Endeca’s Chief Scientist, I’d like to correct a misunderstanding. Unlike most search engines, and actually in contrast to most of the approaches described in the information retrieval literature, we emphasize the transparency of query processing. We do so because our emphasis is on supporting exploratory search, where we expect that a user will progressively refine as he or she learns more about the available information and even about his or her information needs.

As Nicholas Belkin and other library and information science researchers have demonstrated through user studies, transparency is key to supporting effective interaction. If you’d like to learn more about this research, you might start with this classic paper: http://home.cc.gatech.edu/nance/uploads/5/p205-koenemann.pdf

In the spirit of transparency, I encourage you to check out my blog, The Noisy Channel: http://thenoisychannel.blogspot.com/

While I can’t reveal all of Endeca’s secrets there, I do try to communicate the vision of transparent, interactive information retrieval that drives everything we do.

]]>