LegalTech New York 2015 Expectations (5 of 5): From Analytics to Machine Learning

Machine Learning BlogThis is the fifth of five posts discussing key themes that Blue Hill will be watching at LegalTech New York 2015.  [See Parts one, two, three, and four of the series].

Just a year ago, I was hearing serious industry commenters refer to the application of analytics and business intelligence (BI) to support legal operations, strategy, and knowledge management as “tinfoil-hat stuff.”  I’d argue that this was overly conservative at the time. In retrospect, it looks painfully naïve. The last year saw some significant activity by companies looking to insert machine learning and analytics applications into the legal sector, with these capabilities now representing a major presence at LegalTech. The attention these moves has attracted is well-deserved, given how these capabilities stand to improve on and automate aspects of legal work. As Ken Jennings has observed, there are no more potentially disruptive (truly disruptive, not tech-marketing “disruptive”) technology capabilities currently confronting the legal world. Nonetheless, these capabilities still largely sit at the margins of adoption, with most technology buyers struggling to understand the potential value that falls between the buzzwords and the anxiety that follows.

Nonetheless, both analytics and machine learning have a major presence at this year’s show, between dedicated session topics and vendor announcements. For example, Catalyst and Brainspace have focused on the machine learning capabilities of their platforms. Similarly, vendors like Recommind, Nexidia, FTI Consulting, and Content Analyst are brining various applications of analytics to legal environments. Analytics tools in particular seem to be seeing a significant increase in interest. In fact, the 2014 ILTA/InsideLegal Technology Purchasing Survey reported that 10% of surveyed law firms were planning investments in analytics solutions. This is a substantial number, considering the rate of technology spend in the sector.

Because the terms are closely related, and not always used with much rigor, understanding the potential of machine learning in legal requires that we review and distinguish two very different types of legal analytics (performance management “moneyball” tools and legal intelligence solutions), as well as understand machine learning itself. I use three rough working definitions to walk through these distinctions.

Performance Management Insight

While falling under the category of “analytics,” in practice this category often refers to more basic reporting and benchmarking capabilities. Performance management analytics draw data from repositories such as invoices, timekeeping records, court documents, or matter management data to extract measurable performance indicators of attorneys and law firms. This is typically what people mean when discussing “moneyball for lawyers.”

The precise factors considered and ultimate reports drawn will vary based on the use case. For example, performance benchmarking solutions embedded in enterprise legal management (ELM) and spend management platforms will focus on average cost per attorney, hours worked per matter type, and other factors that can be compared across a law department’s pool of firms or against performance averages. By the same token, law firm applications might track average hours billed or realization rates of particular attorneys.

As with machine learning, examples of development in this area over the last year include a range of applications. Thomson Reuters Westlaw Analytics (and discussed here), which helps organizations understand patterns in use of research tools; Lex Machina’s Motion Metrics solution, announced in November, provides organizations with performance trends in federal litigation practice drawn from the PACER database. Tikit’s Carpe Diem suite offers insight into trends in time use and task execution for law firms, while ELM platforms like Thomson Reuters Serengeti, Wolters Kluwer ELM Solutions, Bridgeway eCounsel, and Mitratech Lawtrac, offer similar insights for law departments that consume law firm services based on data extracted from invoices. Recommind’s expanded BI capabilities announced with Axcelerate 5 also fall in this category by permitting users to understand and predict the progress and cost of eDiscovery, although they are certainly not alone in this regard.

The most interesting development in this space in the run up to LegalTech has been Huron Legal’s acquisition of Sky Analytics. Sky Analytics is a standalone legal spend analytics solution, whose reporting and data analysis capabilities go deeper than many of those included in ELM suites. Huron, however, is largely a consulting services provider, rather than a technology provider, offering a compelling opportunity for mixing tools that increase insight with guidance regarding how to apply that insight.

While the increased operational insight that comes with these solutions helps organizations to create meaningful benchmarks of performance and identify trends and opportunities for improvement, this sort of insight is often sorely lacking in legal services environments. However, performance management analytics have little to do with applications of machine learning within legal work.

Legal Intelligence

Where the prior category referred to the management of legal work, legal intelligence solutions involve support for legal analysis. These solutions capture data from legal authorities or other resources in order to help users to identify trends, patterns, or other insights. Examples include Lex Machina’s Case List Analyzer or LexisNexis’s CourtLink and Practice Advisor solutions, all of which identify trends in decisions and types of legal actions and disposition by various jurisdictions and judges. Often, these applications are used to speed insight into a matter or assist with strategic planning. Other uses, such as the applications of IBM Watson, as introduced at ILTA 2014, use text recognition and pattern-matching capabilities to identify relevant cases or legal conclusions. Similarly, Nexidia’s newly launched Neural Phonetic Speech Analytics extracts text and sentiment analysis of large volumes of audio data to provide reviewers with insight into themes and flagged concepts to drive analysis of this data.

Generally, legal intelligence solutions are used to speed insight, or review or identify key points for strategy development and other key decisions. As such, their value relates to both the speed and quality of insight, ultimately impacting the effectiveness and efficiency of attorney representation. As a manual data-interrogation tool, legal intelligence thus has significant value to offer in legal knowledge and strategy analysis. However, the real potential of these tools lies in their combination with machine learning and automation.

Machine Learning

Machine learning refers to the capacity of software to automatically adjust its performance and operations based on the consideration of past results, pattern recognition, and user feedback to predefined rules and heuristics. As such, applications of machine learning involve a legal intelligence engine that automatically improves and recalibrates with use. Predictive coding and technology-assisted review (TAR) (themselves applications of legal intelligence within eDiscovery) represent the most familiar examples of machine learning for legal audiences. However, the degree to which this represents an autonomous aspect of the solution depends on the solution in question.

As a result of its applications in TAR, the technology has found a particularly strong presence within eDiscovery applications. This should not be surprising given its application in predictive coding. Nonetheless, this year’s LegalTech is seeing increasing advancements in the technology, with Catalyst highlighting its Continuous Active Learning approach to predictive coding, which calibrates its predictive coding tool based on ongoing insights across projects to continually improve the solution’s performance. In this same way, IBM Watson’s cognitive computing system stands to take rote research out of the hands of associates based on its ongoing assessments of relevance and responsiveness to the search inquiry. Fastcase’s Bad Law Bot rolled out last year works on a similar principle, although with a somewhat more terrestrial aim of identifying overruled case law and negative treatment.

Whatever the application, the value of machine learning in legal environments largely derives from continuously improving the automation of review of large volumes of content and identification of relevant material. These capabilities often mimic (and ultimately replace) repetitive and time-consuming information acquisition and review tasks that are often performed by junior attorneys, while progressively reducing the involvement required of more senior experts. Critics point to the potential for false positives, misapplied classifications, and a lack of transparency involved with these solutions, all of which represent major areas for improvement and key points to investigate when assessing vendors.

About David Houlihan, Esq.

David Houlihan researches enterprise risk management, compliance and policy management, and legal technology. He is an experienced advisor in legal and technology fields with a unique understanding of complex information environments and business legal needs.
Posted on February 5, 2015 by David Houlihan, Esq.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>