Last  Thursday, Daniel Lewis, co-Founder of Ravel Law (now part of LexisNexis) gave the Keynote address at the annual Ark Best Practices & Management Strategies for Law Firm Library, Research and Information Services  conference in New York. Instead of another frothy,  sermon on the emergence of “robot lawyers,” Lewis delivered a measured analysis of the current state of AI in the legal market.  It was a dramatic counterpoint to some of the overheated  AI rhetoric reverberating througout the recent  Legal Tech conference in New York. Lewis provided a framework for understanding what AI can do today. His talk covered current AI technologies and applications. But the topic which was of greatest interest to me was the a practical outline  of  questions to ask of vendors who are selling AI enabled products.  How do you  distinguish marketing hype from reality? How do you help manage lawyer expectations after they have read about the latest “game changing” AI product — which was acquired by a peer law firm? When the talk was over I felt like standing up and cheering. 


Lewis provided a terrific overview of AI and machine learning. He explained why  humans are not really like computers If a person can perform a mental task in less than one second of thought – that task can probably be automated using AI. There are a wide range of machine learning tasks that are already in use  for tasks such as  photo tagging, loan approving online ads, speech recognition, language translation and self-driving cars. Legal applications of Machine learning include contract analysis, motion analytics, recommendations, document classification, forecasting outcomes and discovery. One of the challenges for law firms is that  AI requires the investment of scarce resources, data to train with and talent to  work on customization and designing models.

  Probabilistic syntactic parsing. Having spent the last 30 years using online research systems I recognized that legal terms had completely different meanings depending on the context. Case law is full of references to prior cases  where a motion has been denied in a cited case,  but that is not indicative of the outcome of the case in which it is cited. I finally got a tutorial on the problem.  Lewis  outlined how legal materials have a complex rhetorical structure which impact meaning –  the same word has a different meaning depending on context:  and many  words can appear in a variety of different contexts within one legal opinion : procedural, referential, adjudicative or abstract.

 What are the right questions to be asking?  Lewis provided a great series of questions which  can be the starting point for evaluating AI products.

                How good is the training data?

                Is it representative?

                What is the validating process?

                What are the evaluation methods?

                What is the data?

                Are the  results  accurate?

                Are the results accurate at scale?

                Who is training the systems users or the company?

                What is the output?

                How comprehensive in the coverage?

                How comprehensive is the taxonomy – results will be limited by the taxonomy provided.

                Who is creating the product?

                What is their technical and legal expertise?

                Is there a data steward?

The Vocabulary of AI Hype: Lewis also suggested that the audience be on the  look out for the red flags of hype.

                Marketing language using the words “transformative,” “cutting edge,” “game changing.

                Anthropomorphism:  language  referring to the product as “understanding, ” ” reading,” ” thinking”

 Will AI Replace Lawyers?

                Even if algorithms can be trained to  do more routine tasks, the majority of  a lawyers analyzing, advising and communicating will not be automated  any time soon.