Last Thursday, Daniel Lewis, co-Founder of Ravel Law (now part of LexisNexis) gave the Keynote address at the annual Ark Best Practices & Management Strategies for Law Firm Library, Research and Information Services conference in New York. Instead of another frothy, sermon on the emergence of “robot lawyers,” Lewis delivered a measured analysis of the current state of AI in the legal market. It was a dramatic counterpoint to some of the overheated AI rhetoric reverberating througout the recent Legal Tech conference in New York. Lewis provided a framework for understanding what AI can do today. His talk covered current AI technologies and applications. But the topic which was of greatest interest to me was the a practical outline of questions to ask of vendors who are selling AI enabled products. How do you distinguish marketing hype from reality? How do you help manage lawyer expectations after they have read about the latest “game changing” AI product — which was acquired by a peer law firm? When the talk was over I felt like standing up and cheering.
Lewis provided a terrific overview of AI and machine learning. He explained why humans are not really like computers If a person can perform a mental task in less than one second of thought – that task can probably be automated using AI. There are a wide range of machine learning tasks that are already in use for tasks such as photo tagging, loan approving online ads, speech recognition, language translation and self-driving cars. Legal applications of Machine learning include contract analysis, motion analytics, recommendations, document classification, forecasting outcomes and discovery. One of the challenges for law firms is that AI requires the investment of scarce resources, data to train with and talent to work on customization and designing models.
Probabilistic syntactic parsing. Having spent the last 30 years using online research systems I recognized that legal terms had completely different meanings depending on the context. Case law is full of references to prior cases where a motion has been denied in a cited case, but that is not indicative of the outcome of the case in which it is cited. I finally got a tutorial on the problem. Lewis outlined how legal materials have a complex rhetorical structure which impact meaning – the same word has a different meaning depending on context: and many words can appear in a variety of different contexts within one legal opinion : procedural, referential, adjudicative or abstract.
What are the right questions to be asking? Lewis provided a great series of questions which can be the starting point for evaluating AI products.
How good is the training data?
Is it representative?
What is the validating process?
What are the evaluation methods?
What is the data?
Are the results accurate?
Are the results accurate at scale?
Who is training the systems users or the company?
What is the output?
How comprehensive in the coverage?
How comprehensive is the taxonomy – results will be limited by the taxonomy provided.
Who is creating the product?
What is their technical and legal expertise?
Is there a data steward?
The Vocabulary of AI Hype: Lewis also suggested that the audience be on the look out for the red flags of hype.
Marketing language using the words “transformative,” “cutting edge,” “game changing.
Anthropomorphism: language referring to the product as “understanding, ” ” reading,” ” thinking”
Will AI Replace Lawyers?
Even if algorithms can be trained to do more routine tasks, the majority of a lawyers analyzing, advising and communicating will not be automated any time soon.