Tomorrow Feit Consulting will be releasing a study “Patent Research: The Importance of Precision a Comparison of Docket Navigator and Lex Machina Data Searches” examining the comparative performance of Docket Navigator and Lex Machina two of the leading patent research and analytics platforms. The study was paid for by Docket Navigator. According to Feit, they invited Lex Machina to participate in the study. Lex Machina’s response is included below. Feit identified four expert patent researchers at ALM 100 firms who had access to both platforms. According to the report – the main question Docket Navigator wanted to answer was: Does AI outperform human search, or the reverse? Docket Navigator is described as “a human-search driven patent litigation platform.” The purpose of the study was to identify and compare the key differences in the quality and quantity of research results of Docket Navigator against those of its main competitor in the market place, Lex Machina, which uses AI in addition to human editors. Download the report here.
Questions that came to mind for me: “Are Docket Navigator and Lex Machina even offering comparable products? Does each product have unique strengths? The questions in the Feit survey seemed to be focused more on the execution of research tasks rather than the creation of analytics reports. Darryl Towell, Docket Navigator CEO described the product this way in response to my query. “We consider DN to be a data company that supports three main types of services:
(i) current awareness (the daily Docket Report email, alert emails, intra-day New Case Alert emails, Weekly Insights, etc.),
(ii) research (database searches), and
(iii) analytics, (report builder, profiles, special reports, etc.).
All three types of service are powered by the same data set that we created largely by hand from publicly available documents. For example, when we process a new case, that data would be delivered immediately as a New Case Alert and would be delivered the following day in the Docket Report email. The data would also be added to the database where it would appear when a user is doing research and also appear in relevant analytics services.”
I have been covering the growth of the analytics market closely for the past few years. Analytics products are complex and evolving. Patent products are even more complex than the average litigation analytics product. The Feit study needs to be evaluated by a patent research expert. I know enough about patent research to know that I am no such expert! Comparative surveys like the Feit survey allow experts dig in and look at the results and debate. This could be a lively panel discussion at the upcoming AALL conference in July 2021!
I can say this — law firms are the winners when vendors keep each other accountable by subjecting their products to public scrutiny. Thank you Docket Navigator for undertaking the research and laying down a challenge to Lex Machina to respond.
The purpose of the study was to identify and compare the key differences in the quality and quantity of research results of Docket Navigator against those of its main competitor in the market place, Lex Machina, which uses AI in addition to
human editors. To coordinate the study, Feit Consulting assembled a team of four representative examples of LM users/patent research experts each current employees on the research/library staff at Am Law 50 law firms and current/regular users of LM. Our role, then, was to provide an environment where researchers could provide candid feedback with no incentive to weigh answers toward one vendor or the other. The researchers were each given a set of 10 research questions drawn from real world examples. Some questions were designed to elicit highly specific results, probing for the most-used IP/Patent outputs, to facilitate comparisons. Feit monitored and compiled the users responses across the 10 research questions and determined for each of the 10 categories which platform
outperformed the other. Docket Navigator outperformed Lex Machina in 8 out of 10 search categories.