Today Judicata released the results of a first of it’s kind ranking – the quality of their briefs. Founder Itai Gurari authored a blogpost on their website which explains the study. Judging Lawyers: Objectively Evaluating Big Law Litigation Departments.
Last November I wrote a post describing how the Judicata Clerk product can be used to evaluate the strength of a brief during the drafting process. Forget the Robots You Might Just Need a Clerk.
Gurari’s post describes Judicata’s goal in releasing this ranking as an effort to illustrate how clients can use data to evaluate the quality of a law firms work product. “Data has yet to significantly impact how clients select their litigation counsel. Lawyer and law firm evaluation has traditionally been a subjective measure: driven by perception, personal relationships, and word-of-mouth recommendations of a law firm’s litigation.”
Guari acknowledges the work of analytics pioneers such as Lex Machina in utilizing docket data to gain granular insights into a vast array of strategic insights such as motion outcomes and judges behavior. “Yet that higher level docket-based analysis is still far removed from the more fundamental motion practice skills at the core of every litigation — the drafting and argumentation that go into the briefs filed in support of a court motion.”
The Ranking Methodology Judicata compared briefs from the 20 largest law firms in California. These 20 law firms are among the 100 largest law firms in the world, and include firms in the global twenty.
Judicata ran 500 briefs filed in California state court through the Clerk evaluation process. The briefs spanned a diverse set of lawsuits, covering everything from individual employment actions, to complex contract disputes, to billion-dollar startup founder fights. The clients included young California companies like Lyft and Snapchat, Silicon Valley behemoths like Apple, Facebook, and Google, and international giants like BP and Toyota. Some of the lawsuits were highly publicized, and some involved well-known celebrities like Kim Kardashian, Carrie Fisher, and Manny Pacquiao.
The result of this evaluation is the first-ever ranking of law firm litigation departments based on an objective measure of their lawyering skills. Although all the briefs that were analyzed were filed in California state court, Guari points out that Clerk ‘s evaluation criteria “are fundamental to motion practice and translate to both federal court and other US jurisdictions.” Here is the Judicata rankings of law firm briefs. The full report including ranking for arguments and drafting is here.
Law Firms are sensitive to their placement in ranking lists. Most of these rankings have been based on either survey responses or financial metrics such as profits per partner or value of deal representation. This is the first survey I am aware of offering an algorithmic assessment of a law firm’s work product. It is likely that more rankings of this kind will be developed. One positive outcome is that the focus on work product rankings will underscore the importance of knowledge management, process improvement and human – machine interaction to improve the quality of work product and reduce reputational risks.