Published on Dec 22, 2023 As we pass the one-year mark since OpenAI’s ChatGPT took the technology and broader world by storm, there remain many questions about the impact Generative AI and similar tools will have on the way we work. The legal industry is in many ways at the forefront of this wave, as it is directly impacted by these rapidly evolving technologies and called upon to help make sense of the myriad questions they raise, e.g., issues around copyright, ethical use, and confidentiality.
While the answers to those questions will evolve with the technology, it is apparent that the legal industry faces a sea change in how much of the day-to-day work of lawyering takes place. This fact is placed in stark relief by the recent study by University of Minnesota Law School professors that sought to quantify the effect of GPT-4 on several basic legal tasks, including drafting a complaint, a contract, a section of an employee handbook, and a client memo. The authors reached multiple conclusions. First, they found that using GPT-4 had a mild positive impact on the quality of the work performed. More importantly, doing so consistently decreased the time it took to complete the tasks. In other words, the use of GPT-4 or similar technologies may not (currently) increase the quality of the ultimate work product but can significantly reduce the time it takes to arrive at the same end product.
As a case in point, I asked GPT-4 to summarize the study by pasting the text directly into ChatGPT. You can see the time it takes and the resulting product in the video below.
The study also demonstrated that the reduction in time and increased quality was more pronounced among participants with lower skill levels, suggesting an inverse relationship between skill level then benefits of using GPT-4 (in this study) and potentially in other tools if this relationship is borne out in other research. Participants whose performance on certain tasks was expected to be lower instead increased relative to those with a higher expected level of performance. The authors concluded that “[GPT-4] seems to benefit the worst performers the most, providing little or no benefit to top performers.” This simple observation may have significant ramifications for the legal industry, where experienced attorneys are at a higher premium, and those performing entry-level work are doing so in conjunction with an AI. Even now, lawyers and legal professionals are often selected based on criteria, e.g., demonstrated judgment and professional relationships, that comes with experience rather than the ability to create meticulously crafted written work products. In fact, at a recent edition of the Masters Conference, several in-house attorneys referenced conversations they had with outside counsel where they indicated that time and budgetary constraints made quick answers based on experience – even if it was “C+ work”—more valuable than slower conclusions presented in an A+ format. This divide is poised to become even more stark as generative technologies increase in use.
The dual observed effects of overall task time reduction and unbalanced increase in quality are likely to be most pronounced in the most text-heavy aspects of the legal profession – namely research and drafting, which can also be some of the most time-consuming aspects. Even though those tasks are often assigned to relatively junior lawyers who may not carry the same hourly cost as more experienced counsel, the cumulative impact of such work may quickly add up. Those cumulative costs on tasks that can be completed in less time with less skill and produce results with similar or greater quality promptly lead to questions of efficiency and even ethical billing practices. Corporations and in-house counsel are already pressuring law firms to reduce costs, and the writing may be on the wall for any hours spent on research and drafting tasks not assisted by an appropriate generative AI tool.
So what does this mean for us in the discovery corner of the legal profession in the here and now? Opportunity. For attorneys and non-attorneys alike, the tools available to satisfy discovery obligations may level up meaningfully (and some have already). It could also unleash additional creativity in the ways technologies interact with data. Imagine, for instance, the ability to ask plain-language questions of your dataset—a true game-changer in many matters. Or, for those who have been frustrated by the black-box aspect of some current predictive coding tools, consider using a technology that can both score documents in a data set and receive a brief explanation as to WHY documents were scored a certain way. Such a tool would offer much-needed transparency to enhance a party’s overall discovery process and substantiate its positions to litigation adversaries and courts.
With the advent of generative AI tools and other technologies, lawyers’ ethical duty of competence may arguably broaden to include an analysis of whether these technological advances should be used to pursue client goals. To better understand available options, lawyers might consider actively seeking education opportunities, whether through engaging experts, attending industry conferences and webinars, or just experimenting with the tools. Before using tools, though, lawyers should consider seeking and obtaining client approval. In addition, counsel should exercise care with both client data and information produced by litigation adversaries. For example, many currently available tools, especially those offered for “free,” may ingest any data users provide them and use that data for training (or even provide snippets of it) when generating information for other users. This is almost certainly a violation of the ethical duty of confidentiality and likely many actual confidentiality agreements, NDAs, etc. Many companies (Samsung perhaps the most loudly) prohibited their employees from using these tools lest they inadvertently reveal trade secrets or other proprietary information.
Fortunately, not all tools are configured this way, and even those that are, often offer architected paid tiers to maintain confidentiality. These offerings will likely form the basis for emerging technologies utilized in the legal sphere, freeing lawyers to focus on the higher skill aspects of the profession, or as a client in Texas once phrased it, “let the lawyers do the lawyering.”
Choi, Jonathan H. and Monahan, Amy and Schwarcz, Daniel, Lawyering in the Age of Artificial Intelligence (November 7, 2023). Minnesota Legal Studies Research Paper No. 23-31, Available at SSRN: https://ssrn.com/abstract=4626276 or http://dx.doi.org/10.2139/ssrn.4626276
