By: Lara Kimmel
April 7, 2023
In an unprecedented development for artificial intelligence, OpenAI’s newest version of its chatbot, ChatGPT-4, has reached incredible achievements that the earlier versions could not attain – including passing the Uniform Bar Examination in the top 10th percentile. The news has both excited and worried many lawyers, but what does this actually mean for the bar exam and the legal profession as a whole?
First, what exactly is the ChatGPT-4, and how did it manage to pass the bar exam? ChatGPT was developed by San Francisco-based startup, OpenAI, a company co-founded in 2015 by Elon Musk and Sam Altman, with the first version of ChatGPT originally launched in November 2022. GPT is the acronym for Generative Pre-trained Transformer, and as the chatbot explained, this means that it “uses deep learning techniques to generate human-like responses to text inputs in a conversational manner.” Further, when asked to describe itself, ChatGPT reported that it is “an AI-powered chatbot developed by OpenAI, based on the GPT (Generative Pretrained Transformer) language model.” According to OpenAI, the newly released version of ChatGPT, ChatGPT-4, is different from previous versions because of its advancements in three key areas: creativity, visual input, and longer context. Additionally, this next-generation model can now receive and respond to images.
ChatGPT-4’s proposed wondrous abilities were put to the test when it took all sections of the July 2022 bar exam. Shockingly, it passed by a significant margin with a combined score of 297. This score is so high that it almost reaches the 90th percentile of test-takers. This incredible score even surpasses the “high threshold of 273 set by Arizona.”
If artificial intelligence bots can pass the Uniform Bar Examination, does this mean AI is smart enough to be a lawyer? Not exactly. ChatGPT is “an artificial intelligence trained on enormous troves of data, from webpages to scholarly texts, which then uses a complex predictive mechanism to generate human-sounding text.” Put simply, just because ChatGPT-4 could pass the Uniform Bar Examination does not mean it would know how to effectively practice the law. Rather, the chatbot efficiently knows how to cram and regurgitate information, which is exactly what makes it successful at taking exams. Thus, ChatGPT-4 prepared for the exam when it was trained on a massive dataset of past exams, study materials, legal cases, and more, leading to its incredible passing score. And the bar exam is not the only exam the chatbot passed; a list of all exams GPT-4 has taken can be found here.
ChatGPT’s success in passing the bar exam shows that the chatbot could be an essential asset for lawyers in the future. While ChatGPT is not advanced enough to give actual legal expertise, nor can it understand the ethics of practicing law, its ability to pass the bar exam proves that it is sufficient at producing legal analyses. As the chatbot stated, “ChatGPT will impact law practices in the very near future by offering AI-powered legal research assistance, document drafting, and contract analysis tools that can save lawyers significant amounts of time and effort. ChatGPT can also help lawyers improve their legal writing skills by providing suggestions for clearer and more concise language. As AI technology continues to improve, ChatGPT will become an increasingly valuable tool for lawyers looking to streamline their workflows and improve the quality of their work.” For example, the chatbot can streamline time-consuming tasks such as legal research and discovery, allowing more time and energy for legal professionals for high-value, complex work. It can also give great templates for rough drafts of briefings and contracts.
While ChatGPT will be a great asset for the legal profession in the future, there are clearly some potential fallbacks. So, what can’t ChatGPT do for the legal profession? First, all of its outputs are derived from previously written work, so there are copyright issues. Further, it does not understand the role of ethics in the profession, and there are ethical concerns in general about artificial intelligence giving legal advice. Lastly, it might not be able to fully comprehend nuanced cases. Thus, while ChatGPT-4 can aid lawyers in their practice, it is obvious that artificial intelligence will not be able to replace lawyers any time in the foreseeable future, as some lawyers seem to fear.
In conclusion, ChatGPT-4’s ability to pass the bar exam is a significant milestone in the development of artificial intelligence for the legal community, with implications extending far beyond simply passing the exam. While artificial intelligence and chatbots are not going to replace lawyers, it can supplement and renovate the way legal professionals work in the future. There are certainly potential risks, as well as high-level tasks that the chatbot cannot undertake, but the possible automation of routine tasks proves that this innovation will transform the legal profession.
