However, it is important to keep in mind that while ChatGPT is impressive, it is not reliable, even with regard to straightforward legal questions, let alone nuanced factual and legal scenarios. As we discovered when testing it with some legal questions, there are major limitations to its abilities. A world in which technologies like ChatGPT replace lawyers and tried-and-tested legal research methodologies and databases, seems far away. As mentioned in its frequently asked questions, ChatGPT 3.5 is not connected to the internet and its knowledge is limited to events before 2021. It can also occasionally produce incorrect or biased responses, which could have dire consequences.

Let us limit this article to one example as a newer version of ChatGPT has just been released and we look forward to testing it out and showing our readers the results.

Example

M, a legal professional working at a reputable law firm, was recently tasked by his senior, D, to conduct research on the term “adequate consideration for the issue of shares” under South African law – in the context of section 40 of the Companies Act 71 of 2008 (Companies Act) and a matter involving the issuance of shares by a company’s board. M was further instructed to find South African judgments around this topic.

Having heard about the impressive capabilities of ChatGPT, M decided to put ChatGPT to the test.

After posing his first question to ChatGPT, M was amazed by the accuracy and speed of the response he received. With just a few keystrokes, ChatGPT was able to provide a comprehensive overview of the legal concept of “adequate consideration for shares” under South African law.

Here are the results:  

This lightning-fast response left M in awe, prompting him to question why he had not been using ChatGPT for all his research tasks.

Feeling emboldened by ChatGPT’s impressive capabilities, M decided to push the boundaries further and ask ChatGPT if there were any relevant South African judgments on the topic at hand. In a matter of seconds, ChatGPT produced a list of case law with concise summaries and citations.

Here are the results:

M wants to go a step further. Instead of searching for these judgments in the relevant case law reports, he decided to ChatGPT it and get links straight from there. In a rather poor effort, M also asks if these judgments are correct.

Here are the results:

M was amazed by ChatGPT’s responses, which enabled him to provide D with information relevant to the topic at hand with links to the relevant South African judgments in a matter of minutes. Excited to showcase the power of ChatGPT to D, M eagerly collated and shared the information and links with D, confident in the accuracy of the information provided. D trusts M and decides to review the information collated by M an hour before the deadline promised to his client.

But, much as it started occurring to the editor in the acclaimed true-life film Shattered Glass, instinctively something already does not seem credible about ChatGPT’s report. Two of the cited cases were handed down some years before the new Companies Act was passed into law in 2008 (and it only came into force on 1 May 2011) – and the concept of “adequate consideration” was not used in the previous Companies Act. Could those judgments really have dealt with the issue, or perhaps they did so in a different but analogous context, to give it the benefit of the doubt? Upon review, D finds the following:

• While the first answer is not bad as a general summary for “adequate consideration”, the test for it is actually largely objective rather than subjective and is based on commercial justifications of the transaction in all the circumstances, and assuming that the parties would be acting in good faith and at arm’s length. Logically the other, diluted shareholders of the company would be severely, and absurdly, prejudiced if they could not challenge the quantum of the subscription consideration on objective grounds. In fairness, perhaps ChatGPT is using the word “subjective” in the sense that the adequacy of the consideration is to be determined on a case-by-case basis with reference to the particular company and its facts, which is unobjectionable.
• The principles as described in the cited judgments seem correct, but after reviewing the relevant case law reports, D finds to his horror that these judgments do not exist.
• The links provided by M either lead to a random pages or judgments which have nothing to do with the topic at hand.

In law, accuracy and attention to detail are paramount, and relying solely on the ChatGPT shortcut has led to embarrassment and disaster. M gets a call and is told that these judgments do not exist. M is frustrated and seeks an explanation from ChatGPT.

Here are the results:

While blind reliance on ChatGPT’s responses can lead to disastrous outcomes, the question still remains: Should you ask your lawyer or should you just ChatGPT it? Well, why don’t we ask ChatGPT itself?

It is evident that artificial intelligence tools like ChatGPT can be helpful in providing general information on diverse topics, including legal matters. However, when it comes to complex questions and applying the law to your specific facts, there is clearly no substitute for a lawyer with experience. Hardly an unexpected conclusion – but nevertheless a relief for lawyers.

So, use ChatGPT for general information, but take advice from professional advisers.

Stay tuned for part two.