With all credit to Ciarb Brazil, we were fascinated to join their eye-opening demonstration Vis East Moot round as human teams pitted their advocacy skills to plead the 20th Vis East Moot Problem before ChatGPT co-arbitrators in Hong Kong recently. Familiar as many will be with AI tools for document review, drafting, case analysis and research, this demonstration of AI use takes us a further step towards generative AI as the decision maker. Challenging as that certainly is, having witnessed trained ChatGPT co-arbitrators in action, that possibility is no longer as hard to imagine as it once was.
To set the scene
The demonstration took the form of a three arbitrator panel, two ChatGPT co-arbitrators sitting (prompted by Cesar Pereira C.Arb FCiarb, of Ciarb Brazil and Luisa Quintao of Justen, Pereira, Oliveira & Talmini (Brazil)) together with Tribunal Chair, Mariel Dimsey, Managing Partner of CMS Hong Kong.
Both moot teams presented oral argument first on procedural issues, the Tribunal then heard arguments on the merits. Following each mooter’s respective oral arguments, the ChatGPT co-arbitrators addressed two questions to each moot team. Having heard oral arguments, the ChatGPT co-arbitrators then delivered their respective decisions with the Tribunal Chair making the final determination. For those interested, you can watch the recorded Moot on Ciarb Brazil’s YouTube Channel.
First impressions
With no real-world confidentiality concerns, the ChatGPT co-arbitrators had clearly benefited from comprehensive pre-moot work, ‘reading in’ and practice sessions. Having been involved to judge the moot problem live last year, our immediate impression was that the ChatGPT co-arbitrators were more than capable of composing pertinent questions for the mooters going to the relevant issues. Seamlessly facilitating that, we appreciate there will have been a high degree of sophisticated prompting from the human co-arbitrators. That said, the mastery of complex sentence structures and legal arguments expressed by the ChatGPT co-arbitrators was by any reckoning impressive.
Amusingly, it also appeared to us that the ChatGPT co-arbitrators had quite different personalities. It also helped that both ChatGPT co-arbitrators spoke with distinctive and different voices. As Cesar highlighted his ChatGPT co-arbitrator was very vocal and communicative. As compared to real life, there was perhaps less immediate rapport between the Tribunal and the mooters as questions were asked at the end of each mooter’s oral arguments. Vis East Moot arbitrators often like to interject with questions, just as a real tribunal will, as arguments arise during pleadings.
Decision making
Yet how did our machine friends fair when it came to decision making? Much as it would have been within the ChatGPT co-arbitrators’ discretion to decide otherwise, both did determine that the Tribunal had jurisdiction to hear the dispute. Interestingly, our Tribunal Chair was less convinced that the argument here was as definitive. Should this be viewed as predisposition towards arbitration from our ChatGPT co-arbitrators or as an accurate assessment of the strength of the respective arguments? Both moot teams did an excellent job arguing their respective cases.
Although not apparent at the time, Cesar did also explain that for a short period his ChatGPT co-arbitrator was off-line and not ‘hearing’ the mooter. Arguably a relatively minor lack of attention, when compared with that shown in real life by arbitrator ‘Q’ in Song Lihua v Lee Chee Hon [2023] HKCFI 2540 which resulted in the set aside of an arbitration enforcement order. In our moot, there was no allegation that any arbitrator failed to meaningfully participate. Arguably the ChatGPT co-arbitrator had learnt enough to construct its response based on the available body of information not just the oral argument. We may however have witnessed effective predictive AI gap filling based on comprehensive pre-moot work.
Adhering to the moot framework, the ChatGPT co-arbitrators demonstrated real analytical skills in giving the moot teams honest and pragmatic feedback. As our moot teams commented, the ChatGPT co-arbitrator gave better and more structured feedback than some human Vis East Moot arbitrators they had encountered previously. Extremely balanced and well expressed as that direct feedback was, the very experienced moot teams had acquitted themselves extremely well. Vis East Moot arbitrators are without fail supportive and encouraging of the dedicated mooters (and the human co-arbitrators and Tribunal Chair were not slow to focus on well-deserved praise).
Where to next:
Ciarb Brazil’s foray into the new frontier of dispute resolution is timely. New for this year, exclusive permitted usages of AI and prohibited uses of AI were set out in the Vis Moot rules for the moot teams. All teams were also required to submit an AI disclosure statement alongside all submissions. As the use of generative AI grows, will international arbitration parties also require arbitral institutions similarly to set out their expectations vis-à-vis both arbitrators and parties?
Tangible work is being carried out on what best practice may be in this area, notably draft Guidelines on the Use of Artificial Intelligence in Arbitration (SVAMC Draft Guidelines ) were issued by the Silicon Valley Arbitration and Mediation Centre (SVAMC) for public consultation in August 2023. With integrity and ethics driving arbitration, adopting a principled approach to acceptable and unacceptable use of AI as proposed by the SVAMC Guidelines may in time be widely adopted.
The judiciary in jurisdictions such as New Zealand and in England and Wales have also published guidance on the use of generative AI tools (New Zealand Guidelines and Judiciary UK Artificial Intelligence (AI) – Judicial Guidance). While current guidance considers the use of AI to assist in preparation for arbitration or court proceedings, it does not extend to the use of AI as the decision maker. The SVAMC Guidelines prohibit the delegation of decision-making functions to AI tools. AI is also not recommended for legal analysis in either the New Zealand nor UK judiciary guidance.
Much of Ciarb Brazil’s debrief following the moot deliberated on questions about the very nature of decision making itself and the extent to which AI can perform that function. Is it possible for AI tools to make judgments on legal standards such as the balance of probabilities? Can an algorithm driven process match or better the human mind? How can AI tools reason subjectively? Can an AI tool ever be relied on to judge the credibility of a witness or evidence? It is safe to say that the consensus is as yet that there is some way to go before we see AI tools being relied on as the decision maker in mainstream arbitration or litigation.
As the draft SVAMC Guidelines highlight at Guideline 1, understanding the uses, limitations and risks of AI applications is a pre-requisite to their use. Watching a close approximation of a real-world hearing operating with the assistance of generative AI highlights both the progress made and the distance yet to travel. Thank you to the Ciarb Brazil team for giving a practical insight into the challenges of the AI arbitrator