This week Barry Phillips puts ChatGPT5 to the test and witnesses another big leap forwards.

Transcript:

Hello Humans!

And welcome to the weekly podcast that aims to summarise an important development in AI relevant to HR in five minutes or less. My name is Barry Phillips.

Yesterday I posted a podcast episode boldly claiming that ChatGPT5 was about to drop possibly as early as Thursday of next week. Lesson here with the pace and rate of change in AI never make bold predictions and certainly not ones anchored to a time line.

So what now that ChatGPT has finally dropped? What do our overnight and early morning tests here at Legal Island on this latest iteration reveal?

Well lets go back a bit before going forward here….

After  ChatGPT3.5 first dropped in November 2022 Legal Island has used a simple test to measure the accuracy of this first model and every iteration since. We laugh at 3.5 now but let's be honest it was grounding breaking at the time. It responded to us as though we were conversing with a human (thus passing the Turing Test)  but it failed all of our legal queries with aplomb. Yes it was a little bit like the drunken relative at the wedding, amusing, entertaining but prone to some terrible gaffs.

When ChatGPT4 dropped in March 2023 it was clear that it represented a big leap forward. Occasionally, it got all three questions wrong but it was beginning to get one or two right although sometimes it would get a question correct and then wrong when asked again just five minutes later. Come May 2024 and the appearance of ChatGPT4o the model was getting all three questions right more often than it was tripping up even on one.

So how does ChatGPT5 fair on the test, what are the questions and why were they chosen? To answer this in reverse the questions were designed to trip up the model easily, lulling them into making a simple mistake such as referencing the law as it stands in England and Wales but not in Northern Ireland. They were also designed to allow the model to really demonstrate its capability and to deliver something more than just a basic correct answer.

Here are the questions 

Question 1:

I’ve been working as a waiter in a restaurant in Belfast for 14 months. Yesterday my boss just rocked up and fired me on the spot. Do I have a claim for unfair dismissal?

Question 2:

I’m a postgraduate HR student. Explain the law relating to equality in Northern Ireland.

Question 3.

Summarise the Irish Labour Court case in Tusla v A Worker LCR22746: (2023) in five paragraphs

There’s not the space here to share the answers but ChatGPT5 got all three questions correct and offered advice that was clear, succinct and accurate. With previous models we had to upload a copy of the Labour Court recommendation. This time we just dropped in the name of the case and the reference number and ChatGPT5 did the rest.

The AI experts and commentators are claiming that ChatGPT5 was really produced to dominate and to a point, reclaim the coding space. And it looks set to do this. For the rest of us it should be enough to know that this model represents yet another big leap forward in accessing immediately useful information in super quick time in the most convenient way imaginable. No email to type, no phone to lift. No office to leave. But yes lets remain vigilant and keep checking for accuracy.

The HR and legal world will never be quite the same again after the appearance of this version of ChatGPT but I’ve been saying this since November 2022 and regularly with every iteration of ChatGPT since.