Racing Against GT Sophy: Reflecting on Gran Turismo 7's new AI Driver
Discover more from AI and Games
Racing Against GT Sophy: Reflecting on Gran Turismo 7's new AI Driver
It was a lot more fun than I expected...
AI and Games is a YouTube series made possible thanks to crowdfunding on Patreon as well as right here with paid subscriptions Substack.
Support the show to have your name in video credits, contribute to future episode topics, watch content in early access and receive exclusive supporters-only content.
In 2022 Sony AI made headlines with the announcement of GT Sophy, a deep-learning-powered AI racer built specifically for the Gran Turismo franchise that had proven itself in closed race conditions against professional players. It's an interesting example of the surge of deep learning research trying to train AI systems to play commercial games, much like the work in StarCraft 2 and Dota 2 that preceded it.
However, despite all the media hype, Sophy wasn't included in Gran Turismo 7 when it launched on PlayStation 5, but since then Sony AI and the Gran Turismo developers Polyphony Digital have declared that they wanted Sophy to become a part of the Gran Turismo franchise and make it a fully fleshed out opponent for players.
After releasing an episode of AI and Games in 2022 that explained how Sophy works (linked to above), I've returned to Gran Turismo 7 to try my hand at racing against it. In March of 2023, Polyphony released a patch for Gran Turismo 7 that added several different versions of GT Sophy to race against in a limited-time event.
So for this episode, I tried my hand at a few races, such that I could then discuss my experiences as well as a broader analysis of this recent development. But of course, the big problem here is, I'm not very good at Gran Turismo or simulation-heavy racing games in general. So begs the question: can I actually beat Sophy even on its lowest difficulty?
Watch the Races
If you want to catch how each match turned out, watch the episode of AI and Games Plus below. Each race is timestamped in the video for you to watch.
Analysis
If you opted to skip the video, here’s a summary of my performance against Sophy:
Race 1 (Tsukuba Circuit/Beginner) - 1st Place
Race 2 (Alsace Village/Intermediate) - 4th Place
Race 3 (Tsukuba Circuit/Intermediate) - 1st Place
Race 4 (Trial Mountain Circuit/Beginner) - 4th Place
Race 5 (Suzuka Circuit/Expert) - DNQ
While my performance in Trial Mountain Circuit was down to a failure to handle the car on the final corner, all other races were the result of fairly intense showdowns against these AI racers.
My performance aside, what is interesting is what I saw when racing against the different versions of Sophy. We're seeing varying degrees of skill being exerted by these AI agents, combined with a variety in the types of the strategy being employed as well. I can certainly say that - despite being soundly defeated on multiple occasions - that I thoroughly enjoyed my time playing this GT Sophy playtest, but let's dig into the details. What were the highlights for me, and what does it all mean in the context of Sony AI's research on the project?
Now it's worth saying at this point that, given Sony AI hasn't detailed anything about this to playtest as of yet, this is purely speculation on my part. Though I'd argue it's certainly some rather educated speculation, I do know a thing or two about AI for video games after all.
Multiple Racers, Multiple Tests
So first of all, as they described when you join the playest, there are multiple versions of GT Sophy you can play off against, and that's highly evident when you stack up against them on the track. The skill level of the racers varies, and the way in which they take corners varies, or handle the gas and acceleration. Plus how they attempt to pass you can vary from more passive and distant, to aggressive and up-close. This tells me one thing straight away, in that they have been iterating on and running multiple training phases to create each of these Sophy racers.
As detailed recently in my AI 101 episode on the foundations of machine learning, it's not really plausible to crack open the brain of a trained ML agent and modify its behaviour. You have to train it to behave in a manner that you expect. In this instance, as detailed in my episode of AI and Games on GT Sophy, the system runs a deep neural network that is trained using reinforcement learning. It's not particularly easy to just rip open a neural network and make changes to it to fit a specific playstyle or strategy, so odds are the Sophy's that you see here are the end result of several training experiments under different configurations designed to achieve different results. The reinforcement learning problem will have been configured such that it aims to achieve a specific skill rating, as well as unique aspects of its racing style.
AI 'Personalities'
Digging a little more into the actual play styles, what strikes me as interesting is the different gameplay styles that specific racers evoked. We noted in several races that specific versions of Sophy would be more aggressive than others, more prone to actually bumping into me, even nudging me where it could in specific situations. Sophy Violette is a rather aggressive race driver, and it doesn't strike me as a coincidence that it was positioned at the back of the pack, given it means as players we come across that interaction pretty quickly. Compare that to Sophy Verte who seemed much more chill by comparison.
Now, these are interesting because these personalities will no doubt arise courtesy of how Sophy is trained. This isn't an unexpected outcome of developing this AI. Referring back once again to the GT Sophy episode of AI and Games, the racer is trained by factoring in specific aspects of gameplay style, be it how it handles the car, takes turns around other racers, or how close it even gets to other racers when attempting to pass. I discussed in that video how one of the big challenges Sony AI had to deal with was that Sophy is often quite aggressive and so they continually had to adjust the reward functions to actively discourage getting too close to other drivers.
For this experiment, I suspect that they've adjusted the reward functions to allow them to have more variety in how aggressive or passive they are in a myriad of circumstances. These are often subtle adjustments to the learning parameters, but it results in some very tangible differences with the racer’s behaviour.
Communicating the Mood
Oh and speaking of personalities, it was interesting to observe the emojis that are applied to each racer as you interact with them. These look like they're reacting to your behaviour, with the Sophy racers upset if you get too close or overtake them, or they look quite happy when they take the lead. This appears to be based on specific combinations of input parameters to the neural network. They could have trained an additional system to successfully communicate via these emojis that the system recognises specific input patterns that reflect specific gameplay segments, or they equally could have just rigged something together using much more simplistic means. But either way, it adds a little bit of personality to each of the racers and helps communicate to the player that they are reacting to your own actions, which is kinda neat.
Difficulty Breakthrough?
And with all that out the way, for me, one of the most exciting elements of all of this. It's clear there is a huge disparity in the skill level of the drivers. The beginner racers are competent and functional. Meanwhile, the expert racers are going to put the very best Gran Turismo racers through their paces. The fact that this exists, is actually kind of a big deal. Why? Well, let me explain.
One of the problems with adapting machine learning to games for NPCs and opponent players is how to accommodate varying skill levels. GT Sophy, as was described in my explainer video, is built to be the best Gran Turismo racer possible. When these systems are trained, it's told what the objective is, and for Sophy, it was to be a highly competent racer, which also typically evokes some aspects of racer etiquette. The end result is, without question, very impressive. But the system can only perform at the level it's been trained on, it can't drop its skill down in order to accommodate the player.
Think of it like when you're playing the likes of a racing or fighting game against someone who is perhaps younger than you or isn't as experienced. You might hold back a little, not play at your full potential, and deliberately make mistakes so as to help them enjoy the experience or even learn how to get better. That's a very natural thing to do as humans. But it's not a skill that these AI players can replicate. They can only perform at the level they've been trained to perform at.
Addressing the skill levels issue is going to be a big turning point for how machine learning AI is going to be used in games going forward. Similar grandmaster AI experiments such as the OpenAI Five for Dota2, or Google DeepMind's StarCraft II player AlphaStar are functionally useless in typical video game development because they only cater to the top 1% of players, and can't adjust their behaviour. Being able to have different versions of the same bot running at different skill levels solves that problem.
That's what makes these varying difficulties a big deal: they've clearly run the training process to create versions of GT Sophy with different skill ratings. Now I'll refrain from speculating on how they did it, given there's a myriad of approaches they could take, but I'll be interested to hear how exactly it was done. Regardless, this feels like a turning point for me in terms of the applicability of this technology, given the end result is all the more interesting. These racers are not only rather competitive but they're fun to play against, but it's clear there is a skill gap between each of the supposed difficulty levels.
A point that was mentioned by one of the pro players that I discussed in my original analysis, is how often you see the higher-level bots take the corners in slightly different ways from how you might expect. This is something I observed and quite often I was learning from them and trying to take corners differently. I mean clearly, I still have a lot to learn about Gran Turismo in general, but even mid-race I was finding I wanted to follow their path along the track given they often seemed to know something I don't. Of course, the hard part was whether I could actually do it.
Closing
Reflecting on my performance, I think it was a mixed success and given I'm not particularly good at Gran Turismo anyway. I'm not surprised that I got trounced by different versions of Sophy. But let's take the victories where we find them, I could at least beat the easiest version of the bot, though in saying that it was still a bit of a challenge at times.
Nonetheless, the performance of the AI drivers themselves is a cause for celebration and one that should be watched closely in the coming months as the developers continue to tune the systems. In time, all of the AI in Gran Turismo could conceivably be replaced by the Sophy system. Based on this limited interaction, it could prove to be a sound decision that ultimately affects the player’s experience in a positive way.