AI has taken things to a whole new level and it’s being used everywhere. A modder has taken advantage of ChatGPT in a very unusual way by giving non-player characters (NPCs) the ability to generate new dialogue on the spot, thus having conversations with players in real time.
The modder named Art From The Machine, modded this in the VR edition of the iconic 2011 video game Skyrim and has made a very interactive virtual world. Although the voices are far from convincing, it is very exciting to have conversations with computer-controlled characters and extend the limited dialogue that the NPCs have in the game.
The modder has done this by plugging in a text-to-speech modulator called xVASynth and OpenAI’s own speech-to-text model Whisper, the modder was able to converse with the in-game characters by simply speaking into their mic, as demonstrated in a YouTube video.
“It seems to be a well-crafted iron sword with a soul gem embedded in the hilt,” NPC Ulfberth War-Bear told the player when the modder asked him to describe a sword. “The enchantment on it allows the wielder to capture the souls of their enemies.”
Not only did the NPCs were able to tell when a certain shop was open and when it was going to close, but they were even able to remember past events and conversations which were already held.
“I have a basic memory system set up where I ask ChatGPT to summarize the conversation on exit to help condense it down for future prompts,” Art From The Machine explained in a post on Reddit.
“There are much more sophisticated tools out there to handle memory though such as Langchain which I am hoping to implement in the future,” the modder added.
Art From The Machine still wants to perfect the mod before making it available to the public.
It is surely very exciting to see AI being incorporated into video games as well in a fun and interactive way which all gamers had wished to see at some point in their lives. AI could thus enhance video games in many more ways besides just allowing virtual characters to have an open and free conversation with the players.