The Right and Wrong Ways for Sony to Use AI in PlayStation 5 Games

Like many companies in the tech world today, Sony and its PlayStation brand are exploring how artificial intelligence (AI) can boost the gaming experience. On the positive side, Sony is looking to enhance the visual quality and performance of its games for players who have invested in the PlayStation 5 Pro, which is commendable. However, the company seems overly eager to incorporate AI-powered non-player characters (NPCs) into games, with results that leave much to be desired.

A recently leaked video revealed Sony's attempt to introduce generative AI in a demonstration featuring Aloy from Horizon: Forbidden West. In this video, Sharwin Raghoebardajal, the director of software engineering at Sony, interacted with a digital version of Aloy using voice prompts. Fortunately, the AI did not attempt to mimic voice actress Ashley Burch, but instead delivered responses in a stiff and robotic manner, with awkward mouth movements for each word.

During the demonstration, when asked about the storyline of Horizon, the AI version of Aloy accidentally revealed major plot points from the first game. The timing of the responses was off, often taking several seconds before replying. The AI explained its surroundings, including details like the time of day.

For instance, AI Aloy declared, "It's the afternoon, and it's quite hot under the sun," followed by an off-putting pause.

The AI's speech was generated using OpenAI's Whisper model combined with Llama 3 from Meta and OpenAI's GPT-4 large language model. Facial animations were created with Sony's own Mockingbird suite, a model not commonly accessible. All this technology was integrated into the Decima engine, which powered both of the Horizon games from Guerrilla Games.

Do Players Want to Converse with AI Characters?

Raghoebardajal described this effort as "a glimpse of what's possible," but the demo did not impress viewers. The initial video was removed by a company linked to Sony, but copies can still be found on platforms like Reddit. Many have compared these attempts to Nvidia's experiments with AI NPCs over the past year, including a demo where users could convince an AI barista to reveal information.

Major players in the gaming industry are racing to embed AI into their upcoming titles, whether gamers want it or not. In a blog post from February, Microsoft discussed creating a generative AI system for game development called Muse. This tool designed a hero shooter game reminiscent of Overwatch by utilizing elements from an existing title called Bleeding Edge, which had been discontinued due to lack of interest. This raises the question: can AI really create something better?

As AI fundamentally relies on existing data, its application in the arts tends to produce derivatives rather than original content. While the AI Aloy project may be a minor initiative for Sony, it highlights broader issues in the gaming sector. Some publishers have openly expressed their intention to use AI to replicate the voices of actors like Burch, often prioritizing cost-effectiveness over quality. Gamers can discern the difference between authentic performances and those generated by AI. In fact, AI tends to perform best when used subtly behind the scenes.

AI Upscaling: A Better Focus for Sony's AI Efforts

Mark Cerny, the lead designer for the PlayStation 5's architecture, has shared exciting news about improvements in AI upscaling for the PS5 Pro. This enhancement will involve integrating AMD's new FidelityFX Super Resolution (FSR) technology into PS5 games supporting PlayStation Super Resolution (PSSR). Cerny projects that FSR 4 capabilities could arrive for the PS5 Pro by 2026, marking the next step forward in visual fidelity.

This initiative is part of a collaboration between Sony and AMD dubbed "Project Amethyst." FSR 4 will be available for a significant portion of the approximately 30 games that currently support it, including titles like the recent Marvel's Spider-Man 2. Cerny plans to incorporate PSSR with existing PS5 Pro games before bringing FSR 4 support to the PS5 itself. Previously, FSR was hardware-agnostic, but with the introduction of RDNA 4 GPU architecture, it now requires specific AMD graphics cards like the Radeon RX 9070 or RX 9070 XT.

Integrating AMD's machine-learning technology into Sony's console is no small feat, but Cerny has indicated that some of FSR 4's neural networks have already been implemented on the PS5 Pro. With a performance capability of 300 8-bit TOPS (trillions of operations per second), the PS5 Pro can match AMD’s latest GPUs, enabling it to improve graphical quality.

AI upscaling enhances lower-resolution frames by using AI algorithms to present them at a higher resolution without sacrificing performance. FSR 4 represents the first machine learning-focused advancement in the FidelityFX line. Tests have shown better performance and sharper visuals in supported games compared to previous versions. PS5 games labeled as "enhanced" should benefit from the PSSR features, potentially resulting in improved frame rates and graphical details.

As PC gaming increasingly utilizes AI upscaling technologies from companies like Intel, AMD, and Nvidia, the PS5 maintains some limitations typical of consoles, even with the Pro version. The FSR implementation will work in the background, so most players may not notice its existence, but they should experience enhancements in frame rates and visual quality.

Some gamers criticize the reliance on AI upscaling and the use of generated frames from systems like FSR and DLSS. However, advancements that make gaming more accessible and visually appealing should be seen as positive developments. If Sony channels its AI resources toward improving game quality and performance instead of developing unwanted AI NPCs, the gaming community could see significant benefits.

AI, gaming, technology