NVIDIA Unveils First On-Device Small Language Model for Enhanced Game Character Roleplay

At Gamescom 2024, NVIDIA announced the deployment of its first on-device small language model (SLM) aimed at enhancing the conversational abilities of game characters, according to the NVIDIA Technical Blog. This innovation is set to debut in Amazing Seasun Games’ ‘Mecha BREAK’, promising a more dynamic and immersive gameplay experience on NVIDIA GeForce RTX AI PCs.

NVIDIA ACE Introduces SLM for Roleplaying

NVIDIA ACE, a suite of digital human technologies, now includes the first NVIDIA SLM, Nemotron-4 4B Instruct. This model is available as an NVIDIA NIM for both cloud and on-device deployment, offering game developers a streamlined way to integrate advanced language models into their games. NVIDIA NIM provides a set of microservices for faster deployment of these models on any cloud or data center.

Nemotron-4 4B Instruct is optimized through distillation, pruning, and INT4 quantization, minimizing VRAM usage to around 2 GB and significantly reducing the time-to-first-token compared to larger language models (LLMs). The model, distilled from the Nemotron-4 15B LLM, is fine-tuned with instructional prompts for better performance in specific tasks, enhancing role-play, retrieval augmented generation (RAG), and function-calling capabilities.

This technology allows game characters to more intuitively comprehend player instructions, respond accurately, and perform relevant actions. Developers can integrate Nemotron-4 4B Instruct NIM into their digital human technology pipeline via the cloud or leverage over 100 million NVIDIA GeForce RTX-powered PCs and laptops for on-device inference. Additionally, the Unreal Engine 5 ACE sample application has been updated to support on-device plugin integration.

First Game Premiere with NVIDIA ACE NIM

‘Mecha BREAK’, developed by Amazing Seasun Games, a subsidiary of Kingsoft Corporation, is the first game to showcase the NVIDIA Nemotron-4 4B Instruct NIM running on-device. The game leverages NVIDIA’s Audio2Face-3D NIM and Whisper, OpenAI’s speech recognition model, for facial animation and speech recognition, while Elevenlabs provides the characters’ voices through cloud services.

In this demo, NVIDIA ACE and digital human technologies enable players to interact with a mechanic character, who assists in mech selection, customization, and preparation for battles, enhancing the overall gaming experience.

Game Developers Explore Latest Digital Human Technologies

NVIDIA ACE and digital human technologies are expanding their influence in the gaming industry. Perfect World Games is advancing its NVIDIA ACE and digital human technology demo, ‘Legends’, with new AI-powered vision capabilities. In this demo, the character Yun Ni can see gamers and identify people and objects in the real world using the computer’s camera, powered by ChatGPT-4o, adding an augmented reality layer to the gameplay.

These advancements promise a new level of immersion and accessibility for PC games. For more information, visit NVIDIA ACE and download the NIM to start building game characters powered by generative AI.

Image source: Shutterstock