AI: Will it Run Doom?

‍

‍

Artificial Intelligence (AI) is accelerating rapidly, transforming industries and redefining what technology can achieve. From healthcare to autonomous vehicles, AI is reshaping the world around us, and gaming is no exception. What was once a process that required painstaking manual coding—such as the creation of video games—is now being streamlined and even replaced by AI. An example of this shift is the recent breakthrough where AI was used to render and play the iconic game Doom in real time. This experience is more than just a novelty; it’s a significant milestone that could reshape the future of game development.

So, to fully grasp this advancement, we have to ask: "Will it run Doom?" 

‍

Will It Run Doom?

"Will it run Doom?" has been more than a question; it’s a meme and a challenge that programmers and hardware enthusiasts have tackled for decades. Originating from the 1993 release of Doom, this groundbreaking first-person shooter quickly became a benchmark for hardware performance. Over the years, people have taken it upon themselves to run Doom on increasingly unconventional devices. It's wild. Here are a few highlights:

• Doom running on calculators powered by old potatoes
• Doom on an ATM
• Doom on a pregnancy test

It's a massive rabbit hole. If you want to explore more, there is a whole subreddit about Doom on weird devices. Have at it.

This meme is significant in our cultural history and reflects a profound desire to push technology to its absolute limits. For many, Doom is more than just a game; it’s a piece of digital history and a relic from when video games were more straightforward yet groundbreaking. The ongoing challenge to run Doom on diverse devices has kept this piece of history alive, continuously reminding us of the game’s legacy. The idea of AI playing Doom isn’t just about showcasing technological prowess—it’s about connecting with a rich cultural tradition that values innovation and nostalgia.

Which brings us to AI Doom.

‍

AI Doom

The project that brought AI-generated Doom to life is called GameNGen, a collaboration between researchers from Google Research, Google DeepMind, and Tel Aviv University. GameNGen represents a novel advancement in AI by generating real-time gameplay for Doom without relying on a traditional game engine.

The creation process involved two distinct phases:

1. Training the AI Agent: A reinforcement learning (RL) agent was trained to play Doom in the first phase. The agent learned by playing the game repeatedly, collecting data on various game states, actions, and outcomes. This data laid the foundation for the next phase.
2. Diffusion Model for Frame Prediction: The data from the RL agent was then used to train a diffusion model, a type of AI that generates new content based on existing data. This model predicted and rendered the next frame in the game, using the sequence of past frames and player inputs as context. The result was a system capable of generating real-time gameplay at approximately 20 frames per second (FPS) on a single TPU (Tensor Processing Unit) v5. 

While 20 FPS might seem low by modern gaming standards, it’s worth noting that the original Doom maxed out at 35 FPS. The visual quality of the AI-generated frames was comparable to lossy JPEG compression, and human raters struggled to distinguish between AI-generated clips and actual gameplay.

Despite these impressive achievements, GameNGen is not without its limitations:

Memory Limitation: The AI model can only store about 3 seconds of gameplay memory, limiting its ability to recognize and respond to long-term relationships in the game, which affects continuity and gameplay quality (source: [The Decoder]).

Performance Constraints: While the AI can achieve 20 FPS, it falls short of the 60+ FPS standard expected in modern first-person shooters. Additionally, while impressive, the system's visual quality still has room for improvement, mainly when rendering complex scenes.

Broader Challenges: The extensive data and computational power required to train such models limit their accessibility and scalability. Moreover, the AI struggled with unexplored game areas, leading to erroneous behaviors, highlighting the current limitations of AI in fully understanding and interacting with complex environments.

These constraints underscore the challenges of using AI to replicate and generate real-time gameplay. However, they also point to the potential for future improvements as technology continues to evolve.

‍

Implications for Game Development

The development of AI Doom could signal a massive shift in how games are conceived, developed, and experienced. Traditionally, game development is a highly detailed and labor-intensive process. Developers typically rely on game engines—like Unity, Unreal Engine, or custom-built engines—that provide a framework for creating game worlds, defining physics, rendering graphics, and managing gameplay logic. These engines serve as the backbone of modern games, allowing developers to code behaviors, design levels, and create the immersive experiences players enjoy.

Game engines provide developers with tools and APIs to create and manipulate game assets. Developers write scripts in programming languages like C#, C++, or Python to control how these assets behave, interact, and respond to player input. The process is incredibly time-consuming and resource-intensive, often involving large teams of programmers, artists, and designers working together over several years to bring a game to life. Every element must be meticulously crafted, tested, and optimized, from character animations to environmental textures.

The introduction of AI systems like GameNGen could fundamentally alter this process. Rather than relying solely on human programmers to write scripts and define game logic, AI tools don’t just follow pre-programmed rules; they learn to play the game, understanding its mechanics through reinforcement learning. This knowledge is then used to predict and render the successive frames of gameplay in real time, a task traditionally handled by the game engine's rendering pipeline.

We’re beginning to see hints of a future where AI automates significant portions of the game development process. Imagine a scenario where an AI is given a high-level concept—a fantasy world with dragons and knights—and can independently generate the game world, characters, and even complex narratives. Developers will no longer need to code every interaction or design every level manually. Instead, they could guide the AI, providing broad directions and allowing it to fill in the details, potentially creating unique and personalized gaming experiences for each player.

AI could develop games that evolve dynamically in response to player actions. Modern games are mostly static builds upon release. The environments, storylines, and character interactions are fixed, with limited variation depending on player choices. AI-driven game engines could effectively make a game’s iterations infinite and ongoing. The game world could be continually generated and regenerated in response to player behavior, creating a unique game mechanism, visuals, and experiences for every playthrough. This could lead to a new genre of games—those that are never the same twice, offering endless replayability.

‍

What’s Next?

GameNGen is probably just the beginning of a significant shift in game development and AI technology. Several key areas are likely to see continued advancements.

Researchers will likely focus on improving the memory and performance of AI models like GameNGen. Enhancing the AI’s ability to store and process longer gameplay sequences will create more seamless and continuous gaming experiences. Additionally, advancements in hardware, such as more powerful TPUs, could lead to higher frame rates and better visual quality. The ultimate goal could be to create fully AI-generated games, where the entire game—story, environment, characters—is procedurally generated in real time based on the player's actions and preferences. This would represent a new frontier in gaming, where no two playthroughs are the same, offering players genuinely unique experiences.

AI-generated Doom is more than just a technological feat; it's a glimpse into a future where the boundaries between human creativity and AI get just a bit fuzzier. The implications for gaming are vast: more efficient game development, personalized gaming experiences, and entirely new genres of interactive entertainment. As AI advances, it could redefine what it means to create and play video games. It remains to be seen what is possible or if entirely AI-generated worlds will stay in the realm of science fiction.