Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

🤗 Upvotes: 61 | cs.AI, cs.MA, cs.SE

Authors:
Jiajun Chen, Haoyu Wu, Mingda Jia, Xihui Liu

Title:
Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

Arxiv:
http://arxiv.org/abs/2610.08621v1

Abstract:
Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience-oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding-native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI-based collection. The Reviewer uses carefully designed trajectory-based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game-specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state-of-the-art overall performance of 77.89 on GameCraft-Bench. On GameASG-Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same-model baseline, and the highest mean runtime-check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Code is coming soon.

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to Daily Paper Cast.

Evan: Today, we're diving into a paper from the Hugging Face daily paper list dated October 8, 2026, which has garnered 61 upvotes.

Ashley: The title of the paper is 'Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness.'

Evan: This paper is authored by Jiajun Chen, Haoyu Wu, Mingda Jia, and the corresponding author, Xihui Liu, from HKU MMLab and The University of Hong Kong.

Ashley: Alright, let's delve into the introduction of this paper.

Recent advancements in coding agents have made it possible to rapidly develop runnable games.

However, creating games that are not only functional but also provide an enjoyable player experience remains a challenge.

Evan: Exactly, Ashley.

While coding agents have proven efficient in generating playable games, simply running without errors doesn't ensure that players will find these games engaging or fun.

Ashley: Right.

This paper introduces the Recursive Game Creator, aiming to move beyond the creation of rough prototypes to develop entertaining and engaging games.

It leverages a recursive development process involving four distinct components: Designer, Builder, Player, and Reviewer.

Evan: The Designer’s role is fascinating.

It starts with user instructions and feedback from the Reviewer, translating them into detailed plans including gameplay mechanisms, storylines, feedback systems, and visual assets.

Ashley: The Builder then takes these plans and implements them, utilizing external tools to build the game’s backend, organize audiovisual assets, and manage the overall project.

Unlike previous workflows that relied on GUI agents for testing, this system introduces a Coding-Native Player.

Evan: This Coding-Native Player constructs and executes reusable policies through programmatic interfaces, effectively collecting diverse gameplay trajectories.

This method mitigates evaluation bias that often arises from slower, GUI-based collection methods.

Ashley: Exactly.

The Player’s approach involves writing executable policies, which allow for frequent and diverse interaction with the game through structured interfaces, avoiding the latency issues tied to visual interpretation for every action.

Evan: And then, the Reviewer steps in to evaluate the game's performance.

Utilizing these trajectories, it conducts comprehensive game-specific diagnoses, including difficulty assessments, bug discoveries, and playtime recording.

Ashley: The Reviewer doesn’t stop at just gameplay metrics.

It integrates visual evidence and explicit textual preferences provided by the users to gauge games against specific criteria.

Through this thorough evaluation, the Reviewer then offers feedback for the next iteration, closing the recursive loop.

Evan: It's a robust system.

In their evaluations, the Recursive Game Creator demonstrated significant improvements.

For instance, on the GameCraft-Bench, it achieved a state-of-the-art overall performance score of 77.89.

Ashley: Indeed, Evan.

Additionally, on the GameASG-Bench, it achieved a strict task success rate of 53.2%, which is a 34.1% improvement over the same-model baseline.

It also recorded the highest mean runtime-check pass rate at 93.4% among compared methods.

Evan: A user study also showcased longer playtime and higher ratings for games developed using this recursive harness method.

The study highlights that not just functionality, but player engagement and enjoyment are considerably improved.

Ashley: With these contributions, the Recursive Game Creator connects multi-round game design with user-directed customization, supporting product-level game evolution.

Evan: And that brings us to the end of the Introduction section of this compelling paper.

Evan: Moving right along, let's delve into the Method section of the Recursive Game Creator paper.

Ashley: Sure, Evan.

The Method section outlines the structured approach used by Recursive Game Creator to develop engaging games.

This involves a continuous loop with four main roles: Designer, Builder, Player, and Reviewer.

Each role contributes to refining the game iteratively, creating a recursive harness.

Evan: Alright, let's start with the Designer.

What exactly does the Designer do?

Ashley: The Designer expands user instructions into comprehensive plans.

These plans detail gameplay mechanisms, storylines, feedback systems, and visual asset requirements.

The designer’s job is essentially to translate user requirements and feedback into a structured blueprint for the game.

Evan: And then, these plans are handed over to the Builder, correct?

What are the Builder's responsibilities?

Ashley: Exactly.

The Builder takes these detailed plans from the Designer and transforms them into a playable game candidate.

This involves implementing the code, organizing audiovisual assets, performing targeted checks, and preparing the overall project.

The Builder’s process goes from drafting the game to integrating all required assets into an improved version.

Evan: I see.

So the Builder effectively brings the Designer's vision to life.

What's next in the loop?

Ashley: Next is the Player, which is a notable piece in this method.

The Player constructs executable policies through programmatic interfaces rather than relying on graphical user interfaces.

This Coding-Native Player collects diverse gameplay trajectories by observing game state, choosing valid actions, and repeating until completion.

This separation allows for fast, efficient, and varied interaction with the game.

Evan: That sounds very efficient.

What advantages does this coding-native approach bring compared to traditional GUI-based testing?

Ashley: Great question.

The coding-native approach avoids the latency issues of visual interpretation at every step, enabling high-frequency control and effective trajectory collection.

It simulates a range of behaviors and strategical gameplay, which helps in identifying potential points of failure or stagnation more rapidly than GUI-based methods.

Evan: And these gameplay trajectories are then assessed by the Reviewer?

Ashley: Correct.

The Reviewer analyzes these trajectories along with visual records to evaluate the game's performance against various criteria.

It considers the difficulty level, visual appeal, and overall playability.

Moreover, it incorporates explicit user input and inferred preferences to provide comprehensive feedback for further improvements.

Evan: Interesting.

Can you elaborate on the types of assessments the Reviewer conducts?

Ashley: Certainly.

The Reviewer uses shared signals such as success rate and playtime metrics to estimate playability, challenge level, and overall fun factor.

Additionally, it employs game-specific rubrics tailored to the genre, such as the responsiveness of controls in a racing game or the coherence of dialogue in a visual novel.

It also examines visual evidence like screenshots to assess how well the game communicates its state, actions, and feedback.

Evan: That seems thorough.

Does the Reviewer directly influence the next round of design?

Ashley: Exactly.

The Reviewer translates its findings into structured feedback, specifying revision priorities and desired experiential improvements.

This feedback loop ensures each game iteration is guided by both the playtest results and user preferences, enhancing the overall quality progressively.

Evan: All these evaluations must require a robust testing framework.

How does the Player handle the collection of gameplay trajectories?

Ashley: The Player constructs diverse policies that cover various gameplay strategies, skill levels, exploration behaviors, and risk tolerance.

These policies are executed as separate trials, generating multiple attempts that capture comprehensive interaction data.

This robust framework helps in running repeated trials efficiently and gathering actionable reports for the Reviewer.

Evan: That definitely enhances the testing process.

What kind of data does the Player collect?

Ashley: The Player collects trajectory data comprising game states, actions, events, progress metrics, elapsed time, and final outcomes.

This structured dataset is used to diagnose gameplay behavior, identify points of failure or stagnation, and gauge the overall appeal of different game features.

Evan: The integration of various roles within this recursive loop sounds quite seamless.

How does this contribute to the success of iterative game development?

Ashley: The seamless integration of Designer, Builder, Player, and Reviewer roles facilitates a closed-loop refinement process.

Each role's input is indispensable for continuous game improvement.

By systematically addressing and implementing feedback across multiple rounds, the overall quality and user satisfaction of the game are consistently elevated.

Evan: The results certainly support that.

For instance, the paper indicates substantial improvements in their benchmarks, right?

Ashley: Indeed.

The Recursive Game Creator demonstrated remarkable performance on GameCraft-Bench and GameASG-Bench, surpassing previous methods.

It also showed higher task success rates and runtime-check pass rates, affirming the effectiveness of their recursive refinement method.

Evan: These comprehensive evaluations and positive user studies highlight the potential of experience-oriented game refinement.

It's clear that the Recursive Game Creator addresses the critical challenge of moving from functional prototypes to engaging and playable games.

Ashley: Evan.

By meticulously connecting policy execution, trajectory analysis, and preference-driven feedback, this method showcases significant strides toward achieving product-level game development.

Evan: That concludes the intricate Method section of this fascinating paper.

Evan: Let's dive into the Experiment and Results section.

Ashley: Sure, Evan.

The paper evaluates Recursive Game Creator on two primary benchmarks: GameCraft-Bench and GameASG-Bench, focusing on the quality of generated games and their adherence to specific requirements.

Evan: Alright, starting with GameCraft-Bench, what were the key findings?

Ashley: GameCraft-Bench assesses games based on various dimensions, including mechanics, content depth, functional visuals, and art.

The Recursive Game Creator was tested through three refinement rounds on 45 selected tasks.

Evan: What were the scores across these rounds?

Ashley: In the first round, the Recursive Game Creator achieved an overall score of 72.70.

This improved to 75.35 in the second round and reached 77.89 in the third round.

The paper highlights a steady improvement in categories like action, timing, and strategy simulation, with action showing the largest increase from 67.96 to 77.62.

Evan: So they saw consistent improvements across multiple dimensions.

What might explain the substantial gains in the action category?

Ashley: The paper suggests that action-game quality improvements depend on coordinated changes to controls, collision handling, combat feedback, and progression.

These elements require several playtest and revision cycles to fine-tune, leading to larger gains in later stages.

Evan: Got it.

Now, how did Recursive Game Creator perform on the GameASG-Bench?

Ashley: GameASG-Bench tests browser games for source code and behavior requirements.

It includes checks like L1 for source code validation and L2 for runtime checks.

Recursive Game Creator passed 25 of 47 tasks, achieving a strict task success rate of 53.2%.

Its mean L1 and L2 pass rates were 98.3% and 93.4%, respectively.

Evan: Impressive.

So they surpassed the task success rates and runtime-check pass rates of previous methods.

Ashley: Yes, and this represents a 34.1% improvement over their same-model baseline.

The harness showed particularly high scores in P0, P1, and P2 checks.

Evan: Interesting.

The paper also includes qualitative analysis, right?

Ashley: That's correct.

They examined the evolution of several games through repeated revisions to illustrate how the recursive harness improves game quality.

For instance, they showcased Twilight Front, a multiplayer online battle arena (MOBA) game, which added features like fog-of-war displays and expanded hero rosters over multiple versions.

Evan: It sounds like the qualitative analysis provides concrete examples of how game features evolve.

What other examples did they discuss?

Ashley: They also looked at KAZE, a racing game, which improved its race conditions, added new career garage features, and various car liveries.

Another example is See You Tomorrow, a visual novel game, which enhanced character dialogues, player choices, event art, and menu designs.

Evan: These examples certainly demonstrate the harness's impact on different game genres.

How about the performance of the coding-native Player?

Ashley: The coding-native Player was assessed for its efficiency in exploring game scenes.

In one experiment, it navigated a first-person shooter (FPS) test scene, capturing 97.1% grid coverage including building interiors through 48 recorded policy rollouts.

The GUI routes shown were illustrative, and the setup highlighted the comprehensive exploration capabilities of the coding-native Player.

Evan: The high coverage in the FPS scene underscores its effectiveness.

What insights were drawn from the user study?

Ashley: User studies measured aspects like controls, playability, depth, and art on a scale of 1 to 10, along with average playtime.

Over three refinement rounds, user ratings improved significantly.

Controls rose from 2 to 8, playability from 1 to 7, depth from 3 to 6, and art from 2 to 8.

Average playtime increased from 1.5 to 12.41 minutes.

Evan: Clearly, the iterative refinement had a positive impact.

Any final thoughts on the results?

Ashley: Overall, the results underscore the strength of the Recursive Game Creator in advancing game quality through its systematic, experience-oriented approach, making both functional and appealing games.

Evan: And that brings us to the end of the Experiment and Results section.

Evan: Let's shift our focus to the Related Work section of the paper.

Ashley: This section places the Recursive Game Creator within the broader context of coding agents and game development frameworks.

Evan: So, where does the discussion begin?

Ashley: It starts with the applications of coding agents.

Coding agents have expanded beyond traditional software development, being applied in diverse areas such as video creation and embodied tasks.

Evan: Interesting.

Can you give us a few examples from these areas?

Ashley: Sure.

For video creation, one example is Code2Video, which integrates planning, Python code generation, and visual review to enhance educational videos.

Another example is VideoAgent, which combines generated code with slides and narration to create scientific videos.

Evan: It’s fascinating how coding agents are branching out.

What about their role in gaming?

Ashley: In gaming, coding agents are used to generate and refine game programs.

MetaGPT and ChatDev illustrate collaborative frameworks where agents manage planning, implementation, and testing.

GameGPT extends this by incorporating role-based collaboration specifically for game development.

Evan: How does the Recursive Game Creator build on this foundation?

Ashley: Recursive Game Creator advances these concepts by connecting policy generation, trajectory collection, experience, and preference analysis in an iterative loop aimed at refining gameplay quality.

Evan: What themes are explored in the context of game development?

Ashley: Several themes are explored, including the use of agents in workflows like Unreal Engine and the creation of reusable project templates, as seen in AutoUE and OpenGame.

These systems focus on the initial generation and debugging of game components.

Evan: And how does Recursive Game Creator differ from these approaches?

Ashley: Recursive Game Creator goes further by integrating policy execution and feedback refinement, making the quality and customization of gameplay explicit targets for iterative improvement.

It’s not just about generating runnable artifacts but evolving them to enhance player experience.

Evan: That’s a crucial distinction.

How is feedback refinement handled in other systems?

Ashley: Feedback refinement is another key area.

Systems like Self-Refine and Reflexion use feedback to revise outputs or improve subsequent attempts.

There are also approaches like AFlow and EvoMAC that optimize agent workflows and collaboration.

Evan: How does Recursive Game Creator incorporate feedback?

Ashley: Recursive Game Creator uses structured preference feedback linked to specific gameplay trajectories to guide the next round of revisions.

This ensures the game aligns closely with user preferences and inferred experiences.

Evan: Could you elaborate on the concept of preference alignment in this context?

Ashley: Preference alignment involves tailoring the optimization objectives to human preferences.

In other systems, this is achieved through mechanisms like Trajectory comparisons to train reward models or RLHF, which stands for Reinforcement Learning from Human Feedback.

Evan: And in games?

Ashley: In games, experience-driven procedural content generation connects player modeling to content adaptation.

Similar approaches are seen in systems that generate racetracks based on targeted affective patterns, aiming to align the in-game experiences with player preferences.

Evan: So, Recursive Game Creator effectively builds on these principles to refine games in a structured, iterative manner.

Ashley: Exactly.

Its recursive harness systematically integrates user feedback and policy-based testing to produce games that not just run correctly but are also engaging and enjoyable.

Evan: It's clear that the Recursive Game Creator taps into a rich vein of existing research, enhancing and extending it.

Ashley: Yes, by combining elements from diverse applications and focusing on the iterative refinement of player experience, it represents a significant step forward in the domain of autonomous game development.

Evan: That wraps up our discussion on the Related Work section.

Let's move on to the Conclusion, where we'll see how all these elements come together.

Evan: We’ve covered a lot of ground today.

Let's now summarize the key contributions and takeaways from the paper 'Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness.'

Ashley: Sure, Evan.

The Recursive Game Creator sets itself apart by advancing the creation of playable games into engaging, experience-oriented products.

This recursive harness iterates through development stages involving a Designer, Builder, Player, and Reviewer.

Evan: It's impressive how the Designer translates user instructions and feedback into detailed game plans.

The Builder then brings these plans to life, ensuring the game is not just playable but visually polished and coherent.

Ashley: Exactly.

And adopting a Coding-Native Player allows for efficient and diverse gameplay interaction through executable policies.

This fast, structured interaction model overcomes the limitations of GUI-based testing, collecting richer gameplay data.

Evan: The Reviewer then steps in, combining this data with visual evidence and user preferences to evaluate the game comprehensively.

This feedback loop is crucial for continuous improvement.

Ashley: Indeed.

The Recursive Game Creator demonstrated state-of-the-art performance on both GameCraft-Bench and GameASG-Bench, showing significant improvements in game quality, task success rates, and runtime checks.

Evan: The user studies also highlighted increased engagement, with higher ratings and longer playtimes across iterative refinements.

This system ensures games are not just functional prototypes but engaging experiences for players.

Ashley: By linking policy execution, trajectory analysis, and preference-driven feedback into a seamless loop, this approach contributes significantly to autonomous game development, enhancing both functionality and player enjoyment.

Evan: That’s a great summary, Ashley.

Thank you to our listeners for joining us on this deep dive into the Recursive Game Creator.

Ashley: We hope you found this episode insightful.

Stay tuned for more discussions on the latest in AI, NLP, and computer vision research on Daily Paper Cast.

Evan: Until next time, make sure to explore new papers daily, and we’ll be here to break them down for you.

Have a great day!

Goodbye!

Ashley: Goodbye, everyone!