ACE Robotics and Nanyang Technological University’s S-Lab have released Mundo, a world model for building explorable and interactive environments for robot training and testing.
Mundo supports scene reconstruction, free-viewpoint exploration and spatiotemporal simulation similar to Atlas. Mundo can also incorporate physical attributes such as mass and friction into scene programs and use a physics engine to calculate motion, contact and collision.
Mundo can call agents to edit objects, attributes and actions in a scene through code, turning a small number of images into a 3D world that can be freely explored, programmed for interaction and used for robot navigation and manipulation training.
The model combines 3D scene representation, physics simulation and video generation. Combined with Astra, it creates new spatial understanding capabilities. The system first captures scenes and objects from imagery. Developers may then edit the scene and define or modify object attributes and actions via code. A physics engine calculates motion and collision, while a video model renders results using reference imagery, forming a complete process from scene reconstruction and code editing to interactive simulation. The scenes can be viewed, respond to actions and record changes, making them environments for robot training and testing.
This capability enables robots to be deployed in concrete commercial scenarios. Within ACE Robotics’ Kairos-centered world model technology stack, Mundo adds environment modeling, training data generation and simulation training capabilities to help robots complete scene adaptation and policy optimization.
Mundo expands a small number of images or videos into a continuously explorable 3D space. The same set of reference images can support multiple camera paths. For larger scenes, it can dynamically select context and continuously generate exploration footage lasting tens of seconds to minutes.
The model starts from context imagery of real places, builds a 3D proxy, determines spatial relationships among views and generates new viewpoints along specified camera paths. With one to 12 input images, it can explore furniture and spatial structures within a room. The same reference images can serve multiple paths, allowing observation positions and angles to change freely.
A 3D detector provides initial positions and extents for main objects, while a vision-language model agent further adjusts the proxy scene through rendering, comparison and correction so that scene structure corresponds to input imagery. For example, users can navigate an office from a desk to a window, and a residence from a courtyard indoors. As the camera moves, input images constrain captured layout, appearance and details, while the model infers and completes uncaptured areas. More reference views provide more ground truth and reduce reconstruction uncertainty.
The capability can also start from a generated image or painting, extending structure and visual style to new viewpoints. For real-world sites, more observations improve the model’s fidelity to the physical location; for creative scenes, a single image provides a starting point for building an explorable space. For larger environments, Mundo uses dozens of context images to support long-path generation spanning tens of seconds to minutes. It generates footage in segments along the same 3D proxy, selects relevant reference views for each segment and connects segments with overlapping frames, so local appearance and global space share a consistent spatial basis.
Mundo organizes captured viewpoints into a space that can be continuously entered and explored. For robot scene adaptation, on-site imagery can serve as an entry point for building task environments and provide a spatial foundation for subsequent navigation and interaction training.
For robots to practice within a scene, the environment must respond to actions. Opening a drawer, pushing a chair or dropping an object requires corresponding position changes and contact interactions.
Mundo can edit the world through code. Scene programs describe object names, sizes and poses and configure physical properties such as mass and friction. Scripts define actions and applied forces. A physics engine calculates responses under gravity and contact constraints, producing motion trajectories, contact locations and collision forces. Mundo then renders these changes into the corresponding scene using reference images.
Users can drag a trash can, push a monitor, pull a chair or open a drawer, and can alter object materials and gravity conditions to observe physical responses. Speed, contact points and impact forces can be displayed alongside the generated visuals.
In this process, the 3D proxy and scene programs record where objects are and what changes occur; the physics engine calculates motion; the video model produces visually consistent renders. Geometry, physics and appearance each carry distinct roles. Traceable states and computable processes for every interaction are key to transforming scenes from merely viewable to functionally usable. Mundo introduces physical interaction into reconstructed environments, providing a foundation for robots to practice actions and analyze outcomes.
Within the same customized scene, Mundo supports robot movement and manipulation, linking environment reconstruction directly to robot testing and training.
For navigation, the 3D proxy provides object categories, positions, obstacles and traversable regions, and a path planner computes routes. Mundo can generate first-person and third-person views of different robots moving along routes, linking spatial layout with what robots observe along their path.
For manipulation, Mundo establishes geometric and physical attributes for interactive objects, allowing robots of different morphologies to attempt identical tasks, such as placing an apple in a bowl, putting a can on a tray or stacking blocks. The system generates external views and robot head and wrist camera views while retaining task trajectories and physical interaction states for training data generation and simulation testing.
Visual observations and interaction states are aligned, allowing researchers to analyze execution by combining what the robot “sees” and “does”, and to compare and refine control policies before deployment into real-world scenes.
Mundo also extends spatial exploration to the time dimension. For dynamic events recorded by multiple cameras, it can reconstruct the process from new viewpoints; for simulated events built from images, it calculates motion via gravity and contact dynamics. Pausing at a selected timestamp and repositioning the camera enables multi-angle inspection of the same instant, creating a “bullet time” effect to examine action details and occlusion relationships.
From navigation to manipulation and 4D spatiotemporal observation, Mundo provides an environment for repeated rehearsal and multi-angle inspection. In one hotel use case, the team deployed it to reconstruct floors and laundry zones, supplying spatial and semantic information for navigation and simulating interactions such as opening washing-machine doors to help robots understand and train for tasks.
The industrial value of world models is measured by their contribution to real-world tasks. Within ACE Robotics’ Kairos-centered world model technology system, Mundo adds dedicated environment modeling and simulation training capabilities, bridging on-site data capture, synthetic data generation and policy optimization, and supporting robots deployed in real-world operational environments.
Project: https://mundo-world-model.github.io/
Blog: https://huggingface.co/blog/Luo-Yihang/mundo-world-model
AGIBOT
Figure AI
UBTECH