AGIBOT open-sourced the fourth installment of its AGIBOT WORLD 2026 dataset series, “Physical Reasoning,” describing it as the first real-world Physical Agent dataset designed to train robots not only to execute actions but also to decide what to do under changing conditions.
The company said the dataset marks a shift in task design from manipulation difficulty to decision difficulty, linking goal understanding, process memory and action execution in real-world tasks. It covers nine categories of logical reasoning scenarios and, for the same task, deliberately captures multiple real-world states and execution branches. It provides both high-level question-answer annotations and low-level action trajectories, supporting coordinated training from task decision-making to real-machine execution.
The release is the first installment of AGIBOT’s AI WAVE series, which the company said will focus first on the data robots need to learn to work. The dataset has already been used to train AGIBOT’s upcoming Agent framework and a new-generation foundation model, providing a training basis for robots to understand tasks, remember processes and select actions. Subsequent AI WAVE releases will show how AGIBOT turns data into intelligence and how it plans to deploy robots in real-world settings such as convenience stores, the company said.
AGIBOT argued that real-world tasks rarely have a standard answer. In one example, a robot asked to restock a hotel minibar to three bottles must distinguish empty bottles from full ones, count existing stock and then decide how many bottles to remove and how many to add; the steps differ depending on the initial state. In another, a robot searching multiple drawers for toiletries must track which drawers have already been checked, even when the current camera view is identical. Such tasks require state assessment, step dependency and understanding of external conditions, which the company said are common in room cleaning, cooking and preparing to go out.
To enable autonomous completion of such tasks, AGIBOT said data on how to execute actions is insufficient; data on how to make decisions based on goals and conditions is also required. The fourth dataset was built around that gap.
The dataset organizes tasks around four key factors: object quantity, occlusion relationships, location occupancy and task progress. For example, when the goal is to retrieve a key from a box, the robot can grasp it directly if space allows, but must first move the box if it is blocked. The same goal under different site conditions requires different action paths.
The nine core reasoning capabilities covered are attribute-constrained selection, situational conditional selection, operational feasibility assessment, precondition reasoning, goal-directed search, state and quantity reasoning, long-horizon planning, memory state maintenance and subtask order reasoning. These capabilities often appear together in real tasks: restocking requires product identification, quantity judgment and state maintenance, while object search combines search strategy, historical records and spatial conditions in continuous decision-making.
Tasks are graded Easy, Medium and Hard. Basic tasks test single judgments such as attribute and shape matching; medium tasks add quantity and preconditions; high-difficulty tasks layer multi-subgoal planning, long-horizon state maintenance and multiple situational constraints. The dataset also annotates execution results, including whether the correct object was selected, whether it was placed correctly and whether the quantity was sufficient, with both successful and error samples to support research on how to adjust after execution deviations.
The data link connects decision-making, process and action. Task instructions describe complete goals, while Task Frames, subtask intervals and action steps task structure—helping researchers align task progress with current subgoals and study task decomposition, stage recognition and next-subgoal prediction. At the action level, Dense Prompts specify target objects, operation parts, left-right arm division, motion processes and expected results, paired with real-machine actions. Spatial annotations such as boxes, points and lines further specify target regions, contact positions and operation paths, mapping language requirements to specific spatial locations. Success and interference annotations provide information for completion judgment, target identification and execution-result checks.
AGIBOT said researchers can use the dataset for joint training and end-to-end training to improve models’ situational understanding, process memory and task reasoning. It can also support decision-execution coordinated training for embodied Agents: selecting subgoals based on current context and execution history, completing operations through action policies, and then updating task status and adjusting subsequent plans based on results.
The company said the dataset provides a training basis for large models to enter the physical world oriented to real tasks. By connecting task decisions, process memory and real-machine execution, it supports foundation models and embodied Agents in translating understanding of user goals into practical action.
Over the longer term, AGIBOT said this could help robots integrate into complete workflows in home services, hotels and catering, and commercial operations, taking on real work that requires continuous judgment and execution, and laying a foundation for scaled application of embodied intelligence in the real economy.
AGIBOT
Figure AI
UBTECH