Apa yang berlaku
Researchers at Stanford University's The Movement Lab have developed HomeBody, a system that allows humanoid robots—specifically the Unitree G1—to navigate and perform long-horizon tasks in previously unseen environments without environment-specific training. By integrating GPT-6 Astra as a high-level reasoning agent, the system enables robots to clean up rooms, retrieve objects, and manage household items by orchestrating a library of reusable motor skills.
HomeBody functions by equipping a humanoid robot with a spatial memory system that connects visual observations with remembered locations. The system uses a three-part architecture where the VLM (GPT-6 Astra) acts as the high-level planner, selecting from a library of pre-defined motor skills such as navigation, picking, placing, and drawer opening.
To operate in an unfamiliar space, the robot first performs an exploration phase, collecting video, LiDAR, and joint pose data. This data is used to build a digital twin in Isaac Sim, which serves as a spatial model. This allows the VLM to reason about objects and locations that are not currently in the robot's ego view.
When given a task, such as 'tidy up the kitchen,' the VLM issues structured tool calls to the robot's local controller. The system includes feedback loops where the robot can retry failed actions or, if a task remains impossible, report the failure back to the VLM to adjust the plan.
The current implementation runs on a Razer Blade laptop equipped with an RTX 4090 GPU, which handles the local perception and motion planning, while the VLM reasoning is performed remotely.
Butiran sumber: tml.stanford.edu ↗
Mengapa ia penting
HomeBody addresses a critical bottleneck in robotics: the ability for a robot to maintain a coherent plan across a large space when objects move out of its immediate field of view. By providing the VLM with persistent spatial memory and a digital twin reconstruction, the system allows for complex, multi-step manipulation without requiring end-to-end training for every specific room. This represents a shift toward using high-level frontier models to directly manage modular robot skills, potentially reducing the need for specialized, environment-specific policy learning. However, the system currently faces practical limitations, including latency from remote model calls, hardware endurance constraints, and the requirement for significant local compute power.
The system demonstrates that high-level frontier models can effectively manage long-horizon tasks by composing modular skills, rather than relying on a single, monolithic policy. This modularity allows for greater flexibility, as new skills can be added to the library without retraining the entire system.
By grounding the VLM's reasoning in a persistent spatial map, the robot can resolve ambiguous requests and navigate to objects that are not immediately visible, a significant step toward more autonomous household robotics.
The reliance on Real2Sim reconstruction and remote API calls introduces meaningful unknowns regarding real-world deployment costs and the scalability of the system in environments with high levels of clutter or dynamic changes.
Mekanisme Interaktif: Bagaimana Ia Berfungsi Sebenarnya
Terokai teknologi asas di sebalik pembangunan ini secara interaktif.
An agent must create a draft calendar event for Tuesday at 2 p.m. Which evidence would establish the requested result?
Apa yang perlu ditonton seterusnya
Future developments will likely focus on reducing the latency between the VLM's reasoning and the robot's physical execution, as well as improving the efficiency of the Real2Sim reconstruction process. Observers should monitor whether this modular approach to 'System 2' orchestration can scale to more complex, unstructured environments beyond a controlled kitchen setting, and how it handles hardware-specific constraints like servo overheating during extended operation.
The researchers noted that the system is currently constrained by the humanoid's physical reach, manipulation capabilities, and hardware endurance, specifically mentioning finger-servo overheating.
The requirement for an RTX 4090-class GPU for local processing suggests that deploying this system on more compact or mobile hardware remains a challenge.
The impact of GPT-6 Astra's reasoning latency on real-time performance is a key area for improvement, as it currently introduces pauses between individual skill executions.