Improving Domestic Robot Multitasking Capabilities

Physical Intelligence (Pi) just dropped Pi-0.5 — a new foundation model for robots that tackles general tasks like cleaning & laundry. Trained via VLM + action tokenization, it performs well even in unseen environments. Trained on 400+ hours of diverse mobile manipulation data and multi-modal sources (web, vision, language, and robot demos), π0.5 can perform complex real-world tasks like cleaning tables, folding laundry, or loading dishwashers—even in unseen homes with new layouts and unfamiliar objects. How it works: π0.5 is co-trained on a rich mix of heterogeneous data—robot demos, image-text tasks (like captioning or object detection), and natural language instructions. It uses a dual-pathway architecture: 🔠 Discrete token inference for high-level planning (“put dishes in sink”) 🔁 Continuous decoding via flow-matching for low-level motor control (e.g., joint movements) The model outputs a sequence of 50-step “action chunks” (~1 second each), chaining high-level reasoning to low-level physical action. It can even transfer behaviors from simpler robots or simulations to more complex hardware. 📊 Performance: In evaluations across new environments, π0.5 achieved strong generalization: - Up to 90%+ task success rates on multi-step cleaning tasks - Significant gains from including diverse training sources (e.g., adding web data improved novel object recognition in out-of-distribution tests). - Generalization improved steadily with training on just 100 distinct environments, nearing the performance of models trained directly on test scenes. This matters because current robots are often limited to fixed settings—like warehouses. π0.5 shows real promise in dynamic, cluttered, everyday spaces like homes, offices, and hospitals. In short, π0.5 doesn’t just learn tasks—it learns how to learn tasks in new places. That’s a massive leap toward general-purpose home and service robotics.

Researchers at UC San Diego developed 𝐖𝐢𝐥𝐝𝐋𝐌𝐚, a framework to enhance quadruped robots' loco-manipulation skills in real-world tasks like cleaning or retrieving objects. Using VR-collected demonstrations, Vision-Language Models (VLMs), and Large Language Models (LLMs), robots can break complex tasks into steps (e.g., "pick—navigate—place"). Attention mechanisms improve adaptability, allowing robots to handle chores like tidying or food delivery. While promising, the system's next goal is greater robustness in dynamic environments, aiming for affordable, accessible home assistant robots. 📝 Research Paper: https://lnkd.in/e8HtbUF9 📊 Project Page: https://wildlma.github.io/

Figure, a leading AI-robotics company, raises $1.5B at $39.5B valuation and aims to build 100k AI-powered robots. Here’s why this matters: → Figure is developing a new AI system called Helix. It allows humanoid robots to perform complex tasks through voice commands. → Helix can handle unfamiliar objects without needing specific training for each one. This breakthrough could revolutionize household chores. → The system combines two powerful AI components. The first is a 7-billion-parameter multimodal language model. It processes speech and visual information at 7-9 Hz. This acts as the robot's brain. The second is an 80-million-parameter AI. It translates the language model's instructions into precise movements at 200 Hz. → Helix can control 35 degrees of freedom. This means it can manage everything from finger movements to head and torso control. → Figure has shown robots responding to voice commands and accurately grasping objects. One demonstration featured two robots placing food items into a refrigerator. They did this without prior training on those specific items. Training with limited data is a huge leap forward in progress in AI robotics. The system needed only 500 hours of training data. This is far less than what other projects require. The robots run on embedded GPUs, making commercial applications more feasible. CEO Brett Adcock sees Helix as key for scaling robots in homes. Unlike traditional robots, Helix adapts to new tasks without reprogramming. Its real-world performance still needs testing. Here’s their latest video showing Helix in action with two robots learning on the fly to put items away in a kitchen.

What if home robots could clean, carry, and navigate, coordinating their entire body like a human? [⚡Join 2500+ Robotics enthusiasts - https://lnkd.in/dYxB9iCh] A team from Stanford University, Columbia University, and Massachusetts Institute of Technology - Yunfan Jiang, Ruohan Zhang, Josiah Wong, Chen Wang, Yanjie Ze, Hang Yin, Cem Gokmen, Shuran Song, Jiajun Wu, and Li Fei-Fei Introduces the BEHAVIOR Robot Suite (BRS), a framework for whole-body manipulation in real-world household tasks. "BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities" Built on a bimanual, wheeled robot with a 4-DoF torso, enabling bimanual coordination, stable navigation, and extensive end-effector reachability. Integrates JoyLo, a cost-effective whole-body teleoperation interface using Nintendo Joy-Con controllers mounted on 3D-printed arms for intuitive control and data collection. Employs WB-VIMA, a novel imitation learning algorithm that models whole-body actions by leveraging the robot’s kinematic hierarchy and dynamically aggregating multi-modal observations using self-attention. Evaluated on five challenging household tasks, including cleaning, organising, and object manipulation in confined spaces, demonstrating robust performance. BRS addresses both hardware and learning challenges, marking a significant step toward enabling real-world whole-body manipulation for everyday household tasks. If robots can perform complex household tasks with such coordination, what new roles could they take on in our daily lives? Paper: https://lnkd.in/gTYz28Z2 Project Page: https://lnkd.in/gahqJMem #WholeBodyManipulation #ImitationLearning #MobileManipulation #RoboticsResearch