How Algorithms Improve Robotic Performance

Robot models get better only when humans feed them more demos. This one improves by learning from its own mistakes. pi*0.6 is a new VLA from Physical Intelligence, that can refine its skills through real-world RL, not just teleop data. The team calls the method Recap, and from what I can see, the gains are not small. A quick summary: ✅ Learns from its own rollouts using a value function trained across all data ✅ Humans only step in when the robot is about to drift too far ✅ Every correction updates the model and improves future rollouts ✅ Works across real tasks like espresso prep, laundry, and box assembly ✅ Throughput more than doubles on hard tasks, with far fewer failure cases What stands out is the structure: a general policy, a shared value function, and a loop where the robot collects data, improves the critic, then improves itself again. No huge fleets of teleoperators. No massive manual resets. If VLAs can reliably self-improve in the real world, the bottleneck shifts. Data becomes cheaper. Deployment becomes the real test bench. Full paper, videos, and method details here: https://lnkd.in/dgCeZdjT

Robotics data is expensive and slow to collect. A lot of videos are available online, but not readily usable by robotics because of lack of action labels. AMPLIFY solves this problem by learning Actionless Motion Priors that unlock better sample efficiency, generalization, and scaling for robot learning. Our key insight is to factor the problem into two stages: The "what": Predict the visual dynamics required to accomplish a task The "how": Map predicted motions to low-level actions This decoupling enables remarkable generalizability: our policy can perform tasks where we have NO action data, only videos. We outperform SOTA BC baselines on this by 27x 🤯 AMPLIFY is composed of three stages: 1. Motion Tokenization: We track dense keypoint grids through videos and compress their trajectories into discrete motion tokens. 2. Forward Dynamics: Given an image and task description (e.g., "open the box"), we autoregressively predict a sequence of motion tokens representing how keypoints should move over the next second or so. This model can train on ANY text-labeled video data - robot demonstrations, human videos, YouTube videos. 3. Inverse Dynamics: We decode predicted motion tokens into robot actions. This module learns the robot-specific mapping from desired motions to actions. This part can train on ANY robot interaction data - not just expert demonstrations (think off-task data, play data, or even random actions). So, does it actually work? Few-shot learning: Given just 2 action-annotated demos per task, AMPLIFY nearly doubles SOTA few-shot performance on LIBERO. This is possible because our Actionless Motion Priors provide a strong inductive bias that dramatically reduces the amount of robot data needed to train a policy. Cross-embodiment learning: We train the forward dynamics model on both human and robot videos, but the inverse model sees only robot actions. Result: 1.4× average improvement on real-world tasks. Our system successfully transfers motion information from human demonstrations to robot execution. And now my favorite result: AMPLIFY enables zero-shot task generalization. We train on LIBERO-90 tasks and evaluate on tasks where we’ve seen no actions, only pixels. While our best baseline achieves ~2% success, AMPLIFY reaches a 60% average success rate, outperforming SOTA behavior cloning baselines by 27x. This is a new way to train VLAs for robotics which dont always start with large scale teleoperation. Instead of collecting millions of robot demonstrations, we just need to teach robots how to read the language of motion. Then, every video becomes training data. led by Jeremy Collins & Loránd Cheng in collaboration with Kunal Aneja, Albert Wilcox, Benjamin Joffe at College of Computing at Georgia Tech Check out our paper and project page for more details: 📄 Paper: https://lnkd.in/eZif-mB7 🌐 Website: https://lnkd.in/ezXhzWGQ

National Institute of Standards and Technology (NIST) is pushing the boundaries of industrial automation through its Agile Robotics for Industrial Automation Competition (ARIAC). This annual competition tests robots on their ability to perform pick-and-place operations, assembly, and kitting in a dynamic environment. ARIAC is not just about a robot's physical capabilities or speed; it's about how adaptable, efficient, and autonomous it can be in a dynamic manufacturing environment. The competition serves as a testbed for developing algorithms that can be applied in real-world manufacturing settings. NIST uses insights gained from the contest to shape standard metrics and test methods for future robotic agility in manufacturing. Participants in the competition use ROS2 and Gazebo to interact with a simulated factory setup and tackle tasks like kitting, assembly, and the combined task. One of the key highlights of the competition was the innovative dynamic pick and place Algorithm. This approach on the conveyor belt helped the floor Cobot efficiently track part movements, optimizing pick and place tasks without additional sensor requirements like break beam. ARIAC also challenges participants to develop autonomous robotic solutions to tackle faults in parts, high-priority orders, and insufficient parts. Strategic resource management and prioritization strategies ensure timely completion of high-priority orders. Additionally, the use of sensors and cameras facilitated accurate part detection and localization. Overall, ARIAC is a valuable platform for developing algorithms that can be applied in real-world manufacturing settings and shaping standard metrics and test methods for future robotic agility in manufacturing. https://lnkd.in/ef6WeaWR

The Future of Robotics Isn’t Just Smarter Machines, It’s Machines That Learn Like HUMANS A breakthrough in reinforcement learning (RL) is quietly rewriting the rules of robotics. Forget rigid, pre-programmed bots—GRPO (Group Relative Policy Optimization) is enabling robots to adapt, compare, and improve like humans. But scaling this tech is harder than it looks. Let’s break it down: Why Traditional Robotics Is Hitting a Wall. Most robots today rely on fixed reward systems: “Pick up cup = +1 point” “Drop cup = -1 point” This works for simple tasks but crumbles in dynamic environments (e.g., handling irregular objects, adapting to human interruptions). GRPO flips the script: Evaluates groups of actions and assigns relative rewards (e.g., “Grip A outperformed Grip B”). Eliminates need for complex value models—cuts compute/memory costs by ~50%. Enables human-like trial-and-error learning through synthetic data. Synthetic Data, The Unsung Hero - Tools like NVIDIA Isaac Sim and DeepSeek’s synthetic engines let robots train 24/7 in hyper-realistic simulations: Autonomous vehicles practice navigating flooded roads. Surgical bots master sutures on virtual patients. Industrial arms adapt to chaotic assembly lines. No real-world risks. No privacy concerns. Just scalable, ethical training. The Roadblocks (and Why They Matter) GRPO isn’t plug-and-play for robotics yet: Sim-to-real gaps: Physics in simulations ≠ real-world friction/noise. Action complexity: Robots deal with continuous movements (e.g., joint angles), not discrete tokens. Compute hunger: Training requires serious GPU firepower (looking at you, NVIDIA L40S). But teams like DeepSeek and Field AI are already showing 5-13% ROI gains in early trials. What This Means for AI Developers Robots trained with GRPO + synthetic data could: Autonomously adapt to factory floor changes. Refine surgical techniques through 10,000 simulated ops. Navigate crowded spaces using “experience” from synthetic NYC sidewalks. The future isn’t just automation—it’s robots that learn on the job. Are you building the next gen of adaptive robots?

How Can Gen AI Revolutionize Robot Learning? MIT’s Computer Science and AI Lab (CSAIL) has unveiled a promising breakthrough in robotics training—LucidSim, a system powered by generative AI that could help robots learn complex tasks more efficiently. Traditionally, robots have struggled with a lack of training data—but LucidSim taps into the power of AI-generated imagery to create diverse, realistic simulations. By combining text-to-image models, physics simulations, and auto-generated prompts, LucidSim can rapidly produce large amounts of training data for robots—whether it’s teaching them to navigate parkour-style obstacles or chase a soccer ball. This system outperforms traditional methods like domain randomization and even human expert imitation in many tasks. Key takeaways: - Generative AI is being used to scale up data generation for robotics training, overcoming the industry’s current data limitations. - LucidSim has shown strong potential for improving robot performance and pushing humanoid robots toward new levels of capability. - Researchers aim to improve robot learning and general intelligence to help robots handle more real-world challenges. With robots continuing to grow in sophistication, this innovative approach could mark a significant step toward more capable, intelligent machines in the future!

Vid2Robot End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers While large-scale robotic systems typically rely on textual instructions for tasks, this work explores a different approach: can robots infer the task directly from observing humans? This shift necessitates the robot's ability to decode human intent and translate it into executable actions within its physical constraints and environment. We introduce Vid2Robot, a novel end-to-end video-based learning framework for robots. Given a video demonstration of a manipulation task and current visual observations, Vid2Robot directly produces robot actions. This is achieved through a unified representation model trained on a large dataset of human video and robot trajectory. The model leverages cross-attention mechanisms to fuse prompt video features to the robot's current state and generate appropriate actions that mimic the observed task. To further improve policy performance, we propose auxiliary contrastive losses that enhance the alignment between human and robot video representations. We evaluate Vid2Robot on real-world robots, demonstrating a 20% improvement in performance compared to other video-conditioned policies when using human demonstration videos. Additionally, our model exhibits emergent capabilities, such as successfully transferring observed motions from one object to another, and long-horizon composition, thus showcasing its potential for real-world applications.

Most autonomous robots today use a traditional "sense, think, act" architecture. That is, separate code (often implemented by separate teams) are responsible for perceiving what is in the environment, deciding on an appropriate course of action, and carrying out that action. What if we could simplify this, and instead have a single AI model sense, think, and act all at once? That is the domain of Robot Learning and Embodied AI. This week, researchers at UC Berkeley announced SERL, a new open source library for "Sample-Efficient Robotic Reinforcement Learning". Instead of supporting many different reinforcement learning algorithms, they selected sensible defaults, optimizing for being able to train their model with as few attempts as possible (that's the "sample-efficient" part). When they put this new library to the test, they were able to learn tasks much faster and more accurately than anyone has previously achieved. For example, it learned the PCB insertion task in this video to 100% accuracy with just 20 demonstrations and 20 minutes of learning! Now, if only I could get their dataset in mcap format I could visualize this nicely in Foxglove 😄 https://lnkd.in/gwQQ5JVq

Minimizing Sim2real Error in Robotics when deploying to Real World: Maybe the best open source content available on internet. This content is written directly by the Reinforcement Learning and Imitation Learning team at Limx Dynamics. https://lnkd.in/gFxXSCYe Deployment: When deploying a trained policy to the real robot, we often encounter stable simulation performance but markedly different real-world behavior – the Sim2Real Gap. This is a common challenge in robot locomotion training. When facing a significant Gap, we need to check if the following two aspects have been properly addressed: 1. Build accurate robot models: 2. Establish an accurate simulation environment: If a large gap persists after checking these conditions, we need to analyze whether our policy is overfitting, meaning it's only applicable within a narrow range of environments. In this case, focus on the appropriateness of Domain Randomization. Adding randomization items or increasing the randomization range can enhance the model's generalization performance. After optimization, the open-source algorithm's performance on the real robot closely approaches that of the deeply optimized algorithm in remote control mode. Opportunities for Further Optimization The algorithm developed this time is implemented based on the CTS framework. To further enhance performance, researchers are encouraged to explore the following directions in-depth: 1. CTS framework enhancement: Improve the CTS framework on multiple levels, such as innovatively designing the Teacher Policy mechanism, using knowledge distillation to improve adaptability in complex terrain, and incorporating insights into robot dynamics into the Student Policy. 2. State representation optimization: The current version explicitly models the robot's linear velocity. Further work could expand implicit feature learning from historical observation data. 3. Improved network architecture: Upgrade the existing MLP policy network to a sequence-aware architecture (e.g., CNN or GRU) to better handle temporal dependencies in motion control. 4.Introducing more observations: Incorporating sensor data like vision can further enrich the robot's environmental perception. Introduce other generative models: Experiment with methods like diffusion models to generate longer action sequence commands. Click here for the website: https://lnkd.in/gFxXSCYe Sales: Gigi YE Cofounder: Li Zhang Github: https://lnkd.in/gpAUqMbZ https://lnkd.in/gFxXSCYe Detailed article: https://lnkd.in/gD-F9iGp Paper:https://lnkd.in/gz7Wp64P #Robotics #BipedalRobot #TRON1 #ReinforcementLearning #RL #EmbodiedIntelligence #AI #MachineLearning #OpenSource #SDK #Simulation #Sim2Real #IsaacGym #CTS #RobotLocomotion #Algorithm #LimX #TechInnovation #Python #ControlSystems

Mapless Navigation for Mobile Robot! Robots navigating unknown environments often encounter local optima, where they get stuck avoiding obstacles but fail to reach their destination. This research introduces a deep reinforcement learning-based mapless navigation framework that enables robots to efficiently escape local optima and improve real-world navigation performance. The researchers introduce: 1. Local Exploration Task: Encourages the robot to explore new paths instead of getting trapped in previously visited areas. 2. Adaptive Temperature Parameter: Adjusts exploration-exploitation trade-offs, stabilizing training while improving decision-making. 3. Soft Actor-Critic (SAC) Enhancement: Improves strategy learning by balancing risk and efficiency. 𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁? The research showed higher success rates and shorter navigation paths compared to existing methods in both simulations and real-world experiments. This research moves us closer to truly autonomous robots capable of navigating complex, unstructured environments without predefined maps! Congrats to Yiming Hu, Shuting Wang, Yuanlong Xie, Shiqi Zheng, Peng Shi, Imre Rudas, and Xiang Cheng! 🔗 Read the full paper: https://lnkd.in/gJDgypCr I post the latest and interesting developments in robotics - 𝗳𝗼𝗹𝗹𝗼𝘄 𝗺𝗲 𝘁𝗼 𝘀𝘁𝗮𝘆 𝘂𝗽𝗱𝗮𝘁𝗲𝗱! #ReinforcementLearning #Robotics #AI #DeepLearning #AutonomousNavigation #RobotLearning P.S. I wasn't able to find some researchers on, please let me know if they're on linkedin!

This week in RL research, a new paper was published on leveraging contrastive learning to improve robotic manipulation with reinforcement learning, and the results are game-changing 🚀 The researchers found that: • By incorporating contrastive learning into the reinforcement learning framework, they achieved significant improvements in robotic manipulation tasks, such as grasping and placing objects. • The proposed method enables the robot to learn more robust and generalizable policies, even in the presence of noisy or incomplete data. • The approach also reduces the need for manual engineering of reward functions, making it a more efficient and scalable solution. This research could be particularly helpful for improving robotic manipulation with reinforcement learning, as it addresses the challenges of learning from high-dimensional sensory inputs and sparse rewards. Read the full paper to dive deeper into the methodology and results: https://lnkd.in/dDQdEiBg. Share your thoughts on this breakthrough research and let's discuss how it can be applied in real-world scenarios 💬 #ReinforcementLearning #RoboticManipulation #RLJobs