Addressing Morphology Challenges in Robot Training

🚀 Teaching humanoids complex skills often relies on retargeting human motions to robot kinematics. But traditional pipelines face two key challenges: ❌ Artifacts such as foot-skating or body penetration caused by the human–robot embodiment gap ❌ Limited handling of rich human–object and human–environment interactions, which are essential for expressive locomotion and loco-manipulation To address this, we present OmniRetarget – an interaction-preserving data generation engine powered by an interaction mesh. 💡 Using these high-quality trajectories, we trained proprioceptive RL policies that enable the Unitree G1 humanoid to perform long-horizon parkour and loco-manipulation skills (up to 30s) — all with just 5 reward terms, simple domain randomization, and no curriculum. This work highlights how preserving interactions is the key to scalable and generalizable humanoid skill learning. #HumanoidRobotics #ReinforcementLearning #MotionRetargeting #UnitreeG1 #EmbodiedAI

The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy spits out a robot-compatible policy! Point Policy uses sparse key points to represent both human demonstrators and robots, bridging the morphology gap. The scene is hence encoded through semantically meaningful key points from minimal human annotations. The overall algorithm is simple: 1. Extract key points from human videos. 2. Train a transformer policy to predict future robot key points. 3. Convert predicted key points to robot actions. This project was an almost solo effort from Siddhant Haldar. And as always, this project is fully opensourced. Project page: https://lnkd.in/e32RtQK9 Paper: https://lnkd.in/emQpENTy

#Robots struggle because our demonstrations are imperfect, not because the task is hard. Fatigue, noisy sensors, mismatched anatomy-- All of this creeps into robot learning. So when robots copy motions too literally, they inherit our mistakes. And when they ignore style, they end up moving like 'machines'. Two recent works from ETH Zürich take an interesting step toward fixing that. 1) 𝐌𝐨𝐭𝐢𝐨𝐧 𝐏𝐫𝐢𝐨𝐫𝐬 𝐑𝐞𝐢𝐦𝐚𝐠𝐢𝐧𝐞𝐝: 𝐀𝐝𝐚𝐩𝐭𝐢𝐧𝐠 𝐅𝐥𝐚𝐭-𝐓𝐞𝐫𝐫𝐚𝐢𝐧 𝐒𝐤𝐢𝐥𝐥𝐬 𝐟𝐨𝐫 𝐂𝐨𝐦𝐩𝐥𝐞𝐱 𝐐𝐮𝐚𝐝𝐫𝐮𝐩𝐞𝐝 𝐌𝐨𝐛𝐢𝐥𝐢𝐭𝐲 If you’ve ever tuned locomotion rewards for legged robots, you know how brittle things can get. This paper takes a neat two-layer approach: - 𝐋𝐨𝐰-𝐥𝐞𝐯𝐞𝐥 𝐩𝐨𝐥𝐢𝐜𝐲 = first learns animal-like motions on flat terrain (walk, pace, canter). - 𝐇𝐢𝐠𝐡-𝐥𝐞𝐯𝐞𝐥 𝐩𝐨𝐥𝐢𝐜𝐲 = learns residual corrections to adapt those motions to uneven terrain, obstacles, stairs & local navigation. The result is a quadruped that moves like an animal, but handles terrain like a robot trained for it. Notably, it works with minimal reward engineering-- just 5 reward terms in some experiments. Project: https://lnkd.in/gFJJgy2D Authors: Zewei Z., Chenhao Li, Takahiro Miki, Marco Hutter 2) 𝐂𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐭 𝐒𝐭𝐲𝐥𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐟𝐫𝐨𝐦 𝐈𝐦𝐩𝐞𝐫𝐟𝐞𝐜𝐭 𝐃𝐞𝐦𝐨𝐧𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧𝐬 𝐮𝐧𝐝𝐞𝐫 𝐓𝐚𝐬𝐤 𝐎𝐩𝐭𝐢𝐦𝐚𝐥𝐢𝐭𝐲 Now this paper takes another route! It looks at a different challenge of what do you do when demonstrations are incomplete, imperfect or just not task-optimal. Instead of treating “style vs. task” as a trade-off, ConsMimic frames it as a constrained optimization: - 𝐒𝐭𝐲𝐥𝐞 𝐫𝐞𝐰𝐚𝐫𝐝 (smoothness, coordination, natural gait) - 𝐓𝐚𝐬𝐤 𝐫𝐞𝐰𝐚𝐫𝐝 (velocity tracking, stability, navigation) - 𝐀𝐝𝐚𝐩𝐭𝐢𝐯𝐞 𝐋𝐚𝐠𝐫𝐚𝐧𝐠𝐢𝐚𝐧 𝐦𝐮𝐥𝐭𝐢𝐩𝐥𝐢𝐞𝐫 = tells the agent when to copy style and when to ignore it. The outcome is: - On GR1 humanoid = more coordinated arm–leg motion, less drift, less wasted knee energy. - On ANYmal-D = 14.5% lower mechanical energy and more agile gaits in real-world testing. Project: https://lnkd.in/gh258W6u Authors: Kehan Wen, Chenhao Li, Junzhe H., Marco Hutter Both of these papers point to the same idea- 𝐑𝐨𝐛𝐨𝐭𝐬 𝐬𝐡𝐨𝐮𝐥𝐝𝐧’𝐭 𝐣𝐮𝐬𝐭 “𝐝𝐨 𝐭𝐡𝐞 𝐭𝐚𝐬𝐤.” 𝐓𝐡𝐞𝐲 𝐬𝐡𝐨𝐮𝐥𝐝 𝐝𝐨 𝐢𝐭 𝐰𝐞𝐥𝐥, 𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭𝐥𝐲, 𝐚𝐧𝐝 𝐢𝐧 𝐚 𝐰𝐚𝐲 𝐭𝐡𝐚𝐭 𝐥𝐨𝐨𝐤𝐬 𝐩𝐡𝐲𝐬𝐢𝐜𝐚𝐥𝐥𝐲 𝐧𝐚𝐭𝐮𝐫𝐚𝐥. A useful shift for anyone scaling LfD + RL to hardware. #leggedrobotics #reinforcementlearning #robotlocomotion #quadrupedrobotics #consmimic #ethzurich #roboticsresearch

Robot dogs are getting more human friendly. Exploiting morphological symmetries can enhance model-free RL sample efficiency, policy optimality, and sim2real transfer in legged locomotion and manipulation. Exploring the two methods to leverage symmetry in actor-critic RL (PPO): 1) Data augmentation (for approximate actor equivariance and critic invariance) [PPOaug] 2) Enforcing strict equivariance/invariance in the NNs parameterizing the actor and critic functions [PPOeqic] Policies with strict equivariance showed superior sample efficiency, better task performance, and more natural gaits. Both methods improved robustness and sim-to-real transfer in real-world tasks. #paper: https://lnkd.in/d4mi5D2v #github: https://lnkd.in/d_sZ87NR #site: https://lnkd.in/dqzbuxef #authors: Zhi Su, Xiaoyu Huang, Daniel Felipe Ordoñez Apraez, Yunfei Li, Zhongyu Li, Qiayuan Liao, Giulio Turrisi, Massimiliano Pontil, Claudio Semini, Yi Wu, Koushil Sreenath Istituto Italiano di Tecnologia - Dynamic Legged Systems Lab, University of California, Berkeley, Tsinghua University, @Shanghai Qi Zhi Institute, at IROS 2024