Humanoid motion tracking performance is greatly determined by retargeting quality! Introducing ๐ข๐บ๐ป๐ถ๐ฅ๐ฒ๐๐ฎ๐ฟ๐ด๐ฒ๐๐ฏ, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with ๐บ๐ถ๐ป๐ถ๐บ๐ฎ๐น RL: - 5 rewards, - 4 DR
Existing retargeting often produces artifacts like foot-skating and penetration โ. To compensate, RL policies rely on complex ad-hoc reward terms, forcing a trade-off between accurate motion tracking and correcting errors like slipping or bad contacts. OmniRetarget fixes this
The result of this high-quality data? We can train diverse skills like box carrying ๐ฆ, slope crawling ๐พ, and platform climbing ๐ง with a radically simplified RL process! All policies use just 5 reward terms, achieving successful zero-shot sim-to-real transfer! ๐ฏโก๏ธ๐ฆพ 3/9
What about scalability? OmniRetarget transforms a SINGLE human demo into diverse motion clips. We can systematically vary terrain height, object size, and initial poses. Best of all, these augmented skills transfer directly from sim to our real-world hardware! ๐คโก๏ธ๐ฆพ 4/9
And it's not just for a specific robot! Our framework is highly general and adapts to different robot embodiments, including the @UnitreeRobotics H1 and the @boosterobotics T1. We can retarget complex object-carrying and platform-climbing skills across these different robots with
But how much better is our data? ๐ค Compared to widely-used baselines, our motions show far fewer physical artifactsโvirtually zero foot-skating and penetrationโwhile better preserving contact. This allows us to use an open-sourced RL framework (BeyondMimic) without
Our grand finale: A complex, long-horizon dynamic sequence, all driven by a proprioceptive-only policy (no vision/LIDAR)! In this task, the robot carries a chair to a platform, uses it as a step to climb up, then leaps off and performs a parkour-style roll to absorb the landing.
Standing on the shoulders of giants! Our work builds on amazing research in the community๐ก. We use the "interaction mesh" ๐ธ๏ธ [1], [2] to preserve spatial relationships and leverage the minimal RL formulation from works like BeyondMimic [3]. Our long-horizon sequence is a nod to
We are open-sourcing over 4 hours of high-quality, retargeted trajectories! Website: https://omniretarget.github.io ArXiv: https://arxiv.org/abs/2509.266... Datasets: https://huggingface.co/dataset... Huge shout out to the amazing team: @lujieyang98, @x_h_ucb, @akanazawa, @pabbeel, @carlo_sferrazza,
@zhenkirito123 Beautiful results!!! And cliffhanger ๐
@brenthyi Thanks Brent! Yeah there are more exciting parkour-style motions on the way ๐๐๐๐
@zhenkirito123 climbed like a real human. Soon, we will see robot parkour competitions.
@zhenkirito123 Very cool work
@zhenkirito123 Incredible work! Really solid result
@zhenkirito123 @Scobleizer This looks like a huge step forward for more natural humanoid motion ๐
@zhenkirito123 This is truly impressive to see how generalizeable this is and also simplifies the skill transfer process to potentially hundreds of humanoid robot vendors. Wonโt be surprised to see lots of robotics companies building upon this work in the future.
@zhenkirito123 Impressive work! Fixing retargeting artifacts at the source rather than with complex reward engineering is the right approach. The long-horizon parkour sequence is stunning!
@zhenkirito123 this is what the community needs:) and these videos are really impressive!
@zhenkirito123 I donโt know you personally yet, but you superstar better than 1000 Kardashians and Ronaldo ๐๐ค
@zhenkirito123 Pelvic and hip mobility is bloody amazing
@zhenkirito123 Impressive work on solving those artifact issues! Streamlined RL with fewer reward terms is a game changer. What's next for scalability challenges?
@zhenkirito123 Thatโs some really nice work! Congratulations!
@zhenkirito123 @grok What's the meaning of retargeting?
@zhenkirito123 @TairanHe99 If you had to choose which is more efficient, learning from Third-Person Human or Learning from Motion Capture
@zhenkirito123 Interesting work! Specially since it doesnโt need to undergo curriculum training. Could OmniRetarget be made adaptive to the downstream RL task or policy uncertainty, dynamically refining trajectories?
@zhenkirito123 Wow interesting stuff and you say it's generalizable to other robots? Def interested ๐
@zhenkirito123 Very impressive
@zhenkirito123 OmniRetarget: Where human grace meets robotic agility. A leap into a new era of interaction! ๐คโจ
@zhenkirito123 @Threadreaderapp unroll
@zhenkirito123 Congrats! When will you open-source the motion retargeting code๐ฅน
@zhenkirito123 wow!!! ๐ฅน๐ซถ amazing job this is so cool!!
@zhenkirito123 That's great.
