AGILE: A Generic Isaac-Lab Based Engine for Humanoid Loco-Manipulation | Robotics Office Hour

Video Description

In this Robotics Office Hours livestream, NVIDIA engineers introduce AGILE, a generic Isaac Lab–based engine for humanoid locomotion and manipulation learning. Learn how AGILE tackles sim-to-real transfer, reinforcement learning workflows, MDP design, training at scale, and deployment to MuJoCo and real robots. Key Takeaways - AGILE provides a practical RL workflow for humanoid locomotion and manipulation using Isaac Lab. - Model verification and deterministic evaluation are critical for sim-to-real success. - Reward curves alone are insufficient; joint limits and tracking metrics matter. - Deployment is streamlined via TorchScript and IO descriptors for sim-to-sim and real robots. Chapters 0:20 – Welcome to Robotics Office Hours 2026 1:04 – Meet the Hosts and AGILE Team (Isaac Sim & Isaac Lab) 2:22 – What Is AGILE? A Generic Isaac Lab Engine for Humanoid Loco-Manipulation 3:06 – From Flashy Humanoid Demos to Real-World Deployment 4:21 – AGILE Mission and System Overview (RL, MDPs, Tasks, Robots) 6:08 – End-to-End Training Workflow: 5 Core Components 7:16 – Pre-Training Model Verification GUI (Joints, PD Gains, Contacts) 10:24 – Scaling Training with OSMO + Weights & Biases (Sweeps & Reproducibility) 14:12 – Deterministic Evaluation and What Predicts Sim-to-Real Transfer 24:32 – Deployment Pipeline: TorchScript, IO Exporter, and MuJoCo Transfer 26:50 – Live Q&A: Robustness, Failure Recovery, and “Child-like” Behavior 28:20 – VLA Pipeline Question + Task-Agnostic Policies (Sonic) 29:16 – Modeling Contact + Friction for Sim-to-Real (USD + Randomization) 31:05 – Adding a New Embodiment (H1) + Cross-Embodiment Training Strategy 34:46 – Is the Verification UI Part of Isaac Sim? (AGILE Tooling Clarification) 35:33 – Teleoperation Options Beyond Keyboard (Isaac Lab Teleop + VR First-Person View) 36:21 – Joint Control Models: Delayed Motor vs Implicit Actuators 38:35 – Modular Architecture: Lower-Body RL + Upper-Body Control + Distillation 41:09 – IK + Locomotion: Improving Manipulation Accuracy 43:31 – Learning From Human Video: Retargeting and Physics Grounding 46:40 – AGILE Wrap-Up: What the Framework Delivers 47:55 – Q&A: Preventing Policy Exploitation Between Upper/Lower Controllers 48:41 – Q&A: Heatmaps and “Pilot View” Visualization (What’s Missing + VR Option) 49:35 – Q&A: 5-Finger vs 3-Finger Retargeting and Grounding Requirements 50:51 – Q&A: Physics Solver Transfer (PhysX vs Newton) 52:07 – Call to Action: Download AGILE from GitHub + Community Support Channels 52:53 – Live Demo: AGILE GitHub Repo Walkthrough (Robot Configs + Tasks) 57:31 – Q&A: Training Multi-Arm Coordination (Two Franka Arms, etc.) 58:16 – Q&A: Best Domain Randomization Strategy --------------------------------------- AGILE Q&A FAQ: https://discord.com/channels/827959428476174346/1460865708463231081 AGILE GitHub: https://github.com/nvidia-isaac/WBC-AGILE Omniverse Discord: https://discord.com/invite/nvidiaomniverse Isaac Sim & Isaac Lab Study Group: Wednesday at 12:30 PM in the NVIDIA Omniverse Discord Community Room AddEvent Livestream Calendar: https://www.addevent.com/calendar/ae483892 #NVIDIA #Robotics #IsaacSim #IsaacLab #HumanoidRobotics #ReinforcementLearning #Sim2Real #PhysicalAI #MuJoCo