Yorg Healer
November 6, 2024 17:59
Hugging Face and NVIDIA are collaborating to advance open-source AI robotics by integrating LeRobot with NVIDIA’s AI tools, aimed at advancing industries such as manufacturing and healthcare.
At the recent Conference on Robotic Learning (CoRL) in Munich, Germany, Hugging Face and NVIDIA unveiled a strategic partnership designed to advance open source AI robotics. According to NVIDIA, the collaboration aims to unite their respective open source robotics communities by leveraging cutting-edge technologies to drive innovation in various sectors such as manufacturing, healthcare, and logistics.
Open source robotics for the physical AI era
Hugging Face’s partnership with NVIDIA promises to enable the era of physical AI, where robots understand the physical properties of their environment. This initiative is poised to transform the industry by providing researchers with an open-source framework for robot training, simulation, and inference and eliminating the need to repeatedly recreate code.
Integrated with NVIDIA’s AI, Omniverse, and Isaac robotics technologies, Hugging Face’s LeRobot platform provides a comprehensive suite of tools for data collection, model training, and simulation environments. This integration allows developers to access and fine-tune the more than 1.5 million models, datasets, and applications available in Hugging Face Hub.
Expanding robot development through simulation
Developing physical AI presents significant challenges, especially when it comes to collecting extensive physical interaction data. NVIDIA’s Isaac Lab, built on the Isaac Sim platform, solves this problem by creating synthetic environments with high-quality rendering and physics simulation. This approach can accelerate robot training, generating massive amounts of training data from a single demonstration.
The iterative process of training policies through imitation learning and deploying them to real robots leverages both the accuracy of real data and the scalability of synthetic data. This fosters a data-sharing community that accelerates the advancement of AI-based robotics by ensuring robust and reliable robotic systems.
Promote collaboration and community engagement
The collaboration also emphasizes community engagement through workflows involving data collection through Isaac Lab’s remote operations and simulations. Data is stored in a standard format to facilitate policy training and evaluation in simulations before deployment on real robots using NVIDIA Jetson for real-time inference.
Early stages of collaboration have shown promising results, including a physically selective setup running on the NVIDIA Jetson Orin Nano. This integration provides a powerful platform for AI model deployment, enhancing the transformative potential of AI robotics.
By combining Hugging Face’s open source community with NVIDIA’s hardware and simulation tools, this initiative aims to accelerate research and innovation in AI robotics. This collaboration is expected to have a revolutionary impact on the entire industry, from transportation to logistics.
Image source: Shutterstock