Nvidia Upgrades Isaac SIM Robotics Development Platform

Nvidia, on Tuesday at the Consumer Electronics Show (CES), announced significant updates to its Isaac Sim robotics simulation tool.

The Isaac SDK is the first open-source robotic AI development platform with simulation, navigation, and manipulation. Developing partners use the software tool to build and test virtual robots in realistic environments across varied operating conditions. Now accessible from the cloud, Isaac Sim is built on Nvidia Omniverse, a platform for creating and managing metaverse applications.

Demand for intelligent robots is growing as more industries embrace automation to address supply chain challenges and labor force shortages. The installed base of industrial and commercial robots will grow more than 6.4x — from 3.1 million in 2020 to 20 million in 2030, per ABI Research.

Developing, validating, and deploying these new AI-based robots requires simulation technology that places them in realistic scenarios, according to Gerard Andrews, product marketing manager for Nvidia’s robotics developer community.

Isaac Sim enables roboticists to import the robot model of their choice and build realistic environments to validate a robot’s physical design and fully exercise its software stack to ensure performance. Users can generate synthetic datasets during simulation to train the robot’s AI models used in the robot’s perception systems. Researchers can leverage the reinforcement learning API to train models in the robot’s control stack.

The latest version focuses on improving performance and functionality for manufacturing and logistics robotics use cases. The software now supports adding people and complex conveyor systems to the simulation environments, and more assets and popular robots are pre-integrated to reduce the time until simulation.

Release Highlights

Robotic Operating System (ROS) developers gain support for ROS 2 Humble and Windows. Robotics researchers get numerous new capabilities aimed at advancing reinforcement learning, collaborative robot programming, and robot learning.

The system improvements focus on the needs of humans working alongside collaborative robots (cobots) or autonomous mobile robots (AMRs). Isaac Sim’s new people simulation abilities add human-like common behaviors to simulations.

For instance, developers can now add human characters to simulations of a warehouse or manufacturing facility tasked with executing common behaviors like stacking packages or pushing carts. Many of the most common behaviors are already supported using a command.

To minimize the difference between results observed in a simulated world versus those seen in the real world, physically accurate sensor models are essential. Nvidia’s RTX technology enables Isaac Sim to render physically accurate data from sensors in real time. In the case of an RTX-simulated Lidar (light detection and ranging), ray tracing with more speed and precise sensor data under various lighting conditions or in response to reflective materials.

More Tools for Robotic Researchers

Isaac Sim also provides numerous new simulation-ready 3D assets critical to building physically accurate simulated environments. Everything from warehouse parts to popular robots comes ready to go, so developers and users can quickly start building, according to Nvidia.

Three new capabilities bolster the toolset for robotics researchers:

• Advances in Isaac Gym reinforce learning.
• Isaac Cortex improves collaborative robot programming.
• A new tool, Isaac Orbit, provides simulation operating environments and benchmarks for robot learning and motion planning.

Isaac SIM supports warehouse conveyor and people simulation. (Image Credit: Nvidia)

Expanded Use of Robotics Underway

According to Nvidia, the robotics ecosystem already spans multiple industries, from logistics and manufacturing to retail, energy, sustainable farming, and more. Its Isaac robotics platform provides advanced AI and simulation software as well as accelerated computing capabilities to the robotics ecosystem. Over a million developers and more than a thousand companies rely on one or many parts of it.

A sampling of robotic operations includes:

• Telexistence deploys beverage restocking robots across 300 convenience stores in Japan.
• To improve safety, Deutsche Bahn, the national railway company of Germany, trains AI models to handle critical but unexpected corner cases that rarely happen in the real world — like luggage falling on a train track.
• Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations.
• Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots.
• Fraunhofer is developing advanced AMRs using the physically accurate and full-fidelity visualization features of Isaac Sim.
• Flexiv is using Isaac Replicator for synthetic data generation to train AI models.