Chapter 4: Digital Twin Simulation

Learning Objectives​

By the end of this chapter, you will be able to:

  • Understand the role of simulation in robotics development
  • Set up and use Gazebo for robot simulation
  • Explore NVIDIA Isaac Sim for photorealistic simulations
  • Apply sim-to-real transfer techniques

Introduction​

Digital twins are virtual replicas of physical robots that enable testing, training, and validation in simulation before deploying to real hardware.

Simulation accelerates development by allowing:

  • Safe testing of dangerous scenarios (falls, collisions)
  • Parallel training of AI models (thousands of virtual robots)
  • Rapid iteration without hardware wear

Core Concepts​

Why Simulate?​

Benefits:​

  1. Cost reduction: No hardware damage during testing
  2. Speed: Train AI models 1000x faster than real-time
  3. Scalability: Run thousands of parallel simulations
  4. Repeatability: Exact scenario reproduction for debugging

Challenges:​

  1. Sim-to-real gap: Physics models don't perfectly match reality
  2. Computational cost: High-fidelity simulation requires GPUs
  3. Modeling complexity: Creating accurate robot and environment models

Gazebo: Open-Source Robot Simulator​

Gazebo is a 3D robot simulator integrated with ROS 2. It provides physics simulation, sensor models, and robot visualization.

Key Features:​

  • Physics engines: ODE, Bullet, DART for dynamics simulation
  • Sensor plugins: Camera, LiDAR, IMU, force/torque sensors
  • ROS 2 integration: Seamless communication with ROS nodes
  • Customizable environments: Build worlds with obstacles, terrain

Gazebo Workflow:​

  1. Define robot model in URDF (Unified Robot Description Format)
  2. Create world file with environment and obstacles
  3. Launch simulation with Gazebo and ROS 2 bridge
  4. Control robot through ROS 2 topics
  5. Collect data (sensor readings, trajectories)

NVIDIA Isaac Sim: Photorealistic Simulation​

Isaac Sim is a GPU-accelerated simulator built on NVIDIA Omniverse. It offers:

  • Ray-traced rendering: Photorealistic visuals for computer vision
  • PhysX physics: Accurate contact dynamics and soft-body simulation
  • Synthetic data generation: Training data for vision models
  • Multi-robot coordination: Simulate fleets of robots

Use Cases:​

  • Warehouse automation: Test AMRs in realistic environments
  • Manipulation: Train grasping policies with accurate physics
  • Autonomous vehicles: Simulate urban driving scenarios
  • Computer vision: Generate labeled datasets for object detection

Sim-to-Real Transfer​

The sim-to-real gap is the performance degradation when transferring policies from simulation to reality.

Mitigation Strategies:​

  1. Domain randomization: Vary simulation parameters (lighting, textures, physics) to make policies robust
  2. System identification: Measure real robot parameters and tune simulation accordingly
  3. Fine-tuning: Train in simulation, then adapt on real hardware
  4. Residual learning: Learn corrections to simulation-based policy

Practical Application​

Example 1: Launching a Robot in Gazebo​

# Install Gazebo (if not installed)
sudo apt install ros-humble-gazebo-ros-pkgs

# Launch Gazebo with empty world
ros2 launch gazebo_ros gazebo.launch.py

# Spawn a robot model
ros2 run gazebo_ros spawn_entity.py -file robot.urdf -entity my_robot

Example 2: Controlling a Simulated Robot​

import rclpy
from rclpy.node import Node
from geometry_msgs.msg import Twist

class RobotController(Node):
    def __init__(self):
        super().__init__('robot_controller')
        self.publisher = self.create_publisher(Twist, '/cmd_vel', 10)
        self.timer = self.create_timer(0.1, self.control_loop)

    def control_loop(self):
        msg = Twist()
        msg.linear.x = 0.5  # Move forward at 0.5 m/s
        msg.angular.z = 0.2  # Turn at 0.2 rad/s
        self.publisher.publish(msg)

def main():
    rclpy.init()
    node = RobotController()
    rclpy.spin(node)
    rclpy.shutdown()

Explanation: This node publishes velocity commands to move the simulated robot forward while turning.

Example 3: Reading Simulated Sensors​

from sensor_msgs.msg import LaserScan

class ObstacleDetector(Node):
    def __init__(self):
        super().__init__('obstacle_detector')
        self.subscription = self.create_subscription(
            LaserScan,
            '/scan',
            self.scan_callback,
            10)

    def scan_callback(self, msg):
        # Find minimum distance in front of robot
        front_ranges = msg.ranges[len(msg.ranges)//2 - 10 : len(msg.ranges)//2 + 10]
        min_distance = min(front_ranges)

        if min_distance < 0.5:  # Obstacle within 0.5m
            self.get_logger().warn(f'Obstacle detected at {min_distance:.2f}m!')

Example 4: Isaac Sim with Python API​

from omni.isaac.kit import SimulationApp

# Initialize Isaac Sim
simulation_app = SimulationApp({"headless": False})

from omni.isaac.core import World
from omni.isaac.core.robots import Robot

# Create world and add robot
world = World()
robot = world.scene.add(Robot(prim_path="/World/MyRobot", name="robot"))

# Run simulation
world.reset()
for i in range(1000):
    world.step(render=True)  # Step physics and render

simulation_app.close()

Summary​

Simulation is essential for modern robotics development. Gazebo provides accessible, open-source simulation for ROS 2, while Isaac Sim offers cutting-edge physics and rendering for AI training.

Understanding how to create robot models, build environments, and transfer learned policies to real hardware is a critical skill for robotics engineers.

Key Takeaways:

  • Simulation enables safe, fast, and scalable robot testing
  • Gazebo integrates seamlessly with ROS 2 for open-source development
  • Isaac Sim provides photorealistic rendering and accurate physics
  • Sim-to-real transfer requires domain randomization and careful tuning

Further Reading​