
If you're interested in robotics and AI, you've probably heard of the [SO-ARM 101](https://www.youtube.com/watch?v=oitT8geMat0), the open-source robotic arm developed by Hugging Face in collaboration with The Robot Studio, and supported by a growing community of makers worldwide. Hugging Face's CEO has called it "the first robot arm any AI builder should buy," and it's quickly becoming one of the most popular platforms for learning imitation and reinforcement learning with real hardware. The best part? It's 3D printable, costs between $100 and $500 depending on how you source it, and integrates directly with the LeRobot framework.

<img src="/images/armatron.png" alt="SO-ARM 101 Robotic Arm" style={{ width: '70%', height: 'auto', display: 'block' }} />

<div style={{ padding: '1rem 1.5rem', backgroundColor: '#2a2a2a', borderLeft: '4px solid #ff8c29', borderRadius: '0.375rem', margin: '1.5rem 0', color: '#e0e0e0' }}>
  <strong style={{ color: '#ff8c29' }}>Project Goal:</strong> Enable real-time control of the SO-ARM 101 digital twin in Isaac Sim using the physical leader arm.
</div>

<div style={{ padding: '1rem 1.5rem', backgroundColor: '#2a2a2a', borderLeft: '4px solid #ff8c29', borderRadius: '0.375rem', margin: '1.5rem 0', color: '#e0e0e0' }}>
  <strong style={{ color: '#ff8c29' }}>📦 Full Source Code:</strong> All updated code, ROS2 integration, and configuration files are publicly available in my forked GitHub repository at <a href="https://github.com/protomota/lerobot-sim" style={{ color: '#ff8c29', textDecoration: 'underline' }}>github.com/protomota/lerobot-sim</a>. My contributions are detailed in PROTOMOTA_README.md.
</div>

In the video below, you'll see a split-screen view: on the left is the physical 3D-printed leader arm from the SO-ARM 101, and on the right is a screen recording of Isaac Sim showing the digital twin responding in real-time to my movements. I'm using the leader arm as a controller for the simulated robot, with every joint movement mirrored instantly in the simulation.

To get this working, I built a ROS2 bridge between the LeRobot teleoperation framework and Isaac Sim. I wrote a custom teleoperation controller that reads joint positions from the physical leader arm and publishes them as ROS2 messages. On the Isaac Sim side, there's an action graph that subscribes to those ROS2 commands and converts them into movements within the simulation, creating seamless real-time control.

<div style={{ position: 'relative', paddingBottom: '56.25%', height: 0, overflow: 'hidden', maxWidth: '100%' }}>
  <iframe
    style={{ position: 'absolute', top: 0, left: 0, width: '100%', height: '100%' }}
    src="https://www.youtube.com/embed/KPL8jyaWyp8"
    title="SO-ARM 101 Teleoperation in Isaac Sim"
    frameBorder="0"
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
    allowFullScreen
  />
</div>

---

## The Challenge with Separate Processes

I started with LycheeAI's approach, which uses separate processes for teleoperation and joint state publishing. It's a clean, logical design - separation of concerns at its finest. However, I ran into challenges getting it to work reliably with my hardware setup.

The core issue is that the Feetech STS3215 motors communicate over USB serial using half-duplex communication. When two processes try to access the same serial port simultaneously, you get packet collisions, corrupted data, and error messages like "There is no status packet" flooding your terminal.

I experimented with various solutions - mutex locks, reduced read frequencies, queue-based IPC - but couldn't achieve the reliability I needed for real-time teleoperation.

## The Integrated Solution

The fix was elegant: integrate ROS2 publishing into the teleoperation loop itself. The loop already reads joint positions every cycle, so just publish them to ROS2 at the same time.

```
┌────────────────────────────────────────────────────────┐
│                   Teleoperation Loop                   │
│                                                        │
│   1. Read leader position (ACM0)                       │
│   2. Read follower position (ACM1)  ──► ROS2 Publish   │
│   3. Write to follower (ACM1)                          │
│                                                        │
└────────────────────────────────────────────────────────┘
```

No port conflicts. 60Hz publishing. Smooth simulation.

The new command is called `lerobot-ros-teleoperate`, and it supports two modes:

**Full mode** is when you have both the leader and follower arms connected. The leader controls the follower, and the follower's positions get published to ROS2.

**Sim-only mode** is when you only have the leader arm. The leader positions get published directly to ROS2, driving the simulation without needing physical follower hardware.

This is huge for development. You can iterate on your simulation setup using just the leader arm, without needing the full dual-arm rig powered up.

## USB Connection Order Matters

One gotcha that tripped me up: USB port assignment is based on plug-in order. The first arm you plug in gets `/dev/ttyACM0`, the second gets `/dev/ttyACM1`.

I standardized on:
1. Leader first, gets ACM0
2. Follower second, gets ACM1

If you get weird behavior where the wrong arm is responding, unplug both and reconnect in the correct order.

## Gripper Calibration

The gripper needed special handling. Once I had basic teleoperation working, I noticed the gripper wasn't behaving correctly in Isaac Sim. It would move, but not through its full range, and the closed position wasn't lining up.

After some debugging, I discovered the issue: the physical gripper outputs values from about 0.02 radians when closed to 1.72 radians when open. But Isaac Sim's Jaw joint needed to reach -11 degrees to fully close.

The fix was two-part:
1. Set Isaac Sim Jaw limits to -11 (lower) and 100 (upper)
2. Apply -0.21 rad offset in code

Now the gripper tracks perfectly.

## Bridging to Isaac Sim

The `lerobot-ros-teleoperate` command publishes joint positions to `/joint_states`, which is the standard ROS2 topic for joint data. But the Isaac Sim USD file from LycheeAI has its action graph configured to listen on `/isaac_joint_command`.

Rather than modify either side, I used `topic_tools relay`, a standard ROS2 utility that republishes messages from one topic to another. It's a one-liner that bridges the two:

```bash
ros2 run topic_tools relay /joint_states /isaac_joint_command
```

This keeps the LeRobot side using standard conventions while matching what Isaac Sim expects.

## Getting the Code

<div style={{ padding: '1rem 1.5rem', backgroundColor: '#2a2a2a', borderLeft: '4px solid #ff8c29', borderRadius: '0.375rem', margin: '1.5rem 0', color: '#e0e0e0' }}>
  <strong style={{ color: '#ff8c29' }}>IMPORTANT:</strong> To get this working, you need to clone my forked repository that includes the integrated ROS2 teleoperation solution and all the modifications discussed in this post.
</div>

Clone the repository:

```bash
git clone https://github.com/protomota/lerobot-sim.git
cd lerobot-sim
```

Follow the installation instructions in the repository README to set up the environment and dependencies.

## Quick Start

```bash
source /opt/ros/humble/setup.bash
conda activate lerobot

# Terminal 1: Start teleoperation (leader arm on ACM0)
lerobot-ros-teleoperate \
    --teleop.type=so101_leader \
    --teleop.port=/dev/ttyACM0 \
    --teleop.id=armatron_leader

# Terminal 2: Relay joint states to Isaac Sim's expected topic
ros2 run topic_tools relay /joint_states /isaac_joint_command
```

Then open Isaac Sim, load your USD file, and press Play. The simulated arm will mirror your physical leader arm in real-time.

## What's Next

With teleoperation working, the next step is adding cameras. The goal is vision-based imitation learning, where the robot learns tasks by watching demonstrations.

The plan:
- **Add camera feeds** to capture what the robot "sees" during teleoperation
- **Record demonstrations** pairing camera frames with joint positions as the operator performs tasks
- **Train vision policies** that map camera input directly to robot actions
- **Deploy to real hardware** using the same camera setup on the physical follower arm

LeRobot already has infrastructure for this. The framework supports recording datasets with synchronized camera and joint data, and includes ACT (Action Chunking Transformer) and other imitation learning policies out of the box.

If you found this useful, let me know in the comments. And if you're building something similar, I'd love to hear about it.

## Credits

- [TheRobotStudio](https://github.com/TheRobotStudio/SO-ARM100) - Original SO-ARM 100 design and hardware
- [LeRobot](https://github.com/huggingface/lerobot) - Hugging Face's framework for real-world robotics
- [LycheeAI](https://lycheeai-hub.com/project-so-arm101-x-isaac-sim-x-isaac-lab-tutorial-series/so-arm-teleoperate-real-isaac-sim) - Isaac Sim URDF modifications, Action Graph info, and initial ROS2 bridge concepts


Building a real-time teleoperation bridge between the SO-ARM 101 robotic arm and NVIDIA Isaac Sim using LeRobot and ROS2.

Real-Time SO-ARM 101 ROS2 Teleoperation in Isaac Sim with LeRobot


<img src="/images/ogre-sim.png" alt="Project Ogre in Isaac Sim" style={{ width: '70%', height: 'auto', display: 'block' }} />

## Building the Digital Twin

This is Part 2 of the Project Ogre series. In [Part 1](/blog/project-ogre-part1-building-robot), we built the physical mecanum drive robot. Now we'll create an accurate digital twin in NVIDIA Isaac Sim that lets us develop and test without touching the real hardware.

<div style={{ padding: '1rem 1.5rem', backgroundColor: '#2a2a2a', borderLeft: '4px solid #ff8c29', borderRadius: '0.375rem', margin: '1.5rem 0', color: '#e0e0e0' }}>
  <strong style={{ color: '#ff8c29' }}>Project Goal:</strong> Build an accurate simulation of Project Ogre in Isaac Sim that uses the same ROS2 interface as the real robot, enabling simulation-first development.
</div>

<div style={{ padding: '1rem 1.5rem', backgroundColor: '#2a2a2a', borderLeft: '4px solid #ff8c29', borderRadius: '0.375rem', margin: '1.5rem 0', color: '#e0e0e0' }}>
  <strong style={{ color: '#ff8c29' }}>📦 Full Source Code:</strong> The USD robot model and Isaac Sim configuration are available at <a href="https://github.com/protomota/ogre-slam" style={{ color: '#ff8c29', textDecoration: 'underline' }}>github.com/protomota/ogre-slam</a>
</div>

## Why Simulation First?

In an ideal workflow, you'd build the simulation before touching hardware. I was too eager to get a working robot, so I built the physical platform first. I made progress on the real robot, but eventually hit enough issues that I realized constructing a digital twin would make debugging much easier.

The benefits of simulation-first development:

- **No hardware damage risk** - Test aggressive maneuvers without breaking motors or sensors
- **Faster iteration** - Reset the simulation in milliseconds vs. repositioning a physical robot
- **Reproducible experiments** - Same initial conditions every time
- **Parallel development** - Work on navigation code while the hardware is being built

Isaac Sim is NVIDIA's robotics simulation platform built on Omniverse. It provides:

- GPU-accelerated physics (PhysX 5)
- Photorealistic rendering with ray tracing
- Native ROS2 integration
- Sensor simulation (LIDAR, cameras, IMU)

## The USD Robot Model

The robot is defined in a Universal Scene Description (USD) file: `usds/ogre_robot.usd`. This format captures geometry, physics properties, and joint definitions in a single file.

### Creating the Model

I built the USD model by:

1. **Importing the chassis** - A simple box representing the steel frame
2. **Adding wheel assemblies** - Four mecanum wheels with proper roller geometry
3. **Defining joints** - Revolute joints connecting each wheel to the chassis
4. **Setting up collision meshes** - Simplified geometry for physics calculations
5. **Adding sensors** - LIDAR and camera mounts

### Physical Dimensions

The USD model matches the real robot's measurements exactly:

```
Body: 200mm (L) x 160mm (W) x 175mm (H)
Wheel Radius: 40mm
Wheelbase: 95mm (front-to-rear axle distance)
Track Width: 205mm (left-to-right wheel centers)
```

Getting these dimensions right is critical - SLAM and navigation algorithms depend on accurate odometry, which depends on knowing exactly how far the wheels travel per rotation.

### Wheel Configuration

Mecanum wheels require careful attention to roller angles:

```
M4 (FL) --- M1 (FR)
   |    X    |
M3 (RL) --- M2 (RR)
```

- Front-left and rear-right wheels have rollers angled one way
- Front-right and rear-left wheels have rollers angled the opposite way

This arrangement creates the characteristic X-pattern when viewed from above. If the roller angles are wrong, the robot won't strafe properly.

## Joint Configuration

Each wheel joint needs proper physics parameters for stable simulation:

### Joint Properties

```python
# Wheel joint configuration
joint_type: "revolute"
axis: [1, 0, 0]  # Rotate around X-axis
damping: 10.0
friction: 0.1
```

### Critical: Maximum Joint Velocity

Isaac Sim's default maximum joint velocity is extremely high (1,000,000 rad/s). This causes mecanum wheels to spin out of control during physics steps.

**Solution:** Set maximum velocity to 10,000 rad/s or lower:

```python
drive:
  maxVelocity: 10000.0  # NOT the default 1000000!
```

This single parameter took hours to debug. The robot would work fine at low speeds but become unstable at higher velocities.

### Wheel Sign Correction

The USD model has the right wheels (FR, RR) with opposite joint axis orientation from the left wheels. Sending `[+10, +10, +10, +10]` to all wheels makes the robot spin instead of moving forward.

The solution is sign correction in the action graph:

```
Left wheels (FL, RL): velocity as-is
Right wheels (FR, RR): velocity × -1
```

This normalization ensures `[+,+,+,+]` always means "all wheels forward" regardless of joint axis conventions.

## Physics Settings

Stable mecanum simulation requires tuned physics parameters:

### Simulation Timestep

```yaml
physics_dt: 1/120  # 120 Hz physics
rendering_dt: 1/60  # 60 Hz rendering
```

The physics timestep must be faster than your control rate. With 30 Hz control, 120 Hz physics gives four physics steps per control step.

### Wheel Friction

Mecanum wheel friction is counterintuitive. The angled rollers mean friction behaves differently along different axes:

```python
# Ground material
static_friction: 0.8
dynamic_friction: 0.6

# Wheel material
static_friction: 0.5
dynamic_friction: 0.4
```

Too much friction and the wheels won't slide sideways. Too little and the robot drifts uncontrollably.

### Joint Damping

```python
damping: 1.0 to 10.0  # Range for stability
```

Damping prevents joint oscillation. Higher values make the wheels feel "heavier" but more stable. Start with 10.0 and reduce if response feels sluggish.

## ROS2 Integration

Isaac Sim connects to ROS2 through Action Graphs - visual programming nodes that define data flow.

### Setting Up the ROS2 Context

Every Isaac Sim scene with ROS2 needs a context node:

1. Create an Action Graph
2. Add "ROS2 Context" node
3. Set `domain_id` to match your terminals (default: 0)

The domain ID must match between Isaac Sim and your ROS2 terminals, or topics won't be visible.

### Publishing Joint States

To publish wheel positions and velocities:

1. Add "Articulation State" node pointing to the robot
2. Connect to "ROS2 Publish Joint State" node
3. Set topic name: `/joint_states`

### Subscribing to Commands

To receive wheel velocity commands:

1. Add "ROS2 Subscribe Joint State" node
2. Set topic: `/joint_command`
3. Connect to "Articulation Controller" node
4. Set control type: "velocity"

## The Three Action Graphs

Rather than cramming everything into one monolithic action graph, I split the ROS2 integration into three focused graphs. Each handles a specific subsystem, making debugging easier and keeping the visual programming manageable.

### Mecanum Drive Controller

<img src="/images/actiongraph_ros2_mecanum_drive_sm.png" alt="Mecanum Drive Action Graph" style={{ width: '100%', height: 'auto', display: 'block', margin: '1rem 0' }} />

This is the core locomotion graph that handles bidirectional communication between ROS2 and the robot's wheels.

**What it does:**
- Subscribes to `/joint_command` for wheel velocity targets
- Applies wheel sign correction (FR, RR × -1) to normalize joint axis conventions
- Drives the Articulation Controller with velocity commands
- Publishes current joint states to `/joint_states` for odometry feedback

**Key nodes:**
- `ROS2 Context` - Establishes the ROS2 bridge with matching domain ID
- `ROS2 Subscribe Joint State` - Receives velocity commands from the policy controller
- `Articulation Controller` - Applies velocities to the simulated joints
- `Articulation State` - Reads current joint positions and velocities
- `ROS2 Publish Joint State` - Broadcasts state for external nodes

The sign correction happens between the subscriber and the articulation controller. Without it, sending identical velocities to all wheels makes the robot spin instead of driving straight.

### LIDAR Sensor Stream

<img src="/images/actiongraph_ros2_lidar_stream_sm.png" alt="LIDAR Stream Action Graph" style={{ width: '100%', height: 'auto', display: 'block', margin: '1rem 0' }} />

This graph handles the simulated RPLIDAR A1 sensor, streaming 2D laser scan data to SLAM and navigation nodes.

**What it does:**
- Reads from the RTX Lidar sensor attached to the robot
- Converts the point cloud to a 2D LaserScan message
- Publishes to `/scan` at the configured rate (~8 Hz to match the real A1)

**Key nodes:**
- `Isaac Read Lidar Point Cloud` - Pulls data from the RTX Lidar sensor
- `Isaac Create Render Product` - Sets up the rendering pipeline for the sensor
- `ROS2 Publish LaserScan` - Outputs sensor_msgs/LaserScan to `/scan`

**Configuration:**
```yaml
horizontal_fov: 360.0      # Full rotation
horizontal_resolution: 1.0  # 1 degree per sample
max_range: 12.0            # Matches RPLIDAR A1 spec
min_range: 0.15            # Minimum detection distance
```

The simulated LIDAR matches the real sensor's characteristics closely enough that slam_toolbox configuration works identically in both environments.

### RealSense Depth Camera

<img src="/images/actiongraph_ros2_real_sense_camera_sm.png" alt="RealSense Camera Action Graph" style={{ width: '100%', height: 'auto', display: 'block', margin: '1rem 0' }} />

This graph simulates the Intel RealSense D435 depth camera, providing both RGB and depth streams.

**What it does:**
- Captures RGB frames from the simulated camera
- Generates depth images from the camera's depth sensor
- Publishes synchronized image streams to standard RealSense topics

**Published topics:**
- `/camera/color/image_raw` - RGB image (sensor_msgs/Image)
- `/camera/depth/image_rect_raw` - Depth image (sensor_msgs/Image)
- `/camera/camera_info` - Camera intrinsics for 3D reconstruction

**Key nodes:**
- `Isaac Create Render Product` - Sets up the camera rendering pipeline
- `ROS2 Camera Helper` - Handles image encoding and publishing
- `Isaac Read Camera Info` - Extracts camera parameters

**Configuration:**
```yaml
resolution: [640, 480]
frame_rate: 30
depth_range: [0.2, 10.0]  # Matches D435 spec
```

The depth stream feeds into Nav2's voxel layer for 3D obstacle detection - critical for spotting obstacles below the LIDAR's scan plane like table legs or stairs.

### Why Three Separate Graphs?

Splitting into three graphs provides several advantages:

1. **Isolation** - A bug in the camera graph doesn't affect locomotion
2. **Performance** - Each graph can run at its optimal rate (wheels at 50Hz, LIDAR at 8Hz, camera at 30Hz)
3. **Debugging** - Easy to disable one sensor while testing another
4. **Reusability** - The mecanum drive graph works for any 4-wheeled robot

You could combine them into one graph, but the visual complexity becomes overwhelming quickly. Three focused graphs are much easier to maintain.

## Testing the Digital Twin

Once the model is configured, test it incrementally:

### Step 1: Basic Motion

```bash
# Terminal 1: Launch Isaac Sim, load ogre.usd, press Play

# Terminal 2: Send a velocity command
ros2 topic pub /joint_command sensor_msgs/msg/JointState \
  "{velocity: [1.0, 1.0, 1.0, 1.0]}" -r 10
```

All wheels should spin. If the robot spins in place instead of moving forward, check the wheel sign correction.

### Step 2: Omnidirectional Motion

```bash
# Forward
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist \
  "{linear: {x: 0.3}}" -r 10

# Strafe left
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist \
  "{linear: {y: 0.3}}" -r 10

# Rotate
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist \
  "{angular: {z: 0.5}}" -r 10
```

Test all motion primitives. The strafe test is the true mecanum validation - if it moves diagonally instead of purely sideways, check the wheel arrangement.

### Step 3: Sensor Verification

```bash
# Check LIDAR
ros2 topic echo /scan --once

# Visualize in RViz
rviz2 -d ~/ros2_ws/src/ogre-slam/rviz/isaac_sim.rviz
```

The LIDAR should show walls and obstacles in the simulation environment.

## Simulation vs. Real Robot

The key advantage of this setup: **the same ROS2 code runs in both environments**.

<table style={{ width: '100%', borderCollapse: 'separate', borderSpacing: '0' }}>
  <thead>
    <tr>
      <th style={{ textAlign: 'left', padding: '12px 16px', borderBottom: '2px solid #444' }}>Component</th>
      <th style={{ textAlign: 'left', padding: '12px 16px', borderBottom: '2px solid #444' }}>Simulation</th>
      <th style={{ textAlign: 'left', padding: '12px 16px', borderBottom: '2px solid #444' }}>Real Robot</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>LIDAR topic</td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>/scan</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>/scan</code></td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Odometry</td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Isaac Sim physics</td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Encoder-based</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Camera</td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Simulated RGB-D</td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>RealSense D435</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px' }}>Joint commands</td>
      <td style={{ padding: '12px 16px' }}><code>/joint_command</code></td>
      <td style={{ padding: '12px 16px' }}><code>/joint_command</code></td>
    </tr>
  </tbody>
</table>

The only difference is launch parameters. The same SLAM, navigation, and control code works in both places.

### Simulation-Only Flags

```bash
# Simulation
ros2 launch ogre_slam mapping.launch.py use_sim_time:=true

# Real robot
ros2 launch ogre_slam mapping.launch.py use_sim_time:=false
```

The `use_sim_time` flag tells ROS2 to use Isaac Sim's clock instead of wall clock. Without this, TF transforms time out because simulation time doesn't match real time.

## Common Issues

### Robot Falls Through Floor

**Cause:** Collision meshes not enabled or ground plane missing

**Solution:**
1. Ensure robot has collision enabled on all rigid bodies
2. Add a ground plane with collision

### Wheels Spin But Robot Doesn't Move

**Cause:** Wheel friction too low or joint damping too high

**Solution:**
1. Increase ground friction
2. Reduce joint damping
3. Check wheel contact with ground

### Robot Vibrates or Explodes

**Cause:** Physics instability from timestep or mass issues

**Solution:**
1. Reduce physics timestep (increase Hz)
2. Check mass distribution is reasonable
3. Reduce joint stiffness

### Topics Not Visible

**Cause:** ROS_DOMAIN_ID mismatch

**Solution:**
1. Check Isaac Sim ROS2 Context domain_id
2. Match in terminal: `export ROS_DOMAIN_ID=0`

---

## Quick Reference

### Key Files

<table style={{ width: '100%', borderCollapse: 'separate', borderSpacing: '0' }}>
  <thead>
    <tr>
      <th style={{ textAlign: 'left', padding: '12px 16px', borderBottom: '2px solid #444' }}>File</th>
      <th style={{ textAlign: 'left', padding: '12px 16px', borderBottom: '2px solid #444' }}>Purpose</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>usds/ogre_robot.usd</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Robot model with physics</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>usds/ogre.usd</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Complete scene with environment</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>actiongraph_ros2_mecanum_drive_sm</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>Wheel control and joint state publishing</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>actiongraph_ros2_lidar_stream_sm</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>LIDAR sensor simulation</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}><code>actiongraph_ros2_real_sense_camera_sm</code></td>
      <td style={{ padding: '12px 16px', borderBottom: '1px solid #333' }}>RGB-D camera simulation</td>
    </tr>
    <tr>
      <td style={{ padding: '12px 16px' }}><code>rviz/isaac_sim.rviz</code></td>
      <td style={{ padding: '12px 16px' }}>RViz config for simulation visualization</td>
    </tr>
  </tbody>
</table>

### Isaac Sim Settings

```yaml
Physics:
  timestep: 1/120 (120 Hz)
  solver_iterations: 4

Wheel Joints:
  max_velocity: 10000 rad/s
  damping: 10.0

Ground:
  static_friction: 0.8
  dynamic_friction: 0.6
```

## What's Next

With the digital twin working, [Part 3](/blog/project-ogre-part3-slam-mapping) covers using slam_toolbox to build maps. We'll run SLAM in both simulation and on the real robot, comparing results and tuning parameters for accurate mapping.

The ability to test SLAM in simulation before deploying to hardware saves significant time - you can iterate on configuration without draining the robot's battery or worrying about collisions.


Creating an accurate digital twin of the mecanum drive robot in NVIDIA Isaac Sim for simulation-first development and testing.

Posts tagged "isaac-sim"

Real-Time SO-ARM 101 ROS2 Teleoperation in Isaac Sim with LeRobot

Project Ogre Part 2: Building the Digital Twin in Isaac Sim