MRPX Demo – ROS Video & Hand Tracking

Stream ROS image topics and track hands with MediaPipe

⚠️ Watch this 5-minute tutorial before using the demo.
Tier: Free Time remaining today: --:-- AVAILABLE Queue: 0 To control: Request control / Turn on robot → Robot Control Mode → Enable Robot Control (hand or joystick).

Live Queue

Users waiting for robot control

No one is waiting right now.

Connection Status

ROS Bridge URL -
Video Base URL -
ROS Connection Disconnected

Robot Control Mode

Choose one control source for the da Vinci teleop stream. Only the active source publishes when you enable robot control.

Watch Hand Tracking Tutorial

Use your camera and hand gestures

Watch Joystick Tutorial

Learn physical joystick controls

Watch Xbox Controller Tutorial

Learn controller button mappings

Selected mode
Hand Tracking
Publishing
Disabled
Hand / camera
Not ready
Controller
—
Control Disabled
Gateway: disconnected
Reset: Idle

Returns the robot to its home pose. Uses the same control stream as joystick and hand tracking.

Controller status
Not connected
Controller name
No controller detected
Gripper command
0% closed
Gripper state
Open
Held object
Feedback unavailable
Extraction
Feedback unavailable
— —

Approach the ring, close the tip, keep it closed while lifting the ring off the peg, and open only after the ring is out.

Joystick mode requires Research, Pro, or Enterprise (logged-in account).

Surgical Robot Camera Views

Hand Tracking

Camera Status: Not started
Hands Detected: -
1.0
Scales x/y/z sent to robot control. Rotation (yaw) unchanged.
💡 If the camera won't start, refresh and allow camera permission.
Free control time: 5:00
Robot control is enabled from the Robot Control Mode panel above (Enable Robot Control). Only one source (hand or joystick) can stream at a time.

Hand

hand_x: -
hand_y: -
hand_z: -
hand_grasp: -
hand_conf: -

Target

target_x: -
target_y: -
target_z: -
target_jaw: -

Actual

actual_x: -
actual_y: -
actual_z: -
actual_jaw: -

Error

err_x: -
err_y: -
err_z: -
total: - mm

State

control: - | map: - | neutral: - | hold: - | orient: -
hand_age: - s | fresh: -
local frame: - | hands: - | stall: -
video t: - | 1st detect: - | pass#: - | ref: -
Y bias (err_y): - mm

Advanced controller interfaces require Research tier or above.

Research, Pro, and Enterprise unlock joystick, Xbox, and extended mapping.