TeleopXR

TeleopXR transforms your VR/AR headset into a powerful, precise robot controller. It provides a lightweight, installation-free teleoperation interface with low-latency video streaming and full WebXR state tracking.

Key Features

• 🕶️ VR/Passthrough: Seamlessly switch between fully immersive VR and high-fidelity AR Passthrough modes, allowing you to choose between total focus and situational awareness.
• 📡 WebRTC Video Streaming: Get ultra-low latency, real-time video feedback directly in the headset, providing a near-instantaneous visual link to your robot's perspective.
• 🤖 Robot Visualization: Benefit from real-time 3D visualization of the robot model, ensuring your digital twin is always perfectly synchronized with the physical robot's state.
• 🕹️ Realtime Teleoperation based on Whole-Body IK: Achieve precise and intuitive control through advanced Whole-Body Inverse Kinematics, enabling complex coordination with minimal effort.

🚀 Quick Start (Demo)

Use the built-in demo to verify connectivity and visualize the XR state data in real-time.

pip install teleop-xr
python -m teleop_xr.demo

Modes

The demo supports two operation modes:

• Teleop Mode (Default): Visualizes raw XR state data and button events.

  python -m teleop_xr.demo --mode teleop
  

• IK Mode: Enables the high-performance IK solver (configured for Unitree H1 by default).

  python -m teleop_xr.demo --mode ik
  

Usage

1. Open the displayed URL (https://<ip>:4443) in your headset.
2. Enter VR mode.
3. Observe the live state data and event logs in your terminal.

📖 Documentation

For detailed guides on integrating TeleopXR into your own projects, including the Generic Python API and ROS2 Interface, please visit our official documentation website:

👉 https://qrafty-ai.github.io/teleop_xr/generic/

Development

For developers contributing to TeleopXR or customizing the frontend:

Prerequisites

• uv (for Python dependency management)
• Node.js & npm (for WebXR frontend)

Setup

1. Clone the repository:

   git clone https://github.com/qrafty-ai/teleop_xr.git
   cd teleop_xr
   

2. Install Python dependencies:

   uv sync
   

3. Build the WebXR frontend:

   cd webxr
   npm install
   npm run build
   

   (The build output will be used by the Python server)

4. Run from source:

   # From the root directory
   uv run python -m teleop_xr.demo
   

Acknowledgments

This project is forked from SpesRobotics/teleop. We are grateful for their foundational work in creating a WebXR-based teleoperation solution.

We also leverage powerful libraries for robotics:

• Pyroki: For high-performance, differentiable Inverse Kinematics and collision checking.
• Ballpark: For robust collision geometry generation and sphere decomposition.

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.