Training SO-101s

SYSTEM OVERVIEW

At one of the largest Physical AI hackathons hosted by Founders, Inc., I trained an SO-101 robotic arm to autonomously perform a multi-step manipulation task: picking up a cup, pouring screws out, and precisely returning the cup to its original position. The system used Action Chunking Transformer (ACT)-based imitation learning with the LeRobot framework, demonstrating how high-quality, low-volume teleoperated demonstrations can outperform larger noisy datasets. This was my first time interfacing with robotic training systems, imitation learning pipelines, and production ML frameworks.

MY CONTRIBUTIONS

• 01

  Collected and curated approximately 50 high-quality teleoperated demonstrations, optimizing data efficiency for manipulation learning

• 02

  Pushed structured datasets to Hugging Face, enabling reproducible training and evaluation across team members

• 03

  Used Solo CLI to interface with LeRobot framework for streamlined recording, replay, and policy training workflows

• 04

  Tuned Action Chunking Transformer (ACT) training parameters to achieve stable convergence with minimal demonstration data

• 05

  Debugged perception-action alignment and action chunking mechanisms to improve task consistency across trials

• 06

  Successfully deployed trained policy to real SO-101 arm hardware at hackathon for live demonstration

RESULTS

• Achieved robust autonomous task execution after ~20,000 training steps despite limited demonstrations

• Successfully generalized across repeated trials without task-specific heuristics or hand-engineered controllers

• Demonstrated that high-quality, low-volume recordings significantly outperform larger noisy datasets

• Received strong positive feedback from industry professionals for system design, training efficiency, and real-world robustness