Using the SO-100 with LeRobot
- A. Source the parts
- B. Install LeRobot
- C. Configure the motors
- D. Assemble the arms
- E. Calibrate
- F. Teleoperate
- G. Record a dataset
- H. Visualize a dataset
- I. Replay an episode
- J. Train a policy
- K. Evaluate your policy
- L. More Information
Follow this README. It contains the bill of materials, with a link to source the parts, as well as the instructions to 3D print the parts, and advice if it's your first time printing or if you don't own a 3D printer.
Before assembling, you will first need to configure your motors. To this end, we provide a nice script, so let's first install LeRobot. After configuration, we will also guide you through assembly.
Tip
We use the Command Prompt (cmd) quite a lot. If you are not comfortable using the cmd or want to brush up using the command line you can have a look here: Command line crash course
On your computer:
Copy paste in your shell: source ~/.bashrc
or for Mac: source ~/.bash_profile
or source ~/.zshrc
if you're using zshell
Video install instructions
lerobot-install.mp4
conda create -y -n lerobot python=3.10
Then activate your conda environment (do this each time you open a shell to use lerobot!):
conda activate lerobot
git clone https://github.com/huggingface/lerobot.git ~/lerobot
cd ~/lerobot && pip install -e ".[feetech]"
EXTRA: For Linux only (not Mac): install extra dependencies for recording datasets:
conda install -y -c conda-forge ffmpeg
pip uninstall -y opencv-python
conda install -y -c conda-forge "opencv>=4.10.0"
Great 🤗! You are now done installing LeRobot and we can begin assembling the SO100 arms 🤖.
Every time you now want to use LeRobot you can go to the ~/lerobot
folder where we installed LeRobot and run one of the commands.
Note
Throughout this tutorial you will find videos on how to do the steps, the full video tutorial can be found here: assembly video.
Designate one bus servo adapter and 6 motors for your leader arm, and similarly the other bus servo adapter and 6 motors for the follower arm. It's convenient to label them and write on each motor if it's for the follower F
or for the leader L
and it's ID from 1 to 6 (F1...F6 and L1...L6).
Video finding port
lerobot-find-motorbus-video.mov
lerobot-find-motorbus.mov
To find the port for each bus servo adapter, run the utility script:
python lerobot/scripts/find_motors_bus_port.py
Example output when identifying the leader arm's port (e.g., /dev/tty.usbmodem575E0031751
on Mac, or possibly /dev/ttyACM0
on Linux):
Finding all available ports for the MotorBus.
['/dev/tty.usbmodem575E0032081', '/dev/tty.usbmodem575E0031751']
Remove the usb cable from your DynamixelMotorsBus and press Enter when done.
[...Disconnect leader arm and press Enter...]
The port of this DynamixelMotorsBus is /dev/tty.usbmodem575E0031751
Reconnect the usb cable.
Example output when identifying the follower arm's port (e.g., /dev/tty.usbmodem575E0032081
, or possibly /dev/ttyACM1
on Linux):
Finding all available ports for the MotorBus.
['/dev/tty.usbmodem575E0032081', '/dev/tty.usbmodem575E0031751']
Remove the usb cable from your DynamixelMotorsBus and press Enter when done.
[...Disconnect follower arm and press Enter...]
The port of this DynamixelMotorsBus is /dev/tty.usbmodem575E0032081
Reconnect the usb cable.
On Linux, you might need to give access to the USB ports by running:
sudo chmod 666 /dev/ttyACM0
sudo chmod 666 /dev/ttyACM1
IMPORTANTLY: Now that you have your ports, update the port default values of SO100RobotConfig
. You will find something like:
@RobotConfig.register_subclass("so100")
@dataclass
class So100RobotConfig(ManipulatorRobotConfig):
calibration_dir: str = ".cache/calibration/so100"
# `max_relative_target` limits the magnitude of the relative positional target vector for safety purposes.
# Set this to a positive scalar to have the same value for all motors, or a list that is the same length as
# the number of motors in your follower arms.
max_relative_target: int | None = None
leader_arms: dict[str, MotorsBusConfig] = field(
default_factory=lambda: {
"main": FeetechMotorsBusConfig(
port="/dev/tty.usbmodem58760431091", <-- UPDATE HERE
motors={
# name: (index, model)
"shoulder_pan": [1, "sts3215"],
"shoulder_lift": [2, "sts3215"],
"elbow_flex": [3, "sts3215"],
"wrist_flex": [4, "sts3215"],
"wrist_roll": [5, "sts3215"],
"gripper": [6, "sts3215"],
},
),
}
)
follower_arms: dict[str, MotorsBusConfig] = field(
default_factory=lambda: {
"main": FeetechMotorsBusConfig(
port="/dev/tty.usbmodem585A0076891", <-- UPDATE HERE
motors={
# name: (index, model)
"shoulder_pan": [1, "sts3215"],
"shoulder_lift": [2, "sts3215"],
"elbow_flex": [3, "sts3215"],
"wrist_flex": [4, "sts3215"],
"wrist_roll": [5, "sts3215"],
"gripper": [6, "sts3215"],
},
),
}
)
Let's begin with assembling the follower arm base
Video configuring motor
lerobot-setid.mov
lerobot-configure-motor.mov
Plug your first motor F1 and run this script to set its ID to 1. It will also set its present position to 2048, so expect your motor to rotate. Replace the text after --port to the corresponding follower control board port and run this command in cmd:
python lerobot/scripts/configure_motor.py \
--port /dev/tty.usbmodem58760432961 \
--brand feetech \
--model sts3215 \
--baudrate 1000000 \
--ID 1
Note
These motors are currently limited. They can take values between 0 and 4096 only, which corresponds to a full turn. They can't turn more than that. 2048 is at the middle of this range, so we can take -2048 steps (180 degrees anticlockwise) and reach the maximum range, or take +2048 steps (180 degrees clockwise) and reach the maximum range. The configuration step also sets the homing offset to 0, so that if you misassembled the arm, you can always update the homing offset to account for a shift up to ± 2048 steps (± 180 degrees).
Then unplug your motor and plug the second motor and set its ID to 2.
python lerobot/scripts/configure_motor.py \
--port /dev/tty.usbmodem58760432961 \
--brand feetech \
--model sts3215 \
--baudrate 1000000 \
--ID 2
Redo the process for all your motors until ID 6. Do the same for the 6 motors of the leader arm.
Video removing gears
lerobot-remove-gears-small.mp4
Follow the video for removing gears. You need to remove the gear for the motors of the leader arm. As a result, you will only use the position encoding of the motor and reduce friction to more easily operate the leader arm.
Video adding motor horn
lerobot-adding-motor-horn.mov
Follow the video for adding the motor horn. For SO-100, you need to align the holes on the motor horn to the motor spline to be approximately 1:30, 4:30, 7:30 and 10:30. Try to avoid rotating the motor while doing so to keep position 2048 set during configuration. It is especially tricky for the leader motors as it is more sensible without the gears, but it's ok if it's a bit rotated.
Video assembling arms
lerobot-assemble-arm.mov
Follow the video for assembling the arms. It is important to insert the cables into the motor that is being assembled before you assemble the motor into the arm! Inserting the cables beforehand is much easier than doing this afterward. The first arm should take a bit more than 1 hour to assemble, but once you get used to it, you can do it under 1 hour for the second arm.
Next, you'll need to calibrate your SO-100 robot to ensure that the leader and follower arms have the same position values when they are in the same physical position. This calibration is essential because it allows a neural network trained on one SO-100 robot to work on another.
Important
Contrarily to step 6 of the assembly video which illustrates the auto calibration, we will actually do manual calibration of follower for now.
You will need to move the follower arm to these positions sequentially:
1. Zero position | 2. Rotated position | 3. Rest position |
---|---|---|
![]() |
![]() |
![]() |
Make sure both arms are connected and run this script to launch manual calibration:
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--robot.cameras='{}' \
--control.type=calibrate \
--control.arms='["main_follower"]'
Follow step 6 of the assembly video which illustrates the manual calibration. You will need to move the leader arm to these positions sequentially:
1. Zero position | 2. Rotated position | 3. Rest position |
---|---|---|
![]() |
![]() |
![]() |
Run this script to launch manual calibration:
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--robot.cameras='{}' \
--control.type=calibrate \
--control.arms='["main_leader"]'
Simple teleop Then you are ready to teleoperate your robot! Run this simple script (it won't connect and display the cameras):
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--robot.cameras='{}' \
--control.type=teleoperate
Follow this guide to setup your cameras. Then you will be able to display the cameras on your computer while you are teleoperating by running the following code. This is useful to prepare your setup before recording your first dataset.
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--control.type=teleoperate
Once you're familiar with teleoperation, you can record your first dataset with SO-100.
If you want to use the Hugging Face hub features for uploading your dataset and you haven't previously done it, make sure you've logged in using a write-access token, which can be generated from the Hugging Face settings:
huggingface-cli login --token ${HUGGINGFACE_TOKEN} --add-to-git-credential
Store your Hugging Face repository name in a variable to run these commands:
HF_USER=$(huggingface-cli whoami | head -n 1)
echo $HF_USER
Record 2 episodes and upload your dataset to the hub:
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--control.type=record \
--control.fps=30 \
--control.single_task="Grasp a lego block and put it in the bin." \
--control.repo_id=${HF_USER}/so100_test \
--control.tags='["so100","tutorial"]' \
--control.warmup_time_s=5 \
--control.episode_time_s=30 \
--control.reset_time_s=30 \
--control.num_episodes=2 \
--control.push_to_hub=true
Note: You can resume recording by adding --control.resume=true
.
If you uploaded your dataset to the hub with --control.push_to_hub=true
, you can visualize your dataset online by copy pasting your repo id given by:
echo ${HF_USER}/so100_test
If you didn't upload with --control.push_to_hub=false
, you can also visualize it locally with (a window can be opened in the browser http://127.0.0.1:9090
with the visualization tool):
python lerobot/scripts/visualize_dataset_html.py \
--repo-id ${HF_USER}/so100_test \
--local-files-only 1
Now try to replay the first episode on your robot:
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--control.type=replay \
--control.fps=30 \
--control.repo_id=${HF_USER}/so100_test \
--control.episode=0
To train a policy to control your robot, use the python lerobot/scripts/train.py
script. A few arguments are required. Here is an example command:
python lerobot/scripts/train.py \
--dataset.repo_id=${HF_USER}/so100_test \
--policy.type=act \
--output_dir=outputs/train/act_so100_test \
--job_name=act_so100_test \
--device=cuda \
--wandb.enable=true
Let's explain it:
- We provided the dataset as argument with
--dataset.repo_id=${HF_USER}/so100_test
. - We provided the policy with
policy.type=act
. This loads configurations fromconfiguration_act.py
. Importantly, this policy will automatically adapt to the number of motor sates, motor actions and cameras of your robot (e.g.laptop
andphone
) which have been saved in your dataset. - We provided
device=cuda
since we are training on a Nvidia GPU, but you could usedevice=mps
to train on Apple silicon. - We provided
wandb.enable=true
to use Weights and Biases for visualizing training plots. This is optional but if you use it, make sure you are logged in by runningwandb login
.
Training should take several hours. You will find checkpoints in outputs/train/act_so100_test/checkpoints
.
You can use the record
function from lerobot/scripts/control_robot.py
but with a policy checkpoint as input. For instance, run this command to record 10 evaluation episodes:
python lerobot/scripts/control_robot.py \
--robot.type=so100 \
--control.type=record \
--control.fps=30 \
--control.single_task="Grasp a lego block and put it in the bin." \
--control.repo_id=${HF_USER}/eval_act_so100_test \
--control.tags='["tutorial"]' \
--control.warmup_time_s=5 \
--control.episode_time_s=30 \
--control.reset_time_s=30 \
--control.num_episodes=10 \
--control.push_to_hub=true \
--control.policy.path=outputs/train/act_so100_test/checkpoints/last/pretrained_model
As you can see, it's almost the same command as previously used to record your training dataset. Two things changed:
- There is an additional
--control.policy.path
argument which indicates the path to your policy checkpoint with (e.g.outputs/train/eval_act_so100_test/checkpoints/last/pretrained_model
). You can also use the model repository if you uploaded a model checkpoint to the hub (e.g.${HF_USER}/act_so100_test
). - The name of dataset begins by
eval
to reflect that you are running inference (e.g.${HF_USER}/eval_act_so100_test
).
Follow this previous tutorial for a more in-depth tutorial on controlling real robots with LeRobot.
Tip
If you have any questions or need help, please reach out on Discord in the channel #so100-arm
.