I really like LeRobot because of the abstraction that lets you record camera frames and proprioception at different frequencies and still get them aligned, a standardized dataset format, and CLI for training, evaluating, recording, and rollouts, with policies just being a plug and play flag value in their scripts
But not sure if its optimised and reliable for real world deployments
- If I want to train and deploy an embodiment I'm actually building (not SO-101/Aloha/Koch), I have to implement my own robot/motor/camera classes from scratch which is expected, but it does mean I'm not really getting LeRobot's abstractions for free, I'm reimplementing their interfaces every time.
- I wanted to run MolmoAct2 through their async inference setup, and it turns out async is only supports a fixed list of policies (ACT, SmolVLA, pi0, pi05, GR00T). MolmoAct2 isn't in that list, so I ended up writing my own runtime for it. This makes me dependent on LeRobot's support. I will eventually be building my own finetuned policies which I will be running on my specific embodiment
- The whole ecosystem is scoped to learned policies / VLAs. There's nothing for classical motion planning (MoveIt-style), grasp planning or segmentation before grasp, or a navigation/SLAM stack. I would ideally want a hybrid system where some dexterous tasks are being done by learned policies, but high volume, low variability tasks can be programmed without foundational models?
- I see companies like https://www.mimicrobotics.com/blog/solving-dexterity-a-full-stack-approach saying that they evaluated ROS2 and LeRobot's IPC layer for their real time control loops and found both fell short on latency and jitter, so they built their own zero copy middleware instead. They made the same call on telemetry: logging/recording has to be async and can't compete with the control loop for cycles, so they built a dedicated system for that too rather than bolting logging onto whatever was already there.
Curious to know if anyone tried using LeRobot, and either stuck to it or replaced it with their own stack?