In the fast-paced race to develop advanced humanoid robots, Tesla is doubling down on dexterity. CEO Elon Musk revealed that the company’s (Optimus robot handling real-world tasks via running entirely end-to-end), will sport a new hand later this year packing an impressive 22 degrees of freedom (DOF) – double its current capabilities, “and the actuators will move almost entirely into the forearm, just like how humans work,” Elon Musk says.
The upgraded hand, with actuators moving into the forearm like humans, shows Tesla isn’t messing around when it comes to Optimus. “They’ve clearly made dexterity a key priority,” said robotics expert Jim Fan, NVIDIA’s Sr. Research Manager and Lead of Embodied AI.
Achieving such high dexterity is no easy feat, as Jim outlines, developing an advanced humanoid robot hand requires solving multiple complex challenges:
Human Data Collection at Massive Scale, “Optimus’ biggest lead is their human data collection farm,” Jim Fan said after analyzing Tesla’s update video. Having humans teleoperate the robot to gather data across factories, homes and more environments is critical but incredibly complex.
“You need a sizeable fleet, well-trained contractors working multiple shifts, an on-call maintenance crew – that’s a ton of operational complexity academic labs don’t even think of,” he said.
Robust Hardware Design, Optimus’ hands are “among the best 5-finger, dexterous robot hands in the world,” according to Jim. With 11 DOF, tactile sensing and robustness for object interaction, the hardware has to be well-designed.
Ultra-Low Latency Teleoperation, Streaming first-person video to VR operators, while translating their precise hand motions to the robot at extremely low latency is non-trivial. “Humans are highly sensitive to even the smallest delay between their motions and the robot’s,” Fan notes.
Task Selection for Maximal Generalization, Figuring out the optimal tasks and environments for data collection is an open research question. “If you only have the budget to collect training data for 1,000 tasks, what would you pick to maximize skill transfer and generalization?” Jim Fan posits.
The path ahead won’t be easy, but few expect the relentlessly driven Musk to back down from this mission. After all, as he noted, there are powerful synergies between Optimus and Tesla’s electric vehicles, which both use end-to-end neural networks and the company’s FSD chips.
“Congrats to the Tesla Optimus team on another stellar update,” Jim concluded. “The video gives us a peek at their human data collection farm, which I believe is Optimus’ biggest lead. What does it take to build such a pipeline? Optimus nailed all of the following…”