For Elon Musk and FSD team at Tesla, the road to full self-driving capability is paved with good intentions – and an increasingly scarce supply of meaningful training data. According to the Chief Twit himself, a mere 1/10,000th of the miles driven by Tesla’s fleet is actually useful for Model Training these days.
Accurate analysis.
Only about 1/10,000 of distance driven is useful for training.
The limiting factor for FSD progress was AI training compute, but now it is validation, as the interventions are so rare. https://t.co/SL0FEZ6zqk
— Elon Musk (@elonmusk) May 7, 2024
Elon highlighted this “fundamental bottleneck” after an investor video praising Tesla’s FSD v12 update. While better AI compute and model optimizations helped accelerate progress initially, the limiting factor now is validating those rare edge cases and disengagements as they become exponentially harder to come by.
It took 12 months to go from 1 disengagement every 100 miles to 1 every 200 miles, optimizing to 1 disengagement every 400 miles could take 24 months or more.
The current status quo puts Autopilot in an awkward spot. Third-party data indicates FSD v12 is cruising along with just 1 disengagement every 300 miles on average – an impressive feat, but one potentially skewed by sampling veteran Tesla owners acting as “OG testers.” The reality could be murkier, raising questions about when these systems will achievehuman parity, let alone 10x superiority.
If Autopilot’s models still lack robust reasoning despite their awe-inspiring stats, Musk warns that simply amassing more data may not cut it, even with Tesla’s fleet numbering over 1 million vehicles globally. Fundamental “qualitatively different approaches” may be needed to push past this bottleneck.
For a company hell-bent on solving self-driving, this invaluable data scarcity could prove to be Autopilot’s biggest roadblock yet. Either way, it’s a fascinating peek behind the curtain at the challenges facing one of AI’s most ambitious moonshots.