Karpathy's presentation is really good. Watch it later if you get some time. The key points are<p>- Telsa is using fleet for learning (Fleet learning). This has 3 components,
a. Trigger infrasture that collects the kind of data telsa is looking for training. The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly.
b. Data Engine which learns from these examples.
c. Shadow mode where they deploy the learned model and test how its doing in real world and iterate this process.<p>- Drivers are themselves acting as labelers and tesla is in a unique position to take advantage of this.<p>- People drive vision only. Visual recognition is essential for autonomy. Lidar has much less information than vision, for example to identify the thing on the road is a plastic bag, lidar provides few points. But vision gives more data for telling this.
I'm sorry but I'm finding it really difficult to watch this and match up what the engineer is saying to what Elon Musk is saying.<p>For example, the engineer says the custom ASIC does 144 TOps for 2 chips vs the NVidia drive Xavier - does 21 TOps. Okay, well yeah I expect your custom ASIC does have a nice performance advantage over the equivalent GPU. at 3.5x advantage probably seems reasonable. Cue Elon Musk:<p>"At first seems improbable, how could it be that Tesla, who has never designed a chip before would design the best chip in the world but that is objectively what has occured. Not the best by a small margin, the best by a huge margin".<p>Mate, it's a dot product with some memory attached, and not a single detail your half hour deep dive has gone into suggests anything other than a bog standard ASIC.<p>"All the cars being produced right now have all the hardware necessary for full self-driving"<p>And this is where I'm totally lost. I want to believe! But he's lied so many times now. This man is sucking the credibility out of every engineer in the room. Don't repeat the same lie twice.
I watch Tesla fairly closely from both the bull and bear sides. In short, I don't believe Tesla is anywhere close to Waymo. They won't achieve FSD in 2020.<p>It's important to see this event in the context of their significant demand and cash problems. Even enthusiastic Tesla investors like Galileo Russell suggested that Tesla is in a cash crunch and should raise money. Which hints at the main mystery about Tesla, why haven't they raised money yet?<p>Take a moment and think about <i>why</i> are they doing this event now. Elon is setting a stage for a capital raise, he's pitching the autonomy narrative after the Model 3 cash cow narrative failed. They're trying to convince investors (and customers) to give them money because money-printing autonomous taxi service is coming next year.<p>Also, think about why basically everyone except Tesla uses Lidars. Is it because they're stupid or because Tesla <i>cannot</i> use Lidars even if they wanted to?<p>P.S.: Nvidia issued a statement saying that Tesla's claims about their chip are incorrect: <a href="https://www.marketwatch.com/story/nvidia-says-tesla-inaccurate-in-self-driving-comparison-2019-04-22" rel="nofollow">https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...</a><p>Edit: In 2012, Waymo reached the milestone of handling 8 100-mile routes, specifically chosen to capture the full complexity of driving. I doubt Tesla is currently at that level. Source: <a href="https://events.technologyreview.com/video/watch/dmitri-dolgov-waymo-autonomous-cars/" rel="nofollow">https://events.technologyreview.com/video/watch/dmitri-dolgo...</a>
...out of all the things I expected to see from a Tesla stream, I definitely did not expect a 25 minute discussion on the cost of 32bit additions, dot products, sram bandwidth, and chip design, at the level of a third year college hardware course.
Holy cow Elon said <i>a lot</i> of things that will <i>drastically</i> change the world in the next 2-3-5 years. I think it's very, very clear they've been doing <i>a lot</i> of stuff behind that scenes that nobody has given them credit for over these last 5 years. Everyone thought they were just throwing everything at the Model 3 ramp and doing nothing else, but it turns out the opposite is true.<p>At the end Elon said autonomy is basically their entire expense sheet!<p>Whether or not Tesla can pull it off is obviously going to be an enormous topic of debate with haters and lovers on each side.<p>This is super, super exciting. I'm going to grab the popcorn and enjoy watching Tesla try. Whether they succeed or fail I admire them aiming so high, and planning so far ahead.
So far, half an hour from the head of IC design. Nice special purpose IC and board. Dual everything for redundancy. Wide special purpose neural net evaluation. 100 watts for the compute system. Code signing. Shipping in new Model 3 cars since last 10 days.<p>"All we need to do is improve the software" - Musk [12:07 PDT]<p>LIDAR "unnecessary" - Musk [12:13 PDT]<p>Computer vision guy is now speaking.<p>Recognizes "driveable space", not just obstacles. Video shown, but just for a freeway. This is crucial to safety. Need to see this is a cluttered environment.
The thing I don't get about Tesla is why they keep heaping more on their plate rather than <i>just</i> (for large values of just) re-doing the motive power bit. That alone is a massive challenge and it is not something they've licked at economies of scale that allow a $10K vehicle to be brought to market, which is what it will take to really make this a success.<p>Every extra they tack on to the base product is one they will be expected to deliver on as well diverting attention from the main problem they are dealing with, which may in the longer term leave open enough room for the competition to wiggle through.<p>Sure, autonomy is a big deal, but it is <i>also</i> something that will once cracked be an instant commodity and there is a lot of money and talent focused on that particular problem, which is surprisingly hard to do well.<p>If Tesla ends up going under because of one of the side shows (Solar City, autonomy, Power Wall etc) that would be a serious loss.
Interesting pieces:<p>- First principles hardware design of focused self driving computer (many times better than any competing existing hardware). Already shipping in all newly produced cars. Currently working on next gen that will be 3x better to ship in a couple years.<p>- Lidar is an unnecessary mistake that competitors are making that won't succeed (too expensive, need too many, unnecessary).<p>- Real world fleet testing is critical to success, simulations are not good enough since there are too many unknown unknowns in the real world. Tesla uses simulations too, but nobody else comes close on real world fleet testing.
Tesla: "We have a global network of cameras that can be queried to find anything on or near roads and send back photos of it."<p>Every law enforcement entity on the planet: <i>drool</i>
At the tail end of Karpathy's presentation, he said something that reminded me of why Peter Norvig decided to join Google: because that's where the data is. In Tesla's case, they have a unique and likely accelerating advantage in having the best source of data up on which to train their models. I think this forms the basis of an enduring moat relative to their other competitors, none of which are collecting data at the same scale.
While you're waiting for the main event to start, here are some recent interviews with Elon about self-driving cars. He's <i>very</i> confident.<p>"To me right now, this seems 'game, set, and match,'" Musk said. "I could be wrong, but it appears to be the case that Tesla is vastly ahead of everyone."<p>I am eager to see what they unveil today.<p><a href="https://www.youtube.com/watch?v=dEv99vxKjVI" rel="nofollow">https://www.youtube.com/watch?v=dEv99vxKjVI</a><p><a href="https://ark-invest.com/research/podcast/elon-musk-podcast" rel="nofollow">https://ark-invest.com/research/podcast/elon-musk-podcast</a>
I really really don't want Tesla to die. I think it's an important company for a sustainable future. Failures in autonomous driving could easily turn into the straw that breaks the camels back.<p>If you call something "autopilot" and promote it as if it will drive for you and then it ends up killing dozens of people ... that's where successful class actions come from.<p>(I obviously don't want people to die either.)
This focus on the hardware is silly. Assume for a second that their new hardware is 50x faster than their last hardware.<p>That does not mean that their cars can self drive today.<p>That does not mean that their cars can self drive three years from now.<p>It's 100% not proven or obvious how car self driving skill and car self driving error rates scale with compute -- but it's surely not linear.
Musk: "You're only going real fast in the forward direction."<p>Dozens of times in my life, I have been driving down the freeway at 70 MPH in a 60 MPH zone, and I've noticed someone weaving through traffic behind me, going closer to 140 MPH.<p>I need to know whether to change lanes, stay in my lane, stop changing lanes, pump my breaks to indicate there is slowdown ahead of me that car might not see, etc.<p>Just food for thought.<p>EDIT: I'd also like my vehicle to be good at avoiding a car that's about to T-Bone me, at night, with no headlights on. I may not be very good at avoiding that kind of accident today, but if LIDAR is necessary to protect me from that kind of accident, then I might think it's a wonderful idea.
Those are some very bold claims. Level 5 by the end of this year? I find the software approach intriguing and Karpathy's segment was enlightening and did a lot to convince me of Tesla's advantages.<p>On the other hand, I've been sensing that Tesla is finding it harder and harder to raise cash and has been getting increasingly desperate. Are we on the cusp of a new transformative technology or the peak of the mother of all bubbles? Time will tell.
Here's the new self-driving demo video from today.[1] From Tesla's HQ in Palo Alto, out to I-280, down one exit, use interchange at Sand Hill to turn around, come back. No visible conflicting traffic on non-freeway streets.<p>Compare the 2016 demo video.[2] That's a tougher route. That's the one where we now know it took a lot of tries to get a clean video.<p>Waymo and Cruise have put up videos of their cars in city traffic. They get criticized for things like getting stuck behind double-parked cars, and being a bit shy of parked cars that project into a traffic lane. But they get where they are going. Tesla is not showing anything near that level.<p>Supposedly the analysts at the meeting got to ride in a self-driving car. Anyone seen reports from them?<p>[1] <a href="https://www.youtube.com/watch?v=tlThdr3O5Qo" rel="nofollow">https://www.youtube.com/watch?v=tlThdr3O5Qo</a>
[2] <a href="https://www.youtube.com/watch?v=eAal0juXXzU" rel="nofollow">https://www.youtube.com/watch?v=eAal0juXXzU</a>
Feels like they're just throwing a bunch of hardware specs at investors to distract from any discussions about the current state of the software beyond "just need to improve it now".
You cannot access the stream from Tesla anymore, I found it here - <a href="https://www.youtube.com/watch?v=tbgtGQIygZQ" rel="nofollow">https://www.youtube.com/watch?v=tbgtGQIygZQ</a>
Karpathy did a great job explaining this, I took Neural Networks a long time back and his back to basics approach refreshed a lot of concepts I'd forgotten!
Currently just showing some generic car footage. Apparently they will demo their self-drivin features: <a href="https://www.teslarati.com/tesla-self-driving-autonomy-day-what-to-expect/" rel="nofollow">https://www.teslarati.com/tesla-self-driving-autonomy-day-wh...</a>
“Anyone using Lidar is doomed”<p>Strong words from Musk about sticking with video only.<p>Later on in the software talk:<p>“Lidar is really a shortcut which sidesteps the fundamental problems...and gives us a false sense of progress”
Seems to me that relying too much on Machine Learning for this is actually the real risk, not LIDAR. Tesla's competitors are using ML, but combining it with sensor fused data from LIDAR, Radar, accurate maps, and visual data. In other words, they're starting from a view of the world with super-human sensing.<p>Starting from a purely visual domain likely dooms your system to making the same types of mistakes that humans make visually in judgement. At least with accurate maps and LIDAR as backup, you can sanity check output of your visual processing against a map and LIDAR. If your claim is that the map might be out of date, or the LIDAR too low rez, it still helps to err on the size of caution.<p>In the worst case, the map tells you you can't go somewhere that you're allowed to go. In the case where it says you are allowed to go somewhere were you shouldn't, well your visual system should be telling you not to go there. If it isn't, you're in a lot worse trouble than having a map with flaws.<p>Likewise, the claims that "LIDAR is expensive" is like claiming EVs are expensive because "batteries are expensive!" If AVs become common, then they'll be a huge demand for LIDAR and the costs will decline. LIDAR costs are already declining. Maybe if Elon took some of Tesla's miracle engineers and had them make a LIDAR, they could not only defeat Nvidia at chip making, but defeat all LIDAR makers as well. (sarcasm) My guess is the real problem with LIDAR is drag and vehicle trim/styling.<p>At this point, Tesla is basically stuck. They bet big early on a shitty sensor suite before AV technology had been worked out, and wave their hands about how using consumers as guinea pigs to feed them fleet data will magically fix deficiencies in sensors. Well, what if this is a false hope and it doesn't? It would mean they'd get sued by everyone who bought the AV suite as an option and have to issue refunds or recalls to upgrade.<p>I've said it before and I'll say it again, you don't ship AV as an MVP on a $100k vehicle and promise magic upgrades and fixes later before the technology is even close. It's risking people's lives and it's already killed people.
Q: "What's the primary design objective of the next-gen chip?"<p>Mumbled Answer: "Safety."<p>...doesn't that mean the current-gen chip... isn't as safe as you want?
Man -- his engineers must cringe quite a bit when he talks about a future self-driving architecture built on DOJO with 'video-in, actuator controls out' ... just a tiny bit of speculation goes into saying something like that ... To my mind, it undermines the presentation quite a bit which otherwise appears to mainly argue that they will reach full self-driving by iterating on their neural network sensor perception model.
Some video of demo rides from someone who was at the day:<p>Lane change on crowded freeway: <a href="https://twitter.com/hamids/status/1120472369762590722" rel="nofollow">https://twitter.com/hamids/status/1120472369762590722</a> - It manages it if not quite as smooth as a human.<p>Summon in parking lot: <a href="https://twitter.com/hamids/status/1120446405410205701" rel="nofollow">https://twitter.com/hamids/status/1120446405410205701</a> likewise.<p>There's also an official Tesla sped up demo vid <a href="https://www.youtube.com/watch?v=tlThdr3O5Qo" rel="nofollow">https://www.youtube.com/watch?v=tlThdr3O5Qo</a> - hard to tell really.<p>Same video slowed with the display magnified and some Reddit discussion <a href="https://www.reddit.com/r/teslamotors/comments/bgb9or/fsd_demo_slowed_down_with_blownup_augmented/" rel="nofollow">https://www.reddit.com/r/teslamotors/comments/bgb9or/fsd_dem...</a>
I think it’s a safe assumption that driver alertness positively correlates with lives saved. But it’s hard to market this product with the sort of rhetoric that would optimize for that, so they’ve found a few ways to console themselves as they name the feature “autopilot” and endeavor to make the tech appear magical.<p>If they think they’ve cleared the bar just by beating the crash statistics of all cars, then they are still in the business of trading lives.
Here is the link to the recording:
<a href="https://www.youtube.com/watch?v=Ucp0TTmvqOE" rel="nofollow">https://www.youtube.com/watch?v=Ucp0TTmvqOE</a>
What kind of event is this and who is it targeted to? Right now there’s a comparison of energy consumption for different instructions on processors, I’m very confused.
To summarize, it appears that their new chips can do 96x96 dot product in a single cycle (multiplying neural network weights by values), and have hardware for activations (ReLU, softmax, tanh) and pooling (as in convolutional neural networks). This results in a crazy 144 trillions ops/second.<p>How does this compare to TPUs and the Neural Engine in iPhone CPUs?
It's interesting to contrast Musk's stated aversion to LIDAR with his previous statements about new technologies always starting at the high end and working their way down [1]. Tesla appears to be trying to start at the low end (HW is included with the purchase of a car even if you decide not to opt for the software to make it function. You might imagine that you would start with all the bells and whistles and pare them down as the technology matures and you invent ways to do without.<p>[1]<a href="https://twitter.com/elonmusk/status/1075126514851602432" rel="nofollow">https://twitter.com/elonmusk/status/1075126514851602432</a>
How can Musk make the prediction of a million autonomous taxis in less than two years? That's just fantasy. There are just too many unknowns to solve. One of the big ones is permission from the city governments. Once a car hits a pedestrian or multiple taxies start having accidents, governments will bring the project to a halt. Tesla might have a small test fleet somewhere but not 1 million taxis.<p>My guess is that Tesla will need another round of financing along the way and they have to make sure that the stock's price stays up. Musk is a great promoter but at some point he needs to come back to reality.
I wonder if they are going to address potential limitations to their systems (bad weather, snow etc). Their tech seems absolutely impressive but I suppose there will always be failure. Does the system fail "gracefully" ?
New link: <a href="https://www.youtube.com/watch?v=tbgtGQIygZQ" rel="nofollow">https://www.youtube.com/watch?v=tbgtGQIygZQ</a><p>(replay after the livestream ended)
It seems like Tesla is at a pivotal moment.<p>On one hand there are hyper-bulls who claim Tesla is a $4000 stock and the future of transportation. On the other, hyper-bears claim the equity should trade around $0-$10. There seems to be no middle ground.<p>It seems like they are almost betting the company on FSD. I don’t think FSD is really even close to a possibility over the next 5-10yrs. I hope I’m wrong, but if I’m right, I don’t see how Tesla keeps going on like this.
I have several questions about Karpathy's presentation:
1. How to label automatically using fleet learning? An example he gave is detecting cut-ins. However, I am wondering if it could be applied to more general and basic cases, for example, labeling lane marking/vehicles.
2. How do you know if someone is a good driver (which is essential for imitation learning)?
I read through all comments. Very informative.<p>How many people and companies are going to freak out if Elon and his team deliver FSD without LIDAR? Say in 2022, not even 2020. My layman logic is that AI is improving exponentially, and we do not know what artificial vision can do in 2 years.
link to the livestream video on youtube: <a href="https://www.youtube.com/watch?v=Ucp0TTmvqOE&t=4150s" rel="nofollow">https://www.youtube.com/watch?v=Ucp0TTmvqOE&t=4150s</a>