That title is most definitely a swipe at OpenAI. It seems there’s a theme of two types of platform companies: the closed one that has the “premium” platform, and the open one that gets market share by commodifying the platform.<p>macOS vs Windows<p>iOS vs Android<p>OpenAI vs Meta AI?<p>Edit: Another observation is Meta seems to open things defensively. Facebook had a rich developer platform until Facebook stopped needing to grab market share. Meta’s VR platform was closed until Apple challenged them with Vision Pro. Then Meta announced open sourcing Horizon OS. I wonder if Meta will truly keep things open if they win, or if it’s more like a case of EEE?
> "This effort pushed our infrastructure to operate across more than 16,000 NVIDIA H100 GPUs, making Llama 3.1 405B the first model in the Llama series to be trained at such a massive scale."<p>So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra).<p>Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up:<p>> "How much has Meta's valuation increased since they released their first open source model"<p>Answer (edited):<p>> Closing price on February 23, 2023: $509.50
> Closing price on October 11, 2024: $573.68
> The increase in stock price is $64.18 per share.
> Total increase = Price increase per share × Number of outstanding shares
> Total increase = $64.18 × 2,534,000,000 = $162,632,100,000
> Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023.<p>They seem to be making the right choices!
Free and open LLMs will be compelling to many of users that cannot or do not want to use online services.<p>It’s good that a large company like Meta seems to be pushing it forward with openllama 3.2.<p>Open, quality LLMs must lead to some tense discussions at OpenAI and Anthropic about whether they should be open or closed.<p>OpenAI Whisper is open source and it’s extremely good although it appeared to me the online version is much faster.
I wonder when Meta, Microsoft and OpenAI will partner on an open chip design to compete with NVIDIA.<p>They’re all blowing billions of dollars on NVIDIA hardware with like 70% margin and with triton backing PyTorch it shouldn’t be that hard to move off of CUDA stack.
And to power all those fused-multiply-add circuits, Meta will likely soon be involved in the construction or restart or life-extension of nuclear fission reactors, just like Microsoft and Google and Amazon (and Oracle, allegedly).<p>Quoting Yann LeCun (Vice-President, Chief AI Scientist at Meta):<p><pre><code> AI datacenters will be built next to energy production sites that can produce
gigawatt-scale, low-cost, low-emission electricity continuously.
Basically, next to nuclear power plants.
The advantage is that there is no need for expensive and wasteful
long-distance distribution infrastructure.
Note: Yes, solar and wind are nice and all, but they require lots of land
and massive-scale energy storage systems for when there is too little sun
and/or wind. Neither simple nor cheap.
</code></pre>
<a href="https://x.com/ylecun/status/1837875035270263014" rel="nofollow">https://x.com/ylecun/status/1837875035270263014</a>