Microsoft says GPT 3.5 has 20B parameters?

It&#x27;s against HN&#x27;s guidelines to editorialize titles like this. From <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html</a>:<p>&quot;<i>Please use the original title, unless it is misleading or linkbait; don&#x27;t editorialize.</i>&quot;<p>If you want to say what you think is important about an article, that&#x27;s fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else&#x27;s: <a href="https:&#x2F;&#x2F;hn.algolia.com&#x2F;?dateRange=all&amp;page=0&amp;prefix=false&amp;sort=byDate&amp;type=comment&amp;query=%22level%20playing%20field%22%20by:dang" rel="nofollow noreferrer">https:&#x2F;&#x2F;hn.algolia.com&#x2F;?dateRange=all&amp;page=0&amp;prefix=false&amp;so...</a>

I just want the 100k context window Anthropic gave Claude, but for GPT. Claude likes to hallucinate when I ask it to build chapter notes for a class I’m taking, and I don’t want to have to break up the text into tiny bits for GPT…

Link to the paper directly <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2310.17680.pdf" rel="nofollow noreferrer">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2310.17680.pdf</a><p>The system that they describe, Codefusion is interesting because it&#x27;s a diffusion model for generating code rather than an autoregressive model like most LLM code generators.

It says gpt3.5-turbo has 20B parameters, which I believe. But it says gpt3.5 (text-davinci-003) has 175B parameters, which I also believe.

This was expected because open source models of the same size already beat GPT-3.5 in many ways. And Mistral 7B makes you think if huge parameters are even needed for something like GPT-3.5 level.

I could be wrong but I believe the original comment that&#x27;s used as the title here comes from this tweet: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;felix_red_panda&#x2F;status&#x2F;1718916631512949248?t=jmCkeVH1Hyyu4vmwY-NhQg&amp;s=19" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;felix_red_panda&#x2F;status&#x2F;17189166315129492...</a>

Didn&#x27;t it leak earlier that it is 100 billion? And GPT 4 is 1.17 Trillion?

That sounds incredible given how powerful the model is.

Mhm, interesting development, the paper has been withdrawn.

That would certainly explain the pricing (gpt-3.5 vs davinci).

So it&#x27;s possible to run gpt-3.5-turbo on a local machine?

Did openAi ever publicize how many params GPT-4 had?

Just eating up water and fossil fuels.