It indeed works and loads quick. I am more interested currently in the Vicuna 7B example from <a href="https://mlc.ai/web-llm/" rel="nofollow">https://mlc.ai/web-llm/</a><p>Also instead of just "Update Chrome to v113" the domain owner could sign up for an origin trial <a href="https://developer.chrome.com/origintrials/#/view_trial/118219490218475521" rel="nofollow">https://developer.chrome.com/origintrials/#/view_trial/11821...</a>
My 250M parameter model runs in 50ms/token ;)<p>Releasing April 26th when Chrome 113 hits stable. Open source NPM library you can add to any project.<p>Preview here:
<a href="https://twitter.com/fleetwood___/status/1646608499126816799?s=20" rel="nofollow">https://twitter.com/fleetwood___/status/1646608499126816799?...</a>
<p><pre><code> > WebGPU is supported in your browser!
> Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds.
</code></pre>
Anyone using Chromium care to chime in?<p>If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.
Question. I can see in the code the WGSL that's needed to implement inference on the GPU. <a href="https://github.com/0hq/WebGPT/blob/main/kernels.js">https://github.com/0hq/WebGPT/blob/main/kernels.js</a><p>Could this code also be used to train models or only for inference?<p>What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?
> At the moment, WebGPT averages ~300ms per token on GPT-2 124M running on a 2020 M1 Mac with Chrome Canary.<p>How do ChatGPT on GPT-3.5 / GPT-4 compare?
It's really a shame that there is no 8-bit float support in the WebGPU spec. Even though few cards support it natively, it'd still massively benefit ML workloads.<p>Another annoying constraint but specific to wgpu (Rust's implementation of WebGPU) is that it does not support f16 yet (which IS in the spec), only through SPIR-V passthrough...
Not available in Firefox yet:
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1602129" rel="nofollow">https://bugzilla.mozilla.org/show_bug.cgi?id=1602129</a>