More info than the twitter post:<p><a href="https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion" rel="nofollow">https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion</a><p>No idea what this thing is, but it appears to be some sort of AI-related thing:<p>><i>While it’s unclear what Microsoft is specifically using our models for, it is believed, this is in preparation for local Co-pilot running with on-device models</i>
For those wondering, the RWKV architecture is an alternative to a transformer arch, and has the nice property of allowing very long inputs. Speculation here that it might be for code assistance would make sense. Early versions of RWKV that I played with took a long time to tokenize input strings, but generation was quick. I could imagine engineering finding a good fit with a codebase that’s going to be mostly static while an engineer is editing only parts of it.
Hey there, Twitter author / guy from the RWKV team.<p>Some updates: Its also arriving to windows 10 (so that's 1.5B deploys)<p>Our main guess would be the copilot beta features being tested<p>- local copilot<p>- local memory recall<p>And it makes sense, especially for our smaller models<p>- we support 100+ languages<p>- we are extremely low in energy cost<p>If anyone has a machine with local copilot / memory recall enabled, please reach out to me on my twitter @picocreator - I want to decompile and trace this down =)
I didn’t know what RWKV.cpp was, so hopefully this helps others: <a href="https://github.com/RWKV/rwkv.cpp">https://github.com/RWKV/rwkv.cpp</a><p>> <i>INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model</i>
“Silently overnight, it’s everywhere, in every Windows 10 and 11 PC.”<p>So they are willing to deploy AI BS to an “old” system (win10), rather than the amd performance fix users desperately need.