The acknowledgements section in the README links to notebook, which is where OP sourced the techniques:<p><a href="https://www.kaggle.com/code/simjeg/platypus2-70b-with-wikipedia-rag" rel="nofollow">https://www.kaggle.com/code/simjeg/platypus2-70b-with-wikipe...</a><p>The notebook might be an easier read than the repo, but I haven't read either yet.<p>EDIT: It's very slow, according to the comments in this thread by people who tried it: <a href="https://github.com/oobabooga/text-generation-webui/issues/4754">https://github.com/oobabooga/text-generation-webui/issues/47...</a>
Could 2024 become a crisis for commercial AI?<p>1. We're only barely getting started with free MOE models and Mistral has already impressed.<p>2. Cloud AI is a poor fit for corporate use at least in EU due to GDPR, the NIS directive and more. You really dont want to exit the EU in where the data processing takes place.<p>3. There are indications of diminishing returns in LLM performance, where a one year later shot at it from Google, despite massive resources in terms of both experts and data set, still doesn't have Gemini Pro clearly surpass GPT 3.5 and Ultra probably not GPT 4. Meanwhile, competition like Mistral is closing in.<p>4. The NY Times lawsuit seems like a harbinger for what is to become a theme for AI companies in 2024. Open collaborations are harder to target as legal entities and there is not nearly as much money to gain if you win.<p>All this points toward a a) convergence of performance that will b) be in the benefit of open models to me.<p>Interesting times anyway especially as we are STILL only getting started.
Whoa, I don't understand enough to figure out if this is real and scalable or not, but if this is true it's a HUGE step forward. Can't wait to try and run a 70b LLM on my 32GB RAM desktop w/ Windows.
I did not dig too deep in the technicalities of it, but is there anything that would stop openAI from also implementing something like this ?<p>Presumably any advances open source community makes towards running on cheap hardware, will also massively benefit the big guys.