ChatGPT turns one today :)<p>What a day to launch the project I'm tinkering with for more than half a year. Welcome new LLM platform suited both for individual research and scaling AI services in production.<p>GitHub: <a href="https://github.com/gotzmann/collider">https://github.com/gotzmann/collider</a><p>Some superpowers:<p>- Built with performance and scaling in mind thanks Golang and C++<p>- No more problems with Python dependencies and broken compatibility<p>- Most of modern CPUs are supported: any Intel/AMD x64 platofrms, server and Mac ARM64<p>- GPUs supported as well: Nvidia CUDA, Apple Metal, OpenCL cards<p>- Split really big models between a number of GPU (warp LLaMA 70B with 2x RTX 3090)<p>- Not bad performance on shy CPU machines, fast as hell inference on monsters with beefy GPUs<p>- Both regular FP16/FP32 models and their quantised versions are supported - 4-bit really rocks!<p>- Popular LLM architectures already there: LLaMA, Starcoder, Baichuan, Mistral, etc...<p>- Special bonus: proprietary Janus Sampling for code generation and non English languages