I've been hard at work on Kiln (3k Github stars, ~10k users):<p>- Gihub: <a href="https://github.com/Kiln-AI/Kiln" rel="nofollow">https://github.com/Kiln-AI/Kiln</a><p>- Demo Video: <a href="https://getkiln.ai#demo" rel="nofollow">https://getkiln.ai#demo</a><p>It's goal is to makes AI model+product development processes better in a bunch of ways:<p>- Truly intuitive desktop apps, on top of our open-source python library<p>- Rapid prototyping: Quickly try a range of state of the art models+techniques (no code needed)<p>- Task specific evals: How does the system perform on your goals (not just global benchmarks)<p>- Iterate with confidence: update to better+faster models/techniques, knowing it works on all past evals/tests<p>- Team collaboration: get the entire team (PM/QA/Eng/Design/DS) engaged early and often. Shared datasets+evals you can sync with Git.<p>Features include:<p>- Fine tune dozens of models<p>- Interactive synthetic data generation to build train/eval datasets<p>- Powerful custom evals: LLM as Judge and G-Eval<p>- Collaboration: share with Git or shared drives<p>- Runs any LLM: locally or remote<p>- Private: it's local software<p>- Open Source Python library<p>- Completely free<p>- Much more: dataset management, ratings, prompt generators, JSON support, LLM repairs, reasoning models...<p>Happy to answer any questions!