Hey Hacker News!<p>I've developed an open-source inference server called Inferenceable with the aim of providing developers with a straightforward solution that can be installed without fussing over dependencies.<p>Inferenceable, built in Node.js, simplifies the process of building and running complex libraries like llama.cpp, making it accessible to developers of all levels.<p>It's also pluggable and can be used with your custom strategies.<p>There are 3 main functions
- General text based inference
- Image inference
- Generate text embeddings<p>Inferenceable also includes a simple UI which can be customised or deployed as is, pluggable authentication, CSP and rate limiter.<p>I'm working on adding more authentication strategies, including social logins, and creating additional examples.<p>Thanks for checking it out, and happy coding!