Hi,<p>Harbor is a toolkit to run LLMs and related services locally. It's not designed as a deployment solution, rather as a convenient local environment.<p>I was rocking a very similar setup (less components and features though) and then decided to polish it a little bit and make it open source.<p>Few interesting details about the project:<p>- Dynamic sub-selection of compose files to run together depending on which services are currently running. For example, Harbor knows that you're running SearXNG and Open WebUI together and injects a config for Web RAG based on SearXNG into the WebUI, same for TTS and LLM backends<p>- CLI supports arguments scrambling via a retry, e.g. `harbor set config` and `harbor config set` run the same command, it's supported for nearly all first pairs of arguments to the CLI<p>- Being an LLM project - after the setup you can ask CLI how to do something: `harbor how to configure vllm model?` (yes, no quotes), 50% gimmick, but very fun<p>Cheers,
Ivan<p>P.S. Forgive me for all the shell sins in the source.