An LLM playground you can run on your laptop

As an alternative for purely local LLMs, I&#x27;ve been having fun with this setup: <a href="https:&#x2F;&#x2F;github.com&#x2F;oobabooga&#x2F;text-generation-webui">https:&#x2F;&#x2F;github.com&#x2F;oobabooga&#x2F;text-generation-webui</a>

This is very neat, thanks for sharing. I was wondering about a related thing — is there a way to query a llama.cpp (or other such local model) via an API from Python? In other words, I see a lot of cool applications being built with langchain + ClosedAPI, so I’m wondering if an API call to a local model could be a drop-in replacement for the ClosedAPI call?

An LLM playground <i>whose UI</i> you can run on your laptop.

Dang, I don&#x27;t have a laptop, can I also run it on my desktop...?<p>;-)

Awesome !! Great … I haven’t gone through it yet but from just what I saw on the GitHub page it would be awesome to have a CLI with standard arguments built into it for everything that can be done through the flask interface. Thanks !!

Why does it use port 5432 by default? That&#x27;s the default port for PostgreSQL. Does this use PostgreSQL?

Awesome! Does it support safetensors, new ggml format, triton or cuda model files?<p>I added the playground to the GUI list here:<p>to <a href="https:&#x2F;&#x2F;github.com&#x2F;underlines&#x2F;awesome-marketing-datascience&#x2F;blob&#x2F;master&#x2F;awesome-ai.md">https:&#x2F;&#x2F;github.com&#x2F;underlines&#x2F;awesome-marketing-datascience&#x2F;...</a>

Thanks for this! The compare feature is very cool, especially being able to play around with GPT4 settings (I&#x27;m still on the waiting list for GPT4 API so having access to this now is fantastic).

Someone with more inclination than I at the moment might be able to say something interesting about this being from Nat Friedman (former CEO of GitHub).