Is someone doing embeddings<>embeddings mapping?<p>For example, mapping embeddings of Llama to GPT-3?<p>That way you can see how similar the models “understand the world”.
Could anyone point me towards a relatively beginner-friendly guide to do something like<p>>download all my tweets (about 20k) and build a semantic searcher on top ?<p>How can utilize 3rd party embeddings with OpenAI's LLM API? Am I correct to understand from this article that this is possible?
I've done some quick-and-dirty testing with OpenAI's embedding API + Zilliz Cloud. The 1st gen embeddings leave something to be desired (<a href="https://medium.com/@nils_reimers/openai-gpt-3-text-embeddings-really-a-new-state-of-the-art-in-dense-text-embeddings-6571fe3ec9d9" rel="nofollow">https://medium.com/@nils_reimers/openai-gpt-3-text-embedding...</a>), but the 2nd gen embeddings are actually fairly performant relative to many open source models with MLM loss.<p>I'll have to dig out the notebook that I created for this, but I'll try to post it here once I find it.
Very interested in this - I've been using embeddings / semantic search doing information retrieval from PDFs, using ada-002, and have been impressed by the results in testing.<p>The reasons the article listed, namely a) lock-in and b) cost, have given me pause with embedding our whole corpus of data. I'd much rather use an open model but don't have much experience in evaluating these embedding models and search performance - still very new to me.<p>Like what you did with ada-002 vs Instruct XL, has there been any papers or prior work done evaluating the different embedding models?
It’s fine to use their embeddings for a proof of concept, but since you don’t own it, you probably shouldn’t rely on it because it could go away at any time.