Using GPT3, Supabase and Pinecone to automate a personalized marketing campaign

&gt; My script read through each of the products we had responses for, called OpenAI&#x27;s embedding api and loaded it into Pinecone - with a reference to the Supabase response entry.<p>OpenAI and the Pinecone database are not really needed for this task. A simple SBERT encoding of the product texts, followed by storing the vectors in a dense numpy array or faiss index would be more than sufficient. Especially if one is operating in batch mode, the locality and simplicity can’t be beat and you can easily scale to 100k-1M texts in your corpus on commodity hardware&#x2F;VPS (though NVME disk will see a nice performance gain over regular SSD)

&gt; And it pretty much worked! Using prompts to find matches is not really ideal, but we want to use GPT&#x27;s semantic understanding. That&#x27;s where Embeddings come in.<p>sounds like you ended up not using GPT3 in the end which is probably wise.<p>i&#x27;m curious if you might see further savings using other cheaper embeddings that are available on huggingface. but its probably not material at this point.<p>did you also consider using pgvector instead of pinecone? <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34684593" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34684593</a> any painpoints with pinecone you can recall?

I don&#x27;t like the unscientific ad for his gf company.<p>&#x27;which helped launch the movement of those opposed to endocrine disruptors, was retracted and its author found to have committed scientific misconduct&#x27;

I have see a lot of people write about how important the interaction between vector DBs and chat-GPT3 (and GPT3) is. I am still not much wiser after this article. Is it that it makes it easier to go from:<p>user query -&gt; GPT3 response -&gt; Lookup in VectorDB -&gt; send response based on closest embedding in VectorDB<p>?

Are you saving the match pairs somewhere? I imagine 1) there are a finite number of them, 2) doing an exact lookup in a DB first will be faster and easier than calling GPT3 and Pinecone every time, and 3) eventually GPT3 APIs will get pricey enough to make you think twice unless you&#x27;re running your own instance on a cluster.

This looks incredible and magical to me. How do you learn to create things like this as a mostly web programmer? Vectorization, etc I had no idea could integrate with gpt etc but honestly it looks kind of obvious&#x2F;effortless to the author.

Pardon my ignorance here. I started to play around with text generation today and came around plenty of resource but hard to make any sense of it. I had this working <a href="https:&#x2F;&#x2F;github.com&#x2F;oobabooga&#x2F;text-generation-webui">https:&#x2F;&#x2F;github.com&#x2F;oobabooga&#x2F;text-generation-webui</a> and instead of it being able to answer question, it revolves around the concept of generating text.<p>In your case and ChatGPT3, does is it provide output based on the data you feed it? If that is the case, is there anything related to training the model to use your data?<p>I am trying to gauge a sense of what is going on.

How do you know if the output that was sent to customers (who believe they are getting accurate results from a knowledgable human being, BTW) is correct?

How long did this take?<p>Did you consider something like openrefine or fuzzy matching &#x2F; levenshtein distance?<p>Seems like a common data cleaning ask with a small amount of data.

i fail to see how this data cleaning could not be solved with proper tokenization and some distance measure. the amount of power used for those api calls is slighty obscene.<p>edit: don&#x27;t want to rant. it&#x27;s not a bad post and i&#x27;m sure there is many and far more wasteful examples than this.

I&#x27;m disappointed that the article doesn’t explain what they ended up doing.

Why not just use GPT-3 or even GPT-2 classifier API? No generative AI needed

&gt; 100s of human hours saved<p>wait till its thousands, millions, billions . . .

What is pinecone and is there a link to a website?

Awesome to see the integration between Klaviyo automation and GPT-3 AI and using it to streamline your girlfriends processes. Keep up the fantastic work!

This is pure spam.

Loved reading it. Will feature this in my newsletter on AI Tools and learning resources, AI Brews <a href="https:&#x2F;&#x2F;aibrews.com" rel="nofollow">https:&#x2F;&#x2F;aibrews.com</a>