Hi HN, I’m Varun, cofounder of Exafunction. We’re excited about all the use cases of large NLP models like text generation and language comprehension, but found the existing offerings to be quite expensive. Our goal is to serve these models in a cost effective way at scale.<p>The first model we’re starting with is GPT-J, the recently released open source large language model by EleutherAI. It’s comparable to OpenAI’s GPT-3 Curie in performance. With lots of optimizations, we are able to serve the model for 6x cheaper per token compared to OpenAI.<p>We have a simple HTTPS API that you can try out for free. Here’s a link with the details - <a href="https://www.exafunction.com/nlp-api" rel="nofollow">https://www.exafunction.com/nlp-api</a><p>We’re also looking into supporting custom and fine-tuned models since it’s required to get good performance for a lot of applications. Especially for smaller models like BERT, we’re excited about the cost savings we can deliver for users who use many different models. In this case, we’re 100 - 1000x cheaper than others offering similar inference APIs like Hugging Face.<p>We hope this is interesting to you all and please email me at varun@exafunction.com if there’s a model you’d like to see supported.
Have you considered adding support for CodeT5? I believe they're supported in Huggingface as well. <a href="https://github.com/salesforce/CodeT5" rel="nofollow">https://github.com/salesforce/CodeT5</a>
Wow! How are you able to achieve the cost reductions? Is it different hardware, software optimizations, or both? Also does this suggest that OpenAI is charging 6x markups on their models... >:(
Awesome. I am testing HuggingFace's API and evaluating if I should pay for a dev account. I will evaluate Exafunction also, it looks interesting. Specifically, I'm interested in using the multilingual LaBSE model. Do you support this model?
This is really remarkable! How hard do you think it will be to support new models, i.e. does the tooling you’ve built generalize to you being able to serve other large scale models easily?