Does a VPS/cloud provider exist that charges by the minute or has flexible RAM/CPU needs? Currently, I have a $5/month VM on Linode for my app but now I want to add some ML features; however, running the ML API on the same VM as my app causes oom errors and kills the process.<p>Of course I could purchase a larger VM at a greater fixed cost per month but since this API will only be called a few hundred times per month there must be an option for per minute or per call pricing while the API mostly sits idly waiting for a request.<p>I am aware of serverless but the time to load the ML models for each call seems like it would take way too long to get a response, unless I have a misunderstanding about serverless then do please inform me.<p>And if it matters for any of the answers I'm using FastAPI and Celery for the web side and task queue then I have Yolov3 to detect objects of interest from an image then pass the object image to another model for OCR and make a prediction of the text it finds. I'm new to ML, so I've got a lot to learn and appreciate all the feedback.
Check out algorithmia - if that works for you.<p>Instead of looking for ML specific services, look at docker or container options. Its better if they have instance warm ups that way you can launch the vm when necessary.<p>Also see if you can split your solution into 2 parts. One that gathers intelligence in an offline batch mode. This data should help the second part to respond faster.
This would work with some Algos only. Not so much for Deep learning<p>Explore deep learning compression techniques to reduce size of the model.<p>There ain't many services for one off models. It doesn't make economical sense for cloud providers.