Increasingly bullish on AWS Bedrock.<p>• Devs forever want choice.<p>• Open-source LLMs are getting better<p>• Anthropic ships fantastic models<p>• Doesn't expose your app’s data to multiple companies<p>• Consolidated security, billing, config in AWS<p>• Power of AWS ecosystem
You can check out this technical deep dive on Serverless GPUs offerings/Pay-as-you-go way.
This includes benchmarks around cold-starts, performance consistency, scalability, and cost-effectiveness for models like Llama2 7Bn & Stable Diffusion across different providers -<a href="https://www.inferless.com/learn/the-state-of-serverless-gpus" rel="nofollow">https://www.inferless.com/learn/the-state-of-serverless-gpus</a>... .Can save months of your time. Do give it a read.<p>P.S: I am from Inferless
I have not read too deeply into this but, do any of these serverless environments offer GPUs? I'm sure there are ... reasons but the lack of GPU support in Lambda and Fargate remains a major paint point for AWS users.<p>It's been keeping me wrangling EC2 instances for ML teams but I do wonder how much longer that will last.