We have tried Microsoft Speech Service and found it to be way too complicated.<p>The Azure OpenAI whisper deployment has a pretty low quota.<p>Running it using ggeranov's whisper on a Mac works fairly well but it's not in our corporate network.<p>I really need to batch transcribe these calls. I am a few weeks behind.<p>I have access to a server with 2x RTX 4090. It is all up and ready to go with the Nvidia drivers.<p>By the way, these calls are an average of 90s. Not long.
I transcribed between 3000 to 4000 of 10s-30s short videos, every day for almost 2 years for fun. A cheap desktop linux with second hand x-mining RTX 3060 and 3080Ti, connected over home network using basic Gradio and faster-whisper, so they can be exposed as public API and called from corporate network. Relatively easy and much cheaper compared to commercial offerings at the time. These GPUs are over powered for the task and every day only spent 1 to 2 hours of actual encoding, it's so quick, and it's using the biggest whisper model with audio preprocessing and VAD to improve success rate.
Does it have to use Whisper? If so, can't you just run it on that server instead of the Mac? <a href="https://github.com/openai/whisper/discussions/1463">https://github.com/openai/whisper/discussions/1463</a><p>If it doesn't, there are a bunch of other speech recognition APIs. Most of them use older techs but might be good enough: <a href="https://www.gladia.io/blog/openai-whisper-vs-google-speech-to-text-vs-amazon-transcribe" rel="nofollow">https://www.gladia.io/blog/openai-whisper-vs-google-speech-t...</a><p>Personally I found Otter.ai works really well for the transcription part, but they don't have an API: <a href="https://otter.ai" rel="nofollow">https://otter.ai</a><p>You can also just upload them all to YouTube in a private playlist and it'll automatically transcribe them for you.
This is a complete shameless plug but I just published some documentation on automatically building Whisper inference engines with TensorRT-LLM which has the batch inference that you're looking for: <a href="https://docs.baseten.co/performance/examples/whisper-trt" rel="nofollow">https://docs.baseten.co/performance/examples/whisper-trt</a>
We use Whisper Large on NLP Cloud (<a href="https://nlpcloud.com/home/playground/asr" rel="nofollow">https://nlpcloud.com/home/playground/asr</a>). It works very well and it's simple to set up in my opinion. If you have a batch to process you could simply subscribe to their pay-as-you-go plan for a couple of weeks/months maybe?