A while ago I used whisper (or rather an OSS subtitle tool that used whisper, sadly can’t remember the name; it also converted burned in subs to proper ones via OCR) to generate subtitles for a season of a Show (4 season DVD set, 1 had proper subs, 2 burned in subs, 1 no subs -.-), too old and not popular enough to have "scene" subs, it worked impressively well. The most memorable thing for me was that a character’s scream was properly attributed to the correct character.<p>I’d love a feature like that for Jellyfin eventually.
There's an art to subtitling that goes beyond mere speech-to-text processing. Sometimes it's better to paraphrase dialog to reduce the amount of text that needs to be read. Sometimes you need to name a voice as unknown, to avoid spoilers. Sometimes the positioning on the screen matters. I hope the model can be made to understand all this.
I recently used some subtitles that I later found out had been AI generated.<p>The experience wasn't really good to be honest - the text was technically correct but the way multiline phrases were split made them somehow extremely hard to parse. The only reason I learned AI was involved is that it was bad enough for me to stop viewing and check what was wrong.<p>Hopefully it's an implementation detail and we'll see better options in the future, as finding subtitles for foreign shows is always a pain.
It's really sad that I'm reading "open source model" and think "hmhm, as if".<p>Maybe they're really using a truly open source model (probably not) but the meaning of the word is muddied already.
VLC is excellent on iOS -- highly recommended!<p>Go-to player with easy wifi loading and ability to connect to file shares to find files. Simple and actually easy to use (of course having a file server is another question)
AI subtitle generation seems like a useful feature. Hopefully they'll integrate with a subtitle sharing service so we don't have computers repeatedly duplicating work.<p>In the ideal case you'd probably generate a first pass of subtitles with AI, have a human review and tweak it as needed, and then share that. There's no reason for people to be repeatedly generating their own subtitles in most cases.
Isn't this what YouTube has been doing with automatically generated subtitles for videos for years?<p>I've always found their quality to be somewhat lacking.
I wonder if VLC tracks their downloads per day or region. Similar to Pornhub Trends, it'd be interesting to correlate a spike of downloads with events like Netflix hiking up their prices/enforcing anti-password-sharing policies, dropping some series from their catalogue, or some hyped-up movie being released...
Would be great to add a system of sharing the results instead of having to waste resources transcribing the same video over and over again with no chance of correction
Will the ML model be fully open?<p>(Like open source software, so that, in theory, someone could see see the source code, source data, and process for how it was trained, and reproduce that? And could they change how that model is built, not just tuning atop it, and distribute that under the same open source license?)
Hmm I wonder if this had anything to do with this? <a href="https://www.youtube.com/watch?v=0dUnY1641WM&themeRefresh=1" rel="nofollow">https://www.youtube.com/watch?v=0dUnY1641WM&themeRefresh=1</a>
As of yet big auto-subtitle providers like YouTube fails miserably at this, and as far as I've tested the Whispers they also do. Would be nice if embedding it wouldn't become the default.
Totally personally skewed perspective, but thinking about the number of systems I've installed VLC on, some multiple times through rebuilds etc, 6 billion feels like a huge underestimate!
Yet another classic garbage-in-garbage-out expected from AI gen subtitles. But it is an excellent undertaking nevertheless. Will help improve viewership in general