This might be one of the few times I’ve seen AI being useful and not just slapped on something for marketing purposes.
this post was submitted on 12 Jan 2025
660 points (98.0% liked)
Technology
60473 readers
5239 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
And not to do evil shit
But the toppings contains potassium benzoate.
As long as the models are OpenSource I have no complains
And the data stays local.
Finally, some good fucking AI
I was just thinking, this is exactly what AI should be used for. Pattern recognition, full stop.
Yup, and if it isn't perfect that is ok as long as it is close enough.
Like getting name spellings wrong or mixing homophones is fine because it isn't trying to be factually accurate.
Problem ist that now people will say that they don't get to create accurate subtitles because VLC is doing the job for them.
Accessibility might suffer from that, because all subtitles are now just "good enough"
Regular old live broadcast closed captioning is pretty much 'good enough' and that is the standard I'm comparing to.
Actual subtitles created ahead of time should be perfect because they have the time to double check.
Or they can get OK ones with this tool, and fix the errors. Might save a lot of time
load more comments
(16 replies)
load more comments
(2 replies)
load more comments
(2 replies)
This sounds like a great thing for deaf people and just in general, but I don't think AI will ever replace anime fansub makers who have no problem throwing a wall of text on screen for a split second just to explain an obscure untranslatable pun.
Bless those subbers. I love those walls of text.
Translator's note: keikaku means plan
They are like the * in any Terry Pratchett (GNU) novel, sometimes a funny joke can have a little more spice added to make it even funnier
It's unlikely to even replace good subtitles, fan or not. It's just a nice thing to have for a lot of content though.
I have family members who can't really understand spoken English because it's a bit fast, and can't read English subtitles again, because again, too fast for them.
Sometimes you download a movie and all the Estonian subtitles are for an older release and they desynchronize. Sometimes you can barely even find synchronized English subtitles, so even that doesn't work.
This seems like a godsend, honestly.
Funnily enough, of all the streaming services, I'm again going to have to commend Apple TV+ here. Their shit has Estonian subtitles. Netflix, Prime, etc, do not. Meaning if I'm watching with a family member who doesn't understand English well, I'll watch Apple TV+ with a subscription, and everything else is going to be pirated for subtitles. So I don't bother subscribing anymore. We're a tiny country, but for some reason Apple of all companies has chosen to acknowledge us. Meanwhile, I was setting up an Xbox for someone a few years ago, and Estonia just... straight up doesn't exist. I'm not talking about language support - you literally couldn't pick it as your LOCATION.
For all their faults, Apple knows accessibility. Good job Timmy.
load more comments
(1 replies)
What’s important is that this is running on your machine locally, offline, without any cloud services. It runs directly inside the executable
YES, thank you JB
load more comments
(7 replies)
The nice thing is, now at least this can be used with live tv from other countries and languages.
Think you want to watch Japanese tv or Korean channels with out bothering about downloading, searching and syncing subtitles
I prefer watching Mexican football announcers, and it would be nice to know what they're saying. Though that might actually detract from the experience.
Amazing. I can finally find out exactly what that nurse is yelling about while she gets railed by the local basketball team.
load more comments
(1 replies)
Will it be possible to export these AI subs?
Imagine the possibilities!
Now I want some AR glasses that display subtitles above someone's head when they talk à la Cyberpunk that also auto-translates. Of course, it has to be done entirely locally.
I guess we have most of the ingredients to make this happen. Software-wise we're there, hardware wise I'm still waiting for AR glasses I can replace my normal glasses with (that I wear 24/7 except for sleep). I'd accept having to carry a spare in a charging case so I swap them out once a day or something but other than that I want them to be close enough in terms of weight and comfort to my regular glasses and just give me AR like overlaid GPS, notifications, etc, and indeed instant translation with subtitles would be a function that I could see having a massive impact on civilization tbh.
load more comments
(5 replies)
As vlc is open source, can we expect this technology to also be available for, say, jellyfin, so that I can for once and for all have subtitles.done right?
Edit: I think it's great that vlc has this, but this sounds like something many other apps could benefit from
It's already available for anyone to use. https://github.com/openai/whisper
They're using OpenAI's Whisper model for this: https://code.videolan.org/videolan/vlc/-/merge_requests/5155
Note that openai's original whisper models are pretty slow; in my experience the distil-whisper project (via a tool like whisperx) is more than 10x faster.
load more comments
(3 replies)
crunchyroll is currently using AI subtitles. it's obvious because when someone says "mothra. Funky..." it captions "mother fucker"
load more comments
(3 replies)
load more comments
(4 replies)
The technology is nowhere near being good though. On synthetic tests, on the data it was trained and tweeked on, maybe, I don't know.
I corun an event when we invite speakers from all over the world, and we tried every way to generate subtitles, all of them run on the level of YouTube autogenerated ones. It's better than nothing, but you can't rely on it really.
Really? This is the opposite of my experience with (distil-)whisper - I use it to generate subtitles for stuff like podcasts and was stunned at first by how high-quality the results are. I typically use distil-whisper/distil-large-v3, locally. Was it among the models you tried?
I unfortunately don't know the specific names of the models, I will comment additionally if I will not forget to ask people who spun up the models themselves.
The difference might be that live vs recorded stuff, I don't know.
is your goal to rely on it, or to have it as a backup?
For my purpose of having backup nearly anything will be better than nothing.
When you do live streaming there is no time for backup, it either works or not. Better than nothing, that's for sure, but also maybe marginally better than whatever we had 10 years ago
load more comments
(4 replies)
And yet they turned down having thumbnails for seeking because it would be too resource intensive. 😐
I mean, it would. For example Jellyfin implements it, but it does so by extracting the pictures ahead of time and saving them. It takes days to do this for my library.
load more comments
(2 replies)
Video decoding is resource intensive. We're used to it, we have hardware acceleration for some of it, but spewing something around 52 million pixels every second from a highly compressed data source is not cheap. I'm not sure how both compare, but small LLM models are not that costly to run if you don't factor their creation in.
load more comments
(1 replies)
load more comments
(1 replies)
I hope Mozilla can benefit of a good local translation engine that could come out of it as well.
load more comments
(4 replies)
view more: next ›