That being said, since it's actually good I now expect it to be nerfed and/or abandoned in about 6 months
Exactly this, Google is where it is because generally its products start out pretty good
this post was submitted on 08 Dec 2024
49 points (94.5% liked)
Android
28186 readers
133 users here now
DROID DOES
Welcome to the droidymcdroidface-iest, Lemmyest (Lemmiest), test, bestest, phoniest, pluckiest, snarkiest, and spiciest Android community on Lemmy (Do not respond)! Here you can participate in amazing discussions and events relating to all things Android.
The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:
Rules
1. All posts must be relevant to Android devices/operating system.
2. Posts cannot be illegal or NSFW material.
3. No spam, self promotion, or upvote farming. Sources engaging in these behavior will be added to the Blacklist.
4. Non-whitelisted bots will be banned.
5. Engage respectfully: Harassment, flamebaiting, bad faith engagement, or agenda posting will result in your posts being removed. Excessive violations will result in temporary or permanent ban, depending on severity.
6. Memes are not allowed to be posts, but are allowed in the comments.
7. Posts from clickbait sources are heavily discouraged. Please de-clickbait titles if it needs to be submitted.
8. Submission statements of any length composed of your own thoughts inside the post text field are mandatory for any microblog posts, and are optional but recommended for article/image/video posts.
Community Resources:
We are Android girls*,
In our Lemmy.world.
The back is plastic,
It's fantastic.
*Well, not just girls: people of all gender identities are welcomed here.
Our Partner Communities:
founded 2 years ago
MODERATORS
Do you know if there's a FOSS project that does any of this? Android does provide a screen capture API, and there are some FOSS OCR and translation apps separately, bit I don't know of one that combines these things.
Not sure what CtS is using, but I don't think it's the normal screen capture API, because it works in apps that block screenshots
isn't it just a new way to access screen based search? i mean, i can do things you tak about by launching google assistant (non-gemini) and choosing "search on my screen", i use it daily to translate difficult japanese in my dulingo, select text and search on my non-pixel/samsung phone, it' there since few years AFAIK, i don't think it's going anywhere, because reverse image search is the future, also you're training their neural networks that way(👉ﾟヮﾟ)👉
for the most part it's just a finer, more intuitive interface to that exact function, imo. seems like mostly just marketting, for what's been there the whole time
Search on my screen stopped working reliably for me a while ago, and it didn't work in apps that are blocking screenshots/sharing
The 'translate what's on my screen' was a thing that google assistant could do around Android 8. I found it very handy when living abroad and not really knowing the language.
That one didn't use OCR, but could only do text in UI elements. Don't know why they killed it. Maybe some security reasons caused the API that reads another apps UI elements to be canned.
It's really helpful as someone trying to learn a language since I can just use it to select words I don't know and immediately translate them or look them up in a dictionary.
I think it has gotten better since it was first released for Samsung. The new translate button is very good.
What I like the most is the music recognition button, because its really quick and convenient, and it also works for content playing on your phone, as long as you use the phone's speakers. I think it doesnt use the media volume but rather the microphones only, but having the phone's volume enabled it probably feeds back into the microphones and it works this way.
Even if its nerfed and abandoned, a heartening new trend that I've seen is cutting edge technology being democratized in FOSS apps. There's all sorts of ai stuff and image stuff that was limited to the top five tech companies a year ago that's now available to everyone.