Technology

59207 readers

2513 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

Has Generative AI Already Peaked? - Computerphile (www.youtube.com)

submitted 6 months ago by [email protected] to c/[email protected]

29 comments fedilink hide all child comments

A new paper suggests diminishing returns from larger and larger generative AI models. Dr Mike Pound discusses.

The Paper (No "Zero-Shot" Without Exponential Data): https://arxiv.org/abs/2404.04125

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] -1 points 6 months ago (1 children)

“we don’t know how” != “it’s not possible”

i think OpenAI more than anyone knows the challenges with scaling data and training. anyone working on AI knows the line: “a baby can learn to recognize elephants from a single instance”. reducing training data and time is fundamental to advancement. don’t get me wrong, it’s great to put numbers to these things. i just don’t think this paper is super groundbreaking or profound. a bit clickbaity and sensational for Computerphile

[–] [email protected] 7 points 6 months ago

...and a baby doesn't use the same architecture, not even close, as generative AIs. Babies are T3 systems, they aren't simply systems which have rules on how to learn, they are systems which have rules on how to develop learning strategies that they then use to learn.

I'm not doubting, in the slightest, that AI can't get there: It's definitely possible. It's just not possible with the current approaches, and the iterative refinements that "oh OpenAI is constantly coming up with new topologies" refers to is just more of the same. Show me a topology that can come up with topologies, then we'll have a chance to break through the need for exponential amounts of data.