this post was submitted on 19 Nov 2023
497 points (86.9% liked)
Technology
60080 readers
3354 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
At this point I want a calendar of at what date people say "AI could never" - like "AI could never explain why a joke it's never seen before is funny" (such as March 2019) - and at what date it happens (in that case April 2022).
(That "explaining the joke" bit is actually what prompted Hinton to quit and switch to worrying about AGI sooner than expected.)
I'd be wary of betting against neural networks, especially if you only have a casual understanding of them.
I mean the limitations of LLMs are very well documented, they aren't going to advance a whole lot more without huge leaps in computing technology. There are limits on how much context they can store for example, so you aren't going to have AIs writing long epic stories without human intervention. And they're fundamentally incapable of originality.
General AI is another thing altogether that we're still very far away from.
Nearly everything you wrote is incorrect.
As an example, rolling context windows paired with RAG would easily allow for building an implementation of LLMs capable of writing long stories.
And I'm not sure where you got the idea that they were fundamentally incapable of originality. This part in particular tells me you really don't know how the tech is working.
A rolling context window isn't a real solution and will not produce works that even come close to matching the quality of human writers. That's like having a writer who can only remember the last 100 pages they wrote.
The tech is trained on human created data. Are you suggesting LLMs are capable of creativity and imagination? Lmao - and you try to act like I'm the one who's full of shit.
That's why you pair it with RAG.
They are trained by iterating through network configurations until there's diminishing returns on how accurately they can complete that human created data.
But they don't just memorize the data. They develop the capabilities to extend it.
So yes, they absolutely are capable of generating original content that's not in the training set. As has been demonstrated over and over. From explaining jokes not found in the training data, solving riddles not found in it, or combining different concepts to result in a new synthesis not found in the original data.
What do you think it's doing? Copy/pasting or something?