this post was submitted on 03 Apr 2024
961 points (99.4% liked)
Technology
60052 readers
2966 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
They absolutely do contain a model of the universe which their answers must conform to. When an LLM hallucinates, it is creating a new answer which fits its internal model.
Statistical associations is not equivalent to a world model, especially because they're neither deterministic nor even tries to prevent giving up conflicting answers. It models only use of language
This phrase, so casually deployed, is doing some seriously heavy lifting. Lanuage is by no means a trivial thing for a computer to meaningfully interpret, and the fact that LLMs do it so well is way more impressive than a casual observer might think.
If you look at earlier procedural attempts to interpret language programmatically, you will see that time and again, the developers get stopped in their tracks because in order to understand a sentence, you need to understand the universe - or at the least a particular corner of it. For example, given the sentence "The stolen painting was found by a tree", you need to know what a tree is in order to interpret this correctly.
You can't really use language *unless* you have a model of the universe.
But it doesn't model the actual universe, it models rumor mills
Today's LLM is the versificator machine of 1984. It cares not for truth, it cares for distracting you
They are remarkably useful. Of course there are dangers relating to how they are used, but sticking your head in the sand and pretending they are useless accomplishes nothing.
They are more useful for quick templates than problem solving