this post was submitted on 14 Jan 2024
264 points (95.5% liked)
Technology
60080 readers
3326 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I agree that it has limits but there are things we could do to make it reasonably good. ChatGPT knows how to execute actions (such as calling an API or doing a web search). It could probably be made to store and look up information in a vector database, essentially giving it a long-term memory.
Given some smaller breakthroughs in performance and model size we could conceivably retrain the network on new input continuously, in order to incorporate new knowledge.
That's the thing, I don't think a database can work as a long term memory here. How would it work? Let's say you tell your AI girlfriend that Interstellar movie was so bad it made you vomit. What would it store in the DB? When would it look that info up? It would be even worse with specific events. Should it remember the exact date of each event perfectly like DB does? It would be unnatural. To actually simulate memory it should alter the model somehow and the scale of the change should be proportional to the emotional charge of the message. I think this is on a completely different level than current models.
...i may be too green to see something here, but wouldn't simply saving month, year, topic, mood and quote enough? If AI needs everything formatted in certain input, run this through API. Teach AI to save only moments where user uses agitated language or smth and to periodically run checks if current convo allows for throwback, for example by topic, with advanced query when user asks if AI remembers something.
Then sell all this data for fat profit.
So imagine a convo:
1 years later:
Now the AI can find the meesage that said 'Interstellar' in the history but without any context. To know you were talking about the movie it would have to analyze the entire conversation again. And the emotional charge of the message can also change instantly:
What would the AI 'remember'? It would require some higher level of understanding of the conversation and the 'memories' would have to be updated all the time. It's just not possible to replicate with simple log.
Thanks for examples, now yeah, that's really ain't that simple...and hard af to foolproof. :/
It’s when people dive into this sort of memory stuff that I always remember: “oh yeah, this is why people call it a stochastic parrot.”
LLMs can do a lot. But without memory, they run into walls fast.