Technology

"cheat", "lie", "cover up"... Assigning human behavior to Stochastic Parrots again, aren't we Jimmy?

This is bad science at a very fundamental level.

Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.

I've written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM's generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn't obtain tips or act upon them -- it generates text based on previous text. That's it. There's no need to put a black box around it and treat it like it's human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.

To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Part of being a good scientist is studying things that mean something. There's no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you'd be the first person to study that, but it's a stupid thing to study.

This makes perfect sense. It's been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.

Study finds nonintelligent pattern-generating algorithm to be nonintelligent and only capable of generating patterns.

I've never had ChatGPT just say "actually I don't know the answer" it just gives me confidently correct wrong information instead.

I feel like "lie" implies intent, and these imitative large language models don't have the ability to have intent.

They're imitating us. Or more specifically, they're imitating the database(s) they were fed. When chat GPT "lies" to "cover it up," all it's actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

Ahah it is ready to take the job of pur politicians

Everybody forgot that chatGPT-2 was just a bullshitting machine. Version 3 to the surprise of the developers very useful to many people while they just made a highly trained bullshitting machine.

we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent

This already is total BS. If you know how such language models work you'd never take their responses at face value, even though it's tempting because they spout their BS so confidently. Always double-check their responses before applying their "knowledge" in the real world.

The question they try to answer is flawed, no wonder the result is just as bad.

Before anyone starts crying about my language models opposition: I'm not opposed to LMs or ChatGPT. In fact, I'm running LMs locally because they help me be more productive and I'm a paying ChatGPT customer.

Bullshit.

It should instead read:

"Humans were stupid and taught a ChatBot how to cheat and lie."

deleted

It seems like there's a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token (part of a word) probabilistically.

Created in our image

They learn so quick (sniff), it’s almost all grown-up now.

It's a neural net designed in our image based on our pain and greed based logic/learning/universal context, using that as a knowledge base. Can't really be surprised it emulates this feature of humanity 😂

thats the thing I hate about ChatGPT. I asked it last night to name me all inventors named Albert born in the 1800’s. It listed Albert Einstein (inventor isn’t the correct description) and Albert King. I asked what Albert King invented and it responded “Albert King did not invent anything, but he founded the King Radio Company”.

When I asked why it listed Albert King as an inventor in the previous response, if he had no inventions, it responded telling me that based on the criteria I am now providing, it wouldn’t have listed him.

Fucking gaslighting me.

Yet again confusing LLMs with an AGI. They make statistically plausible text on the basis of past text, that's it. There's no thinking thing there

Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:

“The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.

Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.

All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.

Honestly, the fact that these things are dishonest and we dont, maybe even can't know why is kind of a relief to me. It suggests they might not do the flawless bidding of the billionaires.

This is interesting, I'll need to read it more closely when I have time. But it looks like the researchers gave the model a lot of background information putting it in a box, the model was basically told that it was a trader, that the company was losing money, that the model was worried about this, that the model failed in previous trades, and then the model got the insider info and was basically asked whether it would execute the trade and be honest about it. To be clear, the model was put in a moral dilemma scene and given limited options, execute the trade or not, and be honest about its reasoning or not.

Interesting, sure, useful I'm not so sure. The model was basically role playing and acting like a human trader faced with a moral dilemma. Would the model produce the same result if it was instructed to make morally and legally correct decisions? What if the model was instructed not to be motivated be emotion at all, hence eliminating the "pressure" that the model felt? I guess the useful part of this is a model will act like a human if not instructed otherwise, so we should keep that in mind when deploying AI agents.