Natanael

Not an LLM specifically, in particular lack of backtracking and the network depth limits as well as interconnectivity limits sets hard limits on capabilities.

https://www.lesswrong.com/posts/XNBZPbxyYhmoqD87F/llms-and-computation-complexity

https://garymarcus.substack.com/p/math-is-hard-if-you-are-an-llm-and

https://arxiv.org/abs/2401.11817

https://www.marktechpost.com/2023/08/01/this-ai-research-dives-into-the-limitations-and-capabilities-of-transformer-large-language-models-llms-empirically-and-theoretically-on-compositional-tasks/?amp

Humans have a completely different memory model and a in large part a very different way of linking together learned concepts to form their world view and to develop interdisciplinary skills, allowing us to solve many kinds of highly complex tasks as long as we can keep enough of it in our memory.

It's training and fine tuning has a lot of specific instructions given to it about what it can and can't do, and if something sounds like something it shouldn't try then it will refuse. Spitting out unbiased random numbers is something it's specifically trained not to do by virtue of being a neural network architecture. Not sure if OpenAI specifically has included an instruction about it being bad at randomness though.

While the model is fed randomness when you prompt it, it doesn't have raw access to those random numbers and can't feed it forward. Instead it's likely to interpret it to give you numbers it sees less often.

The TLDR is that pathways between nodes corresponding to frequently seen patterns (stereotypical sentences) gets strengthened more than others and therefore it becomes more likely that this pathway gets activated over others when giving the model a prompt. These strengths correspond to probabilities.

Have you seen how often they'll sign a requested text with a name placeholder? Have you seen the typical grammar they use? The way they write is a hybridization of the most common types of texts it has seen in samples, weighted by occurrence (which is a statistical property).

It's like how mixing dog breeds often results in something which doesn't look exactly like either breed but which has features from every breed. GPT/LLM models mix in stuff like academic writing, redditisms and stackoverflowisms, quoraisms, linkedin-postings, etc. You get this specific dryish text full of hedging language and mixed types of formalisms, a certain answer structure, etc.

Might be silent removal of posts instead of ban

ChatGPTankie

You can ask your own instance admin if they're willing to defederate

If you try to post again and get a certain error you're banned from the community, then you can check that community's server's mod log (if it's public)

I had a post made up of almost exclusively links to news sources (to back up a previous claim) removed due to being "uncivil"

If there's no averaging, why do they repeat stereotypes so often?

The crawling isn't illegal, what you do with the data might be

It sounds like there's a specific set of CPU instructions (or a specific sequence of them) which are especially affected, which that game engine uses much more than most other software

It started with U2F which may be older?