knightly

joined 1 year ago
[–] [email protected] 13 points 11 months ago (1 children)

See also, Left 4 Dead's "Director" system. https://left4dead.fandom.com/wiki/The_Director Pretty sute that qualifies as prior art.

[–] [email protected] 1 points 11 months ago (1 children)

It can answer questions as well as any person.

The 7th grader and plagiarism comment make me think you haven't played with them much or really tested them.

Of course I have, my employer has me shoehorning ChatGPT into everything, and I agree with what the research says: Children can answer questions better than LLMs can.

https://techxplore.com/news/2023-12-artificial-intelligence-excel-imitation.html

Stochastic plagirism is still plagirism.

[–] [email protected] 1 points 11 months ago (3 children)

But they don't "answer questions", they just respond to prompts. You can't use them to learn anything without checking their responses against authoritative sources you should have used in the first place.

There's no intelligence there, just a plagirism laundromat and some rules for formatting text like a 7th grader.

[–] [email protected] 1 points 11 months ago (5 children)

Auto complete is not a lossy encoding of a database either, it's a product of a dataset, just like you are a product of your experiences, but it is not wholly representative of that dataset.

If LLMs don't encode their training data, then why are they proving susceptible to data exfiltration techniques where they output the content of their training dataset verbatim? https://m.youtube.com/watch?v=L_1plTXF-FE

[–] [email protected] 1 points 11 months ago (7 children)

If LLMs were just lossy encodings of their database they wouldn't be able to answer any questions outside of there training set.

Of course they could, in the same way that hitting the autocomplete key can finish a half-completed sentence you've never written before.

The fact that models can produce useful outputs from novel inputs is the whole reason why we build models. Your argument is functionally equivalent to the claim that wind tunnels are intelligent because they can characterise the aerodynamics of both old and new kinds of planes.

How do you explain the hallucinations if the llm is just a complex lookup engine? You can't lookup something you've never seen.

For the same reason that a random number generator is capable of producing never-before-seen strings of digits. LLM inference engines have a property called "temperature" that governs how much randomness is injected into their responses:

[–] [email protected] 1 points 11 months ago

But it is, necessarily.

For example, when we make shit up, we're aware that the shit we made up isn't real. LLMs are structurally incapable of recognizing the distinction between facts they regurgitate and the ones they manufacture from whole cloth.

You didn't have to consume terabytes of text to build a model for how to form sentences like a human, you did that with a few megabytes of overheard conversation before you were even conscious enough to be aware of it.

There's no model of intelligence so over-simplified to the point of giving LLMs partial credit that wouldn't also give equivalent credence to the "intelligence" of search engines.

[–] [email protected] 1 points 11 months ago* (last edited 11 months ago) (9 children)

This is not how LLMs work, they are not a database nor do they have access to one.

Please do explain how you think they make LLMs without a database of training examples to build a statistical model from.

The llm itself is just what it learned from reading all the training data,

I.e. "a model that encodes a database".

They are a trained neural net with a set of weights on matrices that we don't fully understand.

I.e., "we applied a very lossy compression algorithm to this database".

We do know that it can't possibly have all the information from its training set since the training sets (measured in tb or pb) are orders of magnitude bigger than the models (measured in gb).

Check out the demoscene sometime, you'll be surprised how much complexity can be generated from a very small set of instructions. I've seen entire first person shooter video games less than 100kb in size that algorithmically generate hundreds of megabytes of texture data at runtime. The idea that a mere 1,000x non-lossless compression of text would be impossible is laughable, especially when lossless text compression using neural network techniques achieved a 250x compression ratio years ago.

[–] [email protected] 1 points 11 months ago

Well unlike you suggest, I don't vote blue no matter who, I vote blue because the other choice is demonstrably worse.

I'm sure the party cares deeply about this justification, since that seems to be all they're running on these days.

I remember back in 1984 when such a strategy was considered comical:

    It comes from a very ancient democracy, you see..."
    "You mean, it comes from a world of lizards?"
    "No," said Ford, who by this time was a little more rational and coherent than he had been, having finally had the coffee forced down him, "nothing so simple. Nothing anything like so straightforward. On its world, the people are people. The leaders are lizards. The people hate the lizards and the lizards rule the people."
    "Odd," said Arthur, "I thought you said it was a democracy."
    "I did," said Ford. "It is."
    "So," said Arthur, hoping he wasn't sounding ridiculously obtuse, "why don't people get rid of the lizards?"
    "It honestly doesn't occur to them," said Ford. "They've all got the vote, so they all pretty much assume that the government they've voted in more or less approximates to the government they want."
    "You mean they actually vote for the lizards?"
    "Oh yes," said Ford with a shrug, "of course."
    "But," said Arthur, going for the big one again, "why?"
    "Because if they didn't vote for a lizard," said Ford, "the wrong lizard might get in. Got any gin?"

-Douglas Adams, "So Long, and Thanks for All the Fish" (Hitchhiker's Guide to the Galaxy, #4) 

You wax eloquent about how things need to change and so you choose to do nothing

Says the guy who thinks that repeatedly voting in a game that was rigged from the start isn't "doing the same thing over and over while expecting a different result".

what happens when you recruit enough lefties to your side that the right is able to establish the equivalent of a dictatorship via voter suppression, coups, etc?

Non-fascists will be forced to accept that working within a system that's designed to disenfranchise them can never produce a democratic government and join with the anti-fascists. The dictatorship is subsequently overthrown by popular action at the cost of many lives (which likely includes my own as a nonbinary person with left-wing politics). This creates space for a second constitutional convention to fundamentally reform the law of the land.

At that point you are 100% fucked.

We've been 100% fucked for longer than you've been alive, the only question is whether or not you're comfortable with the current rate of fuckery and what you're willing to risk to change it.

Are you one of these idiots who actively want a civil war?

Lol, of course not. I have merely accepted that nothing short of a second American Revolution could overthrow the two-party system and that things will definitely get worse before they can get better.

Hell, I predicted this way back in November 2016. Trump's win would galvanize the Democrats in 2020, and the subsequent lack of meaningful change depresses the vote enough that 2022 and 2024 go to the republicans. Trump then punishes his percieved enemies while the Democrats continue to chase Republican voters to the right, further fracturing their own left-wing base.

[–] [email protected] 1 points 11 months ago (2 children)

Let me know when the 2 party stranglehold allows you to vote it out.

You're the one insisting I don't have a choice here. Why should anyone bother helping reinforce a system designed to disenfranchise them? To keep the "wrong people" from taking power, as if that wasn't already the case?

If that sounds like Accelerationism to you, then you're reading me correctly. Preservation of the intolerable status quo only delays and inflames the inevitable upheaval.

Meanwhile, completely oblivious that you and people like you are exactly the reason why we don't have enough voting power to enact real change.

Oblivious?

No, I'm counting on it. The Democrats won't change unless they have to, and people like me are the ones they need to change for. People like you, who "vote blue no matter who", can be safely ignored by party leadership since they know you'll vote for them anyway.

[–] [email protected] 2 points 11 months ago (2 children)

It could be described that way, but it wouldn't be a very apt metaphor. We aren't simple, stateful input-to-output algorithms, but a confluence of innate tendencies, learned experiences, acquired habits, unconscious motivations, and capable of modifying our own thought processes and behavior on the fly to suit whatever best fits the local context. Our brains encode a model of the world we live in that includes models of ourselves and the other people we interact with, all built in realtime from our observations without conscious effort.

[–] [email protected] 3 points 11 months ago (11 children)

Yes doing an SAT test with the answer key isn't intelligent because that's in your "database" and is just a matter of copying over the answers. LLMs don't do this though, it doesn't do a lookup of past SAT questions it's seen and answer it, it uses some process of "reasoning" to do it.

You've now reduced the "process of reasoning" to hitting the autocomplete button until your keyboard spits out an answer from a database of prior conversations. It might be cleverly designed, but generative models are no more intelligent than an answer key or a library's card catalog. Any "intelligence" they appear to encode actually comes from the people who did the work to assemble the training database.

[–] [email protected] 1 points 11 months ago (4 children)

Neither party represents my interests and, as you've seen, the "lesser evil" argument doesn't motivate me in the slightest. If you want me to vote for Dems then you'll have to make them worth voting for and stop acting like they can coast to victory on fear of the Republicans alone.

I'm aware of FPTP and its mathematical implications, our country is designed to provide only the appearance of democracy while actually limiting choice to those candidates the parties deem inoffensive to their campaign financiers. And we're stuck with them unless you can convince the two-party state to abolish the leverage it has over third parties, so stop acting like they'll let you vote your way out of this situafion.

Real change would require a political revolution, and I've long since lost my faith that Americans would build something less fascist given the chance.

view more: ‹ prev next ›