mm_maybe

this is learning completely the wrong lesson. it has been well-known for a long time and very well demonstrated that smaller models trained on better-curated data can outperform larger ones trained using brute force "scaling". this idea that "bigger is better" needs to die, quickly, or else we're headed towards not only an AI winter but an even worse climate catastrophe as the energy requirements of AI inference on huge models obliterate progress on decarbonization overall.

those are all classification problems, which is a fundamentally different kind of problem with less open-ended solutions, so it's not surprising that they are easier to train and deploy.

I really wish it were easier to fine-tune and run inference on GPT-J-6B as well... that was a gem of a base model for research purposes, and for a hot minute circa Dolly there were finally some signs it would become more feasible to run locally. But all the effort going into llama.cpp and GGUF kinda left GPT-J behind. GPT4All used to support it, I think, but last I checked the documentation had huge holes as to how exactly that's done.

One of the reasons I love StarCoder, even for non-coding tasks. Trained only on Github means no "instruction finetuning" bullshit ChatGPT-speak.

Well, maybe we need a movement to make physical copies of these games and the consoles needed to play them available in actual public libraries, then? That doesn't seem to be affected by this ruling and there's lots of precedent for it in current practice, which includes lending of things like musical instruments and DVD players. There's a business near me that does something similar, but they restrict access by age to high schoolers and older, and you have to play the games there; you can't rent them out.

r/SubSimGPT2Interactive for the lulz is my #1 use case

i do occasionally ask Copilot programming questions and it gives reasonable answers most of the time.

I use code autocomplete tools in VSCode but often end up turning them off.

Controversial, but Replika actually helped me out during the pandemic when I was in a rough spot. I trained a copyright-safe (theft-free) bot on my own conversations from back then and have been chatting with the me side of that conversation for a little while now. It's like getting to know a long-lost twin brother, which is nice.

Otherwise, i've used small LLMs and classifiers for a wide range of tasks, like sentiment analysis, toxic content detection for moderation bots, AI media detection, summarization... I like using these better than just throwing everything at a huge model like GPT-4o because they're more focused and less computationally costly (hence also better for the environment). I'm working on training some small copyright-safe base models to do certain sequence prediction tasks that come up in the course of my data science work, but they're still a bit too computationally expensive for my clients.

We don't. It probably is. Mastodon is the way, but they need to fix a few things themselves.

It will legit be a fantastic era for Linux on the desktop though... imagine how cheap we'll be able to get perfectly good hardware.

'tis true that women's bodies hold great power, and not irrelevant at all to the discussion at hand. rather than reiterate and attempt to paraphrase jaron Lanier on the topic of how male obsession with creating artifical people is linked to womb envy, I'll just link to a talk in which he explains it himself:

https://youtu.be/rGqiswuJuQI?si=oAKvWrtlji4yrfpd&t=42m05s

Like any occupation, it's a long story, and I'm happy to share more details over DM. But basically due to indecision over my major I took an abnormal amount of math, stats, and environmental science coursework even through my major was in social science, and I just kind of leaned further and further into that quirk as I transitioned into the workforce. bear in mind that data science as a field of study didn't really exist yet when I graduated; these days I'm not sure such an unconventional path is necessary. however I still hear from a lot of junior data scientists in industry who are miserable because they haven't figured out yet that in addition to their technical skills they need a "vertical" niche or topic area of interest (and by the way a public service dimension also does a lot to help a job feel meaningful and worthwhile even on the inevitable rough day here and there).

My "day job" is doing spatial data science work for local and regional governments that have a mandate to addreas climate change in how they allocate resources. We totally use AI, just not the kind that has received all the hype... machine learning helps us recognize patterns in human behavior and system dynamics that we can use to make predictions about how much different courses of action will affect CO2 emissions. I'm even looking at small GPT models as a way to work with some of the relevant data that is sequence-like. But I will never, I repeat never, buy into the idea of spending insane amounts of energy attempting to build an AI god or Oracle that we can simply ask for the "solution to climate change"... I feel like people like me need to do a better job of making the world aware of our work, because the fact that this excuse for profligate energy waste has any traction at all seems related to the general ignorance of our existence.

I think that there are some people working on this, and a few groups that have claimed to do it, but I'm not aware of any that actually meet the description you gave. Can you cite a paper or give a link of some sort?