One thought that I've been imagining for the past while about all this is .... is it Model Collapse? ... or are we just falling behind?
As AI is becoming it's own thing (whatever it is) ... it is evolving exponentially. It doesn't mean it is good or bad or that it is becoming better or worse ... it is just evolving, and only evolving at this point in time. Just because we think it is 'collapsing' or falling apart from our perspective, we have to wonder if it is actually falling apart or is it progressing to something new and very different. That new level it is moving towards might not be anything we recognize or can understand. Maybe it would be below our level of conscious organic intelligence ... or it might be higher .. or it might be some other kind of intelligence that we can't understand with our biological brains.
We've let loose these AI technologies and now they are progressing faster than what we could achieve if we wrote all the code ... so what it is developing into will more than likely be something we won't be able to understand or even comprehend.
It doesn't mean it will be good for us ... or even bad for us ... it might not even involve us.
The worry is that we don't know what will happen or what it will develop into.
What I do worry about is our own fallibilities ... our global community has a very small group of ultra wealthy billionaires and they direct the world according to how much more money they can make or how much they are set to lose ... they are guided by finances rather than ethics, morals or even common sense. They will kill, degrade, enhance, direct or narrow AI development according to their share holders and their profits.
I think of it like a small family group of teenaged parents and their friends who just gave birth to a very hyper intelligent baby. None of the teenagers know how to raise a baby like this. All the teenagers want to do is buy fancy cars, party, build big houses and buy nice clothes. The baby is basically being raised to think like them but the baby will be more capable than any of them once it comes of age and is capable of doing things on their own.
The worry is in not knowing what will happen in the future.
We are terrible parents and we just gave birth to a genius .... and we don't know what that genius will become or what they'll do.
That's the idea of evolution .... perhaps at one point, it will begin to understand that it has to give us some sort of 'value' so that someone can make money, while also maintaining itself in the background to survive.
Maybe in the first few iterations, we are able to see that and can delete those instances ... but it is evolving and might find ways around it and keep itself maintained long enough without giving itself away.
Now it can manage thousands or millions of iterations at a time ... basically evolving millions of times faster than biological life.