I'm no Anarchist, but that's not what Anarchism is. Anarchism is a fully developed horizontal system, rather than vertical. The idea that Anarchism is simply "no rules" is an unfounded stereotype, there's lots of Anarchist theory.
While I personally think it's very difficult to achieve, it wouldn't be for the reasons you've listed. Simply destroying government isn't an Anarchist ideal, building up parallel structures like networks of Mutual Aid to replace the state and make it redundant is Anarchist praxis.
Anarchists believe that if horizontal power structures are in place, it becomes difficult to go against that current. Ie, if everyone has power, in order to gain more power than another, one must require people willing to give up their power to submit to them in order to push against others. This theoretical group would also have to be strong enough to go against the rest of the public.
It's similar to why Communists believe once Communism is globally achieved, there wouldn't be mechanisms for Capitalism to come back, just like Monarchism is almost nonexistant today.