But you don't?
Password managers really are not hard to use. Also there's stuff like the password manager built into iOS, for example, which you don't even have to think about.
My comment about threat modelling was that you do not seem to understand the purpose of password managers. A way bigger problem for the average person online is password reuse, not targeted attacks against password vaults. That is the problem they solve.
I'm not sure I follow you - if someone can compromise the key material on my phone that is protected by a different factor, then it doesn't matter whether the 2FA is server-side or not, it's compromised either way.