Yeah openwrt should be great. It uses nftables as a firewall on a Linux distribution. You can configure it through a pretty nice ui, but you also have ssh access to configure everything directly if you want.
The challenge is going to be what the ISP router supports. If it supports bridge mode then things are easy. You just put your router downstream of it and pretend like it's a modem. Then you configure openwrt like it's the only router in the network. This is the opposite of what you've suggested, using the upstream ISP router in pass through and relying on the openwrt router to get the ipv6 GUA prefix. (You might even be able to get a larger prefix delegated if you set the settings to ask for it)
If you don't have bridge mode then things are harder. There's some helpful information here https://forum.openwrt.org/t/ipv6-only-slaac-dumb-aps/192059/19 even though the situation is slightly different since they also don't want a firewall. But you probably need to configure your upstream side on the openwrt router similarly.
Also looking more, the tplink ax55 isn't supported by openwrt. If you don't already have it, I'd get something that does. (Or if the default software on the ax55 supports what you want, that's fine too. I just like having the full control openwrt and similar gives)
I don't know a lot about tailscale, but I think that's likely not relevant to what's possible (but maybe relevant to how to accomplish it).
It sounds like the main issue here is dns. If you wanted to/were okay with just IP based connections, then you could assign each service to a different port on Bob's box, and then have nginx point those ports at the relevant services. This should be very easy to do with a raw nginx config. I could write one for you if you wanted. It's pretty easy if you're not dealing with https/certificates (in which case this method won't work anyway).
Looking quickly on google for npm (which I've never used), this might require adding the ports to the docker config and then using that port in npn (Like here). This is likely the simplest solution.
If you want hostnames/https, then you need some sort of DNS. This is a bit harder. You can take over their router like you suggested. You could use public DNS that points at a private IP (this is the only way I'm suggesting to get public trusted ssl certificates).
You might be able to use mdns to get local DNS at Bob's house automatically, which would be very clean. You'd basically register like
jellyseer.local
andjellyfin.local
on Bob's network from the box and then setup the proxy manager to proxy based on those domains. You might be able to just doavahi-publish -a -R jellyseer.local 192.168.box.ip
and thenavahi-publish -a -R jellyfin.local 192.168.box.ip
. And then any client that supports mdns/avahi will be able to find the service at that host. You can then register those names nginx/npn and I think things should just workTo answer your questions directly
I'd be happy to try and give more specifics if you choose a path similar to one of the above things.