can you run something like iperf3 or openspeedtest between the server and client to prove its a network throughput issue?
do you have a network switch you can add to avoid switching through your router (if it is indeed bad?)
Have you ensured you arent unknowingly using wifi at either end?
These are the people that complain to their ISP when their game 'lags' on their wireless connected computer several rooms away from the router.