Have you the router setup as an access point? That helps reduce the latency.
Melmac, thanks for the response!
The router doesn't have an AP software setting, but the manufacturer had a tutorial that showed how to manually set it up. All it said to do was to disable DHCP server on the router, and then assign it an IP address on the main router's subnet. Assuming that turns it into an access point, that's what I did.
I used it with VD and seemed to be getting decent results.
However, I also posted the diagram in the VD discord, and the developer suggested that it would be best to put the dedicated router in router mode on it's own subnet, b/c he said all of my traffic was still going through the main router the way I had it set up there.
I reconfigured it to his suggestion, and this is an updated diagram, including where I moved the CAT5e to the WAN port. It seems to be working as intended like this, and I'm waiting for his response as to whether I have it set up correctly.
I played Alyx with it in this config, and was getting 80 FPS and 40-43ms... I dont know how good that is, but it looked/played good to me. Once I get confirmation the setup is correct, I can work on tweaking the performance a bit.
https://i.imgur.com/Misms8c.png