Although i used VD a lot with the quest1, I've only recently found a few things.
The latency you see on your monitor when VD is in desktop (latency desktop) mode means nothing. You need to be playing a game on the quest, take it off, but cover one of the eye pieces so it thinks its still on, then bring streamer app on top of the Pc desktop screen. The latency see now see (latency VR) is the one you are after.
Alyx was giving me absolutely shocking latency, I was seeing 120ms+ at times.
If you see unacceptable latency like that, then, counter-intuitively, you need to pull the bitrate down. Like most, the first thing I did was wack it up to the max. I have no idea why reducing bitrate helps latency. But the VD developer consistenly suggests in on the discord discussion, and I think he has his own around 75-80. I have mine down to 75Mps and my latency is now around 40ms. This is with a wifi5 router, so Wifi6 might be better.