I have not noticed any slowdown or stuttering on Gravel on X.
I tried it earlier and honestly it's pretty unplayable, it must run at 10 fps at times, it's worse when you have the windscreen in view, the reflection especially on the Las Vegas track are terrible and performance takes a hit.
I originally bought it and played it on the PS4 as the developers said the the Xbox version would not have HDR (something to do with the software engine not working well on xbox).
Have you played it recently as I noticed that it does have HDR and wondering if that was patched into it later and ruined performance but when you played it was OK?
Edit.....
Just found the interview with developers who said:
Can you explain why HDR won’t be available on Microsoft’s console?
When the game’s development started, the Unreal Engine 4 didn’t support HDR on the platform yet. This is the reason behind the absence of HDR. Obviously, we’re working so that our next titles will have HDR compatibility on this platform as well.
And I just read that it was patched a couple of weeks ago with HDR, will be interested to know if you've played since then and if the performance has taken a hit as a result.