Fog lights are allowable to be twice as bright as brake lights.When you're sitting in a low down car behind a newish car with blinding LED rear clusters at your eye level then yes it is definitely annoying.
Since brake lights and rear fogs are the same intensity, it's odd that one would annoy you but not the other?
Edit: Allowable brake light brightness is 60-185 candela, allowable rear fog light brightness is 150 -300 candela