Edit : haha, thanks to the pendulum demo I actually understood where the problem is coming from ! It's the way the screen handles backlighting !
Even in the ultimate engine range without triggering LFC, 50 Hz for example is much brighter than 100 Hz ! So when it goes to 40 FPS it goes to 80Hz, which is brighter than 50 Hz !
It's basically just the brightness of the monitor that is not properly set to follow the frequency precisely ! If it were, there wouldn't be any problems !
I noticed something similar, though mainly when trying either Samsung Magic Upscale or Game mode, can't remember which. Depending on what's happening on-screen the brightness would change and look similar to flickering. It was essentially unplayable for me so I made sure to keep both turned off. Maybe that transitional FreeSync flicker I notice at 48 fps is also the brightness switching.
PITA monitor really to figure out what's what, but I agree that when it's happily in its optimal FreeSync range it's top notch even with an Nvidia card*.
*But not in borderless full screen for me.