When you are in the UEFI your CPU is not idle, at least one core is being utilized constantly with monitoring and other UEFI tasks. So when you see a difference between your UEFI temp and your in OS idle temp it only makes sense that idle temps will be lower.
Operating systems are optimized to minimize CPU usage when "idle" to reduce power consumption and decrease heat output from the CPU/GPU. The UEFI/BIOS is not optimized in this way, it would require a lot more code and likely larger ROM chips for the BIOS itself.
To give an idea, I watch my CPU temps on my Z170 Pro4+ 6600k jump from 29c idle to 56c just by opening Libre Writer. It spikes up there briefly then drops to the mid 30s. If just one, rather lightweight app can cause this then it is easily understandable that the UEFI could cause the temps we are seeing. Yes, I said we, my board reports the same/similar BIOS temps.
At the end of the day, the temps are well within threshold for any CPU and go down to normal idle levels in the OS. This is not a problem, more of a misunderstanding about how the UEFI works.
