For the first time in 10 years I've just had a TV license inspector knock on my door. I didn't feel like arguing so I showed him our TV, turned it on and changed the source to Freeview. No picture was displayed so he was happy and left.
Not really sure of my rights in this situation, but I had nothing to hide.
The only time they have a "legal" right to enter your property (including you letting them in of course ;p), is if they've gained a warrant to do so. They're supposed to provide proof to the courts that you have a tv &/or are watching live broadcasts, but I've seen plenty of reports (you can demand access to courts/beeb as to what was provided as evidence after the fact) that they often lie about what "they saw as proof".
If they just turn up, you can legally tell them to *expletive* & also remove their "implied right of access " (right to enter your garden/drive etc to come pester you.
No idea where this stands when it comes to blocks of flats etc.