They're not accusing you of anything. They're simply saying "we know you don't have a TV licence and would like to confirm why this is".
Given that you have failed to reply or get in touch with them, obviously they now think that you are not replying as you don't have a licence, and therefore have a right to take you to court to understand the situation. If the court finds that you ought to have done something to remedy the situation (ie getting back in touch) they will probably order you to pay costs.
So its ok for Apple to send me a letter asking why I haven't bought an Apple device, and if I refuse to respond its okay for them to take me to court to find out why I haven't bought an Apple device?
What makes the TV licence any different. If I need one, I'll buy one, no one has the right to ask me why I haven't and no one can automatically treat me like I'm breaking the law by assuming I both don't have one AND need one because I do watch live TV.
No one has a right to take you to court to understand a situation... thats absurd. They have no right to know why I don't have a tv licence, they can take me to court to try andprove I do require a licence and don't have one, they can't do it on a whim with the sole intention of finding out if I need one or not.
Either way, while I don't assume that this kind of thing will all lead to 1984 style situations, I still feel very uncomfortable with the ease with which people accept they must hand over information when asked by anyone without ANY reason for having that information. YOU should get to choose who you give any of your own information to, how much of it and when. You can't harass everyone hoping to catch one bad egg, you can't treat everyone as guilty with no proof.. the police can't do that, let alone an agency with zero authority over anything at all.
Every time you randomly give up information WITH NO NEED and for no purpose, you simply make it easier for more people/companies to ask for more information and you do eventually end up with a society which will struggle to keep the right to privacy if everyone gives it away without thinking for so long it becomes normal. The simple fact is the TV licencing thing is PROOF this is the case, because an agency with no authority to act or demand anything routinely barges into peoples homes with most people unaware they have no authority, its become so normal most people give in instantly without thinking they can simply tell them to **** off.
EDIT:- whats the legality of watching live streams of say football illegally? Do you need a tv licence to do so because its a live broadcast, or does the fact its illegal mean its not covered. slightly more seriously, is the tv licence only for specific live broadcasts from English sources, IE is it for any normally accessible live broadcast in the UK, do you need one if you use a sat box and get some eastern European tv package for sports or whatever?