The best succinct distinction between erotica and pornography I recall was a quote in a feminist anti-censorship book I read a while ago - something along the lines of it's erotica when they drink wine afterwards. The distinction is, IMO, more of a class thing. Erotica is posh pornography. Or pornography with some style, if you prefer. Or sometimes not. I recall a porn brand from a while back called "Swedish Erotica" that certainly wasn't very refined. Most often IME, "erotica" means "pornography that the person calling it erotica approves of". Which ties in with the class thing, of course.
In a sense it's true that he's not a pornographer. But only because he's retired. He defintely was a pornographer. And, as you say, the general public doesn't care. The general public wouldn't have known if he wasn't currently making such a big deal about it. He's being rather silly.