You need a certain amount of interaction with the beer. Some seasoned brewers will even take taste samples whilst in the FV.
Beer has been brewed for thousands of years. There didn't used to be steriliser, or plastic equipment which cleans well, or well sealed FV / kegs. Some people now still brew with nothing more than a cotton cloth over the top of their FV.
If you manage to infect a brew in a FV by drawing off a sample or popping in a sterilised hydrometer then you will probably be one of the most unlucky brewers around. That said I still thoroughly sterilise my equipment, just don't be afraid of the brewing process!
It's not about right or wrong, everyone's experience will be different, but I think to enjoy it, it needs to be hands on
*goes off to grab a glass of beer from my just about finished beer in the FV
*