Many car dealers are idiots and will form an opinion on a person based on how they are dressed without any form of discussion or engagement. I have experienced this on more than one occasion and I simply walk out as it's ALWAYS their loss but at the same time I am not seeking to simply 'have a drive' in a few cars!
If I walk into a showroom there is a good opportunity for someone to sell me something as I am interested. How you dress will have an impact on the idiots above but any good sales person and there are plenty out there will engage you in conversation and how you come across at this point will determine how serious you are taken, which is fair in my book as sales people have a job to do and they need to qualify each and every opportunity. If they were all accomplished our OP wouldn't get a test drive but would leave feeling well served as he has no intention of buying a car and the "yes but I will in the future and want to evalutate some options" is, we all know, crap in most cases. Good sales people will spot this, deal with you professionaly and let you leave while they focus on winning proper business.
The good news for our OP is much of the motor trade is made up of poor sales people so you will get attitdue, with no real customer engagement based on how you dress, rudeness......and plenty of test drives.