In theory yes, wasn't Shelias wheels successfully sued for not providing insurance to men? If they can be then why should we treat clothes shops differently?
What I can never wrap my head around in these situations is, if you get refused service for some reason in a shop then most people would just next door (or 5 doors, 10 doors etc.) to a competitors shop who will do it.
Personally I don't think it is discrimination, refusal of making that specific cake =/= refusal of service.
Sheila's Wheels always did sell to men ( I got a quote from them), and were never sued. They just market themselves as a specialist for women.
Clothing shops are, to some extent, a symptom of a societal bias in how we expect men and women to dress differently. This is, in itself, a form of discrimination, albeit a widely accepted one.
Of course, there are also size differences between men and women, necessitating a different cut. Men only and women only shops are acceptable since there is, as a whole, not a problem with discrimination one way or the other (i.e. the market is balanced to the extent that no-one is disadvantaged).
In addition, anyone can walk in to a men's or women's clothing store and buy anything off the peg. A bespoke men's tailor may, in theory, struggle to refuse to produce a suit for a woman due to being caught up with discrimination issues, but could cite practical reasons for the refusal - the different cut required wouldn't fit with standard starting patterns, and they would likely lack the specific expertise.
There were no such practical reasons for the cake shop.