I've been doing something similar in our modern house, not quite finished as other jobs on the makeover are also being worked on. Our house is a 3 story house and we only have a technicolor router on the ground floor in the hallway which manages to serve the whole house. Admittedly not perfect in the loft room but it covers the rest of the house well.
I personally would over specify the number of sockets since you only really want the chore of running cables to different rooms done once. The downstairs cabling is the worst to achieve, if you can get the developer to do it before the house is finished then all the better. Otherwise it's a ball ache to take up floor boards on the first floor to get cables to the required positions to achieve a neat finish.
With your setup I would look at reducing the wireless routers to two as this gives the network some resilience to always having a spare one in case of failure.
If one of the locations where you've marked for a router upstairs or downstairs is a suitable cupboard where things are less likely to be tampered with and cool, then I'd choose that to mount a patch panel and switch in a wall mounted 4u bracket quite high up in some dead space. Judging by your diagram you might only have plaster board to mount something to so you might need to mount something like plywood between the wall studs and then mount a bracket through the plywood.
Plan where you might want sockets, especially if you might want to make use of sockets as alternative telephone sockets. You might want telephone sockets next to TV's if you want Sky or Sky multiroom or if you might want a VOIP setup at some point rather than conventional telephone points where everyone can access the outside line simultaneously, eg eavesdrop.
I'd consider between 6 and 8 behind the Lounge PC. The reason this sounds high is because 4 of them can be used to connect between the router ports and the patch panel to provide resilience in the event that the switch fails, allowing any socket to be patched through directly to the main router. The sockets at this location would serve the PC, but perhaps also a smart TV, Blu ray, games console, Sky, Telephone point, Fax?
I'd put 4 sockets the opposite end to the PC and 2 in the dinning room.
As for upstairs, if the cupboard up there is a humid airing cupboard I might be inclined to use the downstairs cupboard for the patch panel and switch. The problem with putting the patch panel and switch downstairs though is that i'd then be inclined to take all the spare capacity connections from the patch panel that weren't used for the downstairs and wire these with cable lengths long enough to reach the farthest point upstairs via the loft plus some spare slack on each cable.
The bedrooms I'd consider which might have TV's, PC, games console and telephone requirements as they also may not all be required in the same location of a particular room. At least with the upstairs you can either have all the cabling either connected to the downstairs patch panel and just have it on hand in the loft ready to drop a connection down to a bedroom or run the connections as you need them, to having the patch panel in the upstairs cupboard. I'm provisioning 8 connections in my main bedrooms here as I'm putting a double point behind flat screen TV on wall plus another below at normal height so as to be easily accessible. The other 4 i'm putting on the wall behind the bed so I have the possibility of phones beside bed or internet radio or network point for a laptop to use in bed.
So in short, I'd use a decent wireless router as your main router, plus a wireless router as an access point on the 1st floor, a semi managed / smart switch and a patch panel. Use a big enough patch panel to satisfy plenty of connections as above and pair it with a switch that has enough ports for your current needs.
