Switch Topology

anything I don't mind · 19 Feb 2013 at 20:32

I am currently trying to setup a new switch topology at one of our clients. This is the first time I have done any topology design with switches and was looking for some tips. We have:

6x HP 2810-48g (2x 1 gigabit uplink gbic)
2x HP 24 port gigabit with 6x gbic ports didn't take down model number (existing core switches)
2x fiber switches

The storage is all configured and was easy enough to set up. Although we did not have to set up any round robin configuration in vmware, we tested by turning off one of the switches and the hp p2000 continued to work ok even with one switch. We only just got the kit so we are just in testing phase at the moment. The switches are stackable but it is not real stackable as they don't have stack ports on the back like cisco or juniper, basically all the stacking does is allow it to be manageable from one IP. They still have to be connected to each other using gigabit uplink or standard ports.

My initial plan is to have 2x core switches and then 2 switch stacks, one on the ground floor and one on the fifth floor.

2x core switches will be 2x 24 port gigabit switches.
4x 48 port switches in one stack in basement
2x 48 port switch in fifth floor

Connect two core switches together using 2x 1 gbit ports and then run a 2x 1 gigabit from each core switch to each basement switch (using lacp?) then for resilience run a 1x gbit link from each switch in the stack to the other two switches in the stack.

Using a total of 8 ports in each core switch and 5 ports in each 48 port switch.

Then run 2x fiber gbic down from commander switch in fifth floor stack to each core switch. Then run a 2x 1 gigabit from the second switch to the main switch in the fifth floor stack.

-Question I have is how can i improve this topology or configuration.
-Where would i use lacp and is that only for use in stackable switches or could i use that for the 2x 1gigabit connections to each switch.
-Should I run a third gbic down from the second switch for resiliance and if so where to connect it, one of the core switches?

anything I don't mind · 21 Feb 2013 at 12:32

Ok I am trying to make a diagram at the moment but visio 2010 is a nightmare, the connectors are retarded.

The FC swich is a HP San switch 24/8

FC switch:
http://h18006.www1.hp.com/products/storageworks/sanswitch824/specs.html
hi res image of fc switch
http://core4solutions.com/media/cat...ab33525d08d6e5fb8d27136e95/a/m/am868a_1_2.jpg

5x 2810-48G switch:
http://h10010.www1.hp.com/wwpc/ca/e...2136298-12136326-12702378-77904477.html?dnr=1

1x ProCurve Switch 2848:
http://h10010.www1.hp.com/wwpc/ca/e...2136298-12136316-12136322-29584733.html?dnr=1

2x ProCurve Switch 2824:
http://h10010.www1.hp.com/wwpc/ca/e...2136298-12136316-12136322-29584735.html?dnr=1

Due to the specs on the 24 port (as it has less bandwidth) I am thinking of using 2x 2810-48g as the core switches and just add the 24 port to the basement stack.

anything I don't mind · 21 Feb 2013 at 18:10

This is what I think i will be going with, any opinions would be appreciated.

It will be 24 port ESXi and 12 port uplink + 4 port gbic uplink leaving a lot of free ports out of the 96 ports available.

We got about 600 write and read iops on exchange 2010 2 hour performance test in raid 10. We have 300gb 15k sas disk.

Still think it seems crazy to use such disks when we have 1000+ iops SSD that can be bought for relatively low price.

anything I don't mind · 28 Feb 2013 at 20:42

The fiber channel switches are not the core switches. The FC switches are only for storage traffic between the esxi and the p2000 san.

Where exactly is the loop? Yes i am new to network design but I have asked several people from different locations and no one has said that this design is terrible and will create loops. Sure if we had another 5k to spend we could get some proper core switches to replace the 2x 2810 which are definitely not core switches.

What is the other alterantive exactly, i should have labelled the switches.

The reason I linked (daisy chained) the client switches in the basement with 1gigabit like that was because someone suggested that they should be connected together as well as in to the core. Initially i just had 4xgigabit going to the 2810 core. They do not do real stacking like cisco, ie no shared bandwidth across them, but you can still stack them for management, the reason why the person suggested linking them together, for resilience.

The only vlans in use will be the voip, management/vmotion and dmz.

The only alternative i can think of is to have one stack in the basement, that would be from the image 2x the core and the 3x client switches in one stack and then just connect each switch to every other switch or daisy chain them. But then if one switch goes down it can affect the whole network.

I designed it this way so that we could lose a "core" switch and everything will still work.

What have your EX2010 IOPS reqs got to do with this, btw?

Nothing i just put that in for information purposes.

I wouldn't bother linking the "access" switches together unless they do actual stacking and can do LACP across the switches. If not, you're probably better off linking each switch to each "core" switch and allowing STP to shut down one of the links. Use LACP if you have the cable count back to the cores because it'll converge faster than STP for a single link failure (so you have two links per LACP port channel, two port channels per switch).

Clearly this uses up a fair number of your ports but I'm guessing these switches aren't full/busy or they would be having some pretty noticeable issues by now...

Best practice would have course be to relegate all of those switches to just the access layer, put a stackable pair of L3 switches in (even a couple of 24-port 3750s with IP Base would perform adequately) and have single 2-port LACP trunks from each switch. Keep the broadcast domains down by using one VLAN per switch and reduce the IP usage by using /26 (or /25 if you have switches off switches...) per switch to avoid flooding the uplinks with chatty windows traffic.

Ok thanks I will see if we can get some cisco 3750. I will remove the 1gigabit link between the client switches and have 1x 2gigabit lacp to each core switch. I will take a look at seeing if i can specify the STP root on the 2810.

I was talking to some vmware guys and they suggested that i should do 2x port per core switch for servers and then 1x per core switch for management and vmotion on a vlan. I will see if our license can do a vds but i think it will have to be a vswitch. Instead of having 6 nic for servers and a separate physical switch for vmotion.

anything I don't mind · 10 Mar 2013 at 22:56

I didn't buy the switches, my boss did and he is not very good in my opinion. But now we have five of those hp 2810. I spoke to another guy at work from a different site, who i think knows his switching/storage/vmware quite well. He said that the switch topology won't work too well for various reasons. He suggested just making one stack with the 2810 and put everything in to that, with standard up links.

He also said that i had to use lacp on the ports on the switches that are used by esxi or the esxi networking won't perform. But from what I am aware you can't do lacp without a vds and a enterprise plus license. I also spoke to some vmware people about it and they denied that having to use lacp for more than one physical nic on a vswitch was a requirement. But he seemed pretty sure of himself.

I think I will suggest to my boss that we get a network specialist in just to make sure it is right first time, to be honest it is realy him who should be doing the switching. I don't have the luxury of being able to focus just on switches or just on x. I am expected to do everything from desktops to all servers technologies. But I thought i would make a diagram and see what i would learn and I have learned a fair amount.

anything I don't mind · 11 Mar 2013 at 14:46

You can do lacp on a vmware distributed switch as i confirmed with vmware guys and in one of my train signal vmware courses that covers vds they do say that lacp is supported on it. But that is irrelevant as we don't have the license for it anyway.

I will take a look at the etherchannel.

Poor outfit? I blame my boss because he should be a senior technical manager but i don't think he is technical enough. He used to do my job (my boss) and then the guy who was the senior technical manager left and he took that job but i don't think he is technical enough. I am fairly new to the third line role and it does not help that my technical boss is not very technical. But i wouldn't class it as a poor outfit, we do well at many other areas, but when it comes to switching that should be the senior technical manager responsibility but he is not up to it. But irrelevant politics but thought i would add it, for clarity.

While i'm in the position i thought i would give it a go (the switching) but after hearing a lot of feedback from various sources, I think it is bit of out scope of my ability at current and will try and pass it on my boss who should be doing it anyway or get a third party in to do it.

Thanks for your help though.