Thoughts on Cisco SG550 vs 3850X vs 9000 series

ginjaninja · 30 Jul 2019 at 20:28

Hi

i need 2x9 10Gbe UTP connections (over 2 stacked switches) for my core - supporting backup server and all flash 4 node vsan cluster .

i cant find anything in the enterprise Cisco stable that provides a good solution for 10Gbe utp (cat 6)....i am not even sure Cisco provide an SFP module for UTP?

then i came across the SG550 range, seems perfectly good enough with UTP ports...
I dont require much functionality, no routing protocols, just static routing and access lists between a few vlans and 10GBit UTP ports for the servers with stacking and redundant power. Dont care about no IOS particularly if they do the job. The vsan PCI NVME nodes consume ~ 3Gbps peak per port, across 4 storage links and 4 data links any one time, peak. Most of time they are idling.

any thoughts on SG550 and choices for 10GB UTP. Any real reasons not to consider SG range. Cisco do their level best to hide the range.

Am considering SG range for access switches (4x48 port POE (30W)) as well.

thanks for feedback.

khrall · 30 Jul 2019 at 22:37

You need a deep packet buffer for HCI based solutions. For a production environment, I don't think the SG550's will cut it, you might find they drop a large amount of packets.

ginjaninja · 31 Jul 2019 at 08:58

ah..yes

we are currently using Dell N4032 for the vsan nodes...they have 72Mbit packet buffer, the SG550s we were looking at have 12Mbit, so quite a difference.

presumably though...reading is always going to be faster than writing....so even if the switch was top of the line...no amount of switch buffering would be enough..the weak link in the chain will be the write speed of the disks (once the memory buffer on the vsan nodes is full).

Hellsmk2 · 31 Jul 2019 at 10:53

Have a look at the Dell S4048 or S4148 switches. Both perfect for vsan deployments, and a pair can be had for 12k ish via a good reseller.

If you're familiar with ios, then you won't have any issues with these.

Throrik · 31 Jul 2019 at 12:56

I wouldn't use the SG series switches for a VSAN Cluster or even anything other than an office environment. We use SG300s in our office LAN but proper enterprise switches in our Private Cloud offering mostly 4500s as Cores and 3850 Stacks as Edge/Access switches. The 9ks are probably more 'suited to a VSAN deployment on paper, but I run pretty high end environments on Catalysts too so bear that in mind.

ginjaninja · 31 Jul 2019 at 21:23

Hellsmk2 said:
Have a look at the Dell S4048 or S4148 switches. Both perfect for vsan deployments, and a pair can be had for 12k ish via a good reseller.

If you're familiar with ios, then you won't have any issues with these.

looks good these have 12-16MB packet buffer,,,,the small 12 port might have been perfect but they dont stack...will looking into the 24 port ones... thanks for suggestion.

ginjaninja · 11 Aug 2019 at 20:29

khrall said:
You need a deep packet buffer for HCI based solutions. For a production environment, I don't think the SG550's will cut it, you might find they drop a large amount of packets.

I spoke to VMwares VSAN presales via my account manager, who said they dont care about packet buffer.
but i think they were just reading from the public build recommendations as opposed to real world understanding...frustrating..

khrall · 12 Aug 2019 at 23:02

ginjaninja said:
I spoke to VMwares VSAN presales via my account manager, who said they dont care about packet buffer.
but i think they were just reading from the public build recommendations as opposed to real world understanding...frustrating..

They even mention it in one of their troubleshooting guides!

https://storagehub.vmware.com/section-assets/troubleshootingvsanperformance

Switchgear class
VMware does not provide any qualified list of approved switches for vSAN or any other solution.
vSAN heavily depends on network communication to present distributed storage to the cluster,and therefore, the class of switches chosen can be critically important to the performance capabilities of vSAN.
Switchgear have different performance capabilities that go far beyond the advertised port speed. For switchgear supporting a vSAN environment, look for enterprise grade, storage-class switchgear using large dedicated port buffers, as well as sufficient processing and
backplane bandwidth to ensure that each connection can run at the advertised wire-speed.

The use of fabric extenders can result in a non-optimal communication path, and are not recommended for storage fabrics, including vSAN.

ginjaninja · 14 Aug 2019 at 14:10

khrall said:
They even mention it in one of their troubleshooting guides!

https://storagehub.vmware.com/section-assets/troubleshootingvsanperformance

Switchgear class
VMware does not provide any qualified list of approved switches for vSAN or any other solution.
vSAN heavily depends on network communication to present distributed storage to the cluster,and therefore, the class of switches chosen can be critically important to the performance capabilities of vSAN.
Switchgear have different performance capabilities that go far beyond the advertised port speed. For switchgear supporting a vSAN environment, look for enterprise grade, storage-class switchgear using large dedicated port buffers, as well as sufficient processing and
backplane bandwidth to ensure that each connection can run at the advertised wire-speed.

The use of fabric extenders can result in a non-optimal communication path, and are not recommended for storage fabrics, including vSAN.

thanks that clears that up.