AMD announce EPYC

Vince · 20 Sep 2019 at 13:15

Little update - It is in and working a treat! All the fiber is set up (even though i only have 8gb/s and this was provided with 16gb modules

- A couple of new FC switches might be in order!) I have set up esxi, set up the SAN, set up vsphere and transferred a single server from Intel to AMD. Easy work!

Still some licensing, testing, cleanup etc to do but she is singing now!

SupraWez · 20 Sep 2019 at 13:46

Very nice, beastly boxes

Vince · 20 Sep 2019 at 14:43

SupraWez said:
Very nice, beastly boxes

They are little beasts

Room for another socket per box and also a serious amount of memory. I would say they should easily do for the foreseeable future! It's also been an absolute doddle moving servers over during my testing. All I have to do now is wait for some old school sfp+ modules to arrive and ill hook up my 4500's 24TB's onto the fiber fabric as well as the other two EPYC's. I think the migration should be done and dusted much quicker than I thought it might.

Do need to sort out that non persistent storage for logs error as well but frankly that can wait. For now I think i'm just going to sit back and chill for a bit safe in the knowledge that the right kit is now in and working.

ChrisD. · 20 Sep 2019 at 14:51

Vince said:
Do need to sort out that non persistent storage for logs error as well but frankly that can wait.

Easily done. Select the host, configure, advanced system settings, edit and browse for Syslog.global.logDir (filter for syslog, it'll show) and point it to a persistent datastore in the format of [datastore]/logfolder (you have to create the folder first or it'll error). If you're having more than one host logging to the datastore then I'd suggest setting Syslog.global.logDirUnique to true.

How did you migrate the test VMs over?

Vince · 20 Sep 2019 at 14:57

ChrisD. said:
Easily done. Select the host, configure, advanced system settings, edit and browse for Syslog.global.logDir (filter for syslog, it'll show) and point it to a persistent datastore in the format of [datastore]/logfolder (you have to create the folder first or it'll error). If you're having more than one host logging to the datastore then I'd suggest setting Syslog.global.logDirUnique to true.

How did you migrate the test VMs over?

It was as easy as shutting down, remove from inventory, add to the inventory on epyc server (shared FC connected storage), press the go button. One restart, re-license windows and you are done. There is literally nothing to it it's so incredibly simple that I'm not sure what I was ever concerned about.

ChrisD. · 20 Sep 2019 at 14:59

Vince said:
It was as easy as shutting down, remove from inventory, add to the inventory on epyc server (shared FC connected storage), press the go button. One restart, re-license windows and you are done. There is literally nothing to it it's so incredibly simple that I'm not sure what I was ever concerned about.

Yup very simple especially with shared storage.

Vince · 20 Sep 2019 at 15:07

ChrisD. said:
Yup very simple especially with shared storage.

The only thing holding me back from having them all running and hitting all the migrations this weekend is that I need specific SFP+ modules for these old Brocade FC switches. I need AJ716A transceivers at the switches and whatever goes at the other end (AJ176A are the only ones I can seem to get working switch side) but i've only got stacks of 16gb/s and AJ718 8gb/s transceivers so I guess now I wait until the 20 I found on ebay get here, and yes i'm buying modules from ebay so that I don't need to buy new FC switches as i'm caining through budget like crazy, in fact i'm so over budget for this year it's not even funny

I do think I can talk my way out of the discrepancy by looking at my under spend the last two FY's and using that variance to cover my overspend so far. The question is will that difference see me through the year? I'm less confident on that one.

ChrisD. · 20 Sep 2019 at 15:09

Did you look on fs.com? Delivery may take a bit longer but they're usually very cheap.

Vince · 20 Sep 2019 at 15:58

ChrisD. said:
Did you look on fs.com? Delivery may take a bit longer but they're usually very cheap.

I didn't, I found 20 of them on the bay for £33 so just went with that. Dirt cheap for a load of adapters and the seller tells me they will be here by Wednesday so that is ok in my book even if not all of them work.

I would be interested to know if anybody else is going Rome and the experience you have.

jigger · 21 Sep 2019 at 13:46

Vince said:
The only thing holding me back from having them all running and hitting all the migrations this weekend is that I need specific SFP+ modules for these old Brocade FC switches. I need AJ716A transceivers at the switches and whatever goes at the other end (AJ176A are the only ones I can seem to get working switch side) but i've only got stacks of 16gb/s and AJ718 8gb/s transceivers so I guess now I wait until the 20 I found on ebay get here, and yes i'm buying modules from ebay so that I don't need to buy new FC switches as i'm caining through budget like crazy, in fact i'm so over budget for this year it's not even funny I do think I can talk my way out of the discrepancy by looking at my under spend the last two FY's and using that variance to cover my overspend so far. The question is will that difference see me through the year? I'm less confident on that one.

Ah, but you have done the right thing for the company while dealing with serious and unforeseen security problems and also improving system longevity and power use. If you do get in hot water I’m working on a new project, I’m sure we could fit you in.

Vince · 21 Sep 2019 at 14:58

jigger said:
Ah, but you have done the right thing for the company while dealing with serious and unforeseen security problems and also improving system longevity and power use. If you do get in hot water I’m working on a new project, I’m sure we could fit you in.

I'm sure I'll be fine but do appreciate the sentiment dude. At the end of the day it's the right thing to do and in context isnt really that much money.

jigger · 21 Sep 2019 at 15:27

Vince said:
I'm sure I'll be fine but do appreciate the sentiment dude. At the end of the day it's the right thing to do and in context isnt really that much money.

TBH the move will pay dividends. I’d ask for a pay increase

Vince · 21 Sep 2019 at 16:59

jigger said:
TBH the move will pay dividends. I’d ask for a pay increase

I know right, space for growth (lots of - that we didn't have before), a much better throughput to our backup infrastructure (veeam & 4500 StoreOnce devices) which was a significant bottleneck before. All new software from esxi 5 to esxi 6.7. While I am at it i'm going to introduce some 10gb/e networking to a few places where copper throughput is a bit of an issue.

Once I'm finished on this refresh the disk infrastructure and 8gb/s brocade switches will likely be the biggest bottleneck. I wan't to add more and faster storage but I think it might be sensible to schedule that in 2021 giving epyc 12 months to settle in.

Vince · 26 Sep 2019 at 14:31

@ChrisD. - Setting up some fiber from my StoreOnce using Catalyst to work alongside Veeam is killing me, I can't seem to work out how to set fiber up from a Guest server using NPIV, I have enabled NPIV at the san switches, in the VM options I have enabled it and generated new WWNs. I have zoned the switch and as far as I can see it should work but nothing, literally nothing happens.

The switch sees no virtual devices, all the ports stick physical, the VM when I boot it does bugger all, I mean it boots but there is no sign of any devices etc. If I try and add an RDM disk it also sees nothing at the LUN selection screen. Basically ive spent days and have no idea what to look at and where best to focus my attention. Any pointers would be massively appreciated.

Cheers dude.

ChrisD. · 26 Sep 2019 at 18:36

Is the Veeam server a VM on the hosts? Can you see the paths to the storage on the hosts at all?

Vince · 26 Sep 2019 at 18:47

ChrisD. said:
Is the Veeam server a VM on the hosts? Can you see the paths to the storage on the hosts at all?

Yes VM is a virtual machine running on a FC connected host. When you say path to storage on the hosts, you mean in the paths within the virtual machine config and not the host?

ChrisD. · 26 Sep 2019 at 18:51

Vince said:
Yes VM is a virtual machine running on a FC connected host. When you say path to storage on the hosts, you mean in the paths within the virtual machine config?

No, I mean from the host itself can you see the WWN's? I haven't used FC for such a long time I forget where exactly to look but it should be under host > configure > storage adaptors.

Vince · 26 Sep 2019 at 18:57

ChrisD. said:
No, I mean from the host itself can you see the WWN's? I haven't used FC for such a long time I forget where exactly to look but it should be under host > configure > storage adaptors.

Ill take some screenshots... might be easier to explain give me 2 mins.

Vince · 26 Sep 2019 at 19:10

ChrisD. said:
No, I mean from the host itself can you see the WWN's? I haven't used FC for such a long time I forget where exactly to look but it should be under host > configure > storage adaptors.

Ok so my setup for the world to see:

Image 1 is the Catalyst setup on the StoreOnce 4500, Image 2 is the storage adapters on the host, Image 3 The VM setup showing the NPIVs, Image 4 is the paths on the host. My understanding is that the host presents scsi processors to the VM (the number of initiator ports correlates exactly with the number of SCSi processors) which I can then do stuff with, only the host isn't presenting any new hardware to the VM:

ChrisD. · 26 Sep 2019 at 20:16

Unless I’m missing something all your LUNs are showing dead or error.