DDR InfiniBand network by a N00b

Associate
Joined
8 Dec 2018
Posts
3
Ok so I replied to a post here as a google search got me her and im looking for help.
I was given hardware and want to see if this is possible on my current setup with this old hardware.

I have posted on other forums and here is what I said there..


Ok so I'm new to this and I have a test lab at work.
I have 4 esxi 6.7 boxes - will get to them later
a windows 10 pc (dell precision 5810)
Our guys were getting rid of a mess of hardware so I grabbed it and figured time to play around.

I stuck a mellanox card in the win10 box and drivers installed, albeit, MS drivers. but it seems to be good?

I also connected a cable to the switch and the light on the switch lit up but in win10 the NIC says cable unattached.

switch - Flextronics F-x430066 8 Port 4x SDR Infiniband
cable - Mellanox Mcc4l30-300 Microgigacn Latch 0.3 M Infiniband Cable
cards - HPE InfiniBand 4X DDR Conn-X PCI-E G2 Dual Port HCA (483514-B21)

so few questions...
what is needed on win10 side other than what I have done?
if I put another card in a secondary pc, and just attach both to the switch, just give static IP's and that is it?

so out of the manual... I get the green light with just the cable from pc to switch and nothing else to switch...

The GREEN LED indicator to the left of each port will light when the physical connection is established (that is, when the unit is powered on and a cable is plugged into the port with a functioning port plugged into the other end of the connector)

in Windows 10 dev manager - shows using Mellanox drivers signed by Microsoft
ipoib6x.sys - 4/25/2018 ver. 5.50.14643.0

C:\>mst status -v
MST devices:
------------
mt26418_pci_cr0 bus:dev.fn=01:00.0
mt26418_pciconf0 bus:dev.fn=01:00.0

C:\>mlxfwmanager -d mt26418_pci_cr0 --query
Querying Mellanox devices firmware ...
FATAL - Can't find device id.
FATAL - Can't find device id.
Device #1:
----------
Device Type: N/A
Part Number: --
Description:
PSID:
PCI Device Name: mt26418_pci_cr0
Port1 MAC: N/A
Port1 GUID: N/A
Port2 MAC: N/A
Port2 GUID: N/A
Versions: Current Available
FW --
Status: Failed to open device

with that last command, it doesn't look like im ready for this... although windows 10 device manager says device is running ok, I cant query on mlxfwmanager...

so lookin for help ..

if this isn't doable on my hw let me know. I have tons of cards and tons of cables so Id like to see if this works...
 
Associate
OP
Joined
8 Dec 2018
Posts
3
so I just plugged a cable into my card and into the switch and I get the following... am I good? just give ip addresses to the card?

[Dec-08-2018 18:36:49:905][4CA8] 0x03 -> OpenSM 3.3.11 UMAD
[Dec-08-2018 18:36:49:905][4CA8] 0x80 -> OpenSM 3.3.11 UMAD
[Dec-08-2018 18:36:49:908][4CA8] 0x02 -> osm_vendor_init: 1000 pending umads specified
[Dec-08-2018 18:36:49:909][4CA8] 0x80 -> Entering DISCOVERING state
[Dec-08-2018 18:36:49:914][4CA8] 0x02 -> osm_vendor_bind: Binding to port 0x1635ffffbf0bb5
[Dec-08-2018 18:36:49:937][4CA8] 0x02 -> osm_vendor_bind: Binding to port 0x1635ffffbf0bb5
[Dec-08-2018 18:36:49:937][4CA8] 0x02 -> osm_opensm_bind: Setting IS_SM on port 0x001635ffffbf0bb5
[Dec-08-2018 18:36:49:938][5144] 0x01 -> osm_si_rcv_process: ERR 3610:
Bad LinearFDBTop value = 0xC000 on switch 0xb8cffff00472b
Forcing internal correction to 0x0
[Dec-08-2018 18:36:53:385][2AA4] 0x80 -> Entering MASTER state
[Dec-08-2018 18:36:53:387][2AA4] 0x02 -> osm_ucast_mgr_process: minhop tables configured on all switches
[Dec-08-2018 18:36:53:388][2AA4] 0x80 -> SUBNET UP
[Dec-08-2018 18:36:53:393][3C54] 0x01 -> log_trap_info: Received Generic Notice type:4 num:144 (CapabilityMask, NodeDescription, Link [Width|Speed] Enabled, SM priority changed) Producer:1 (Channel Adapter) from LID:1 TID:0x0000000000000002
[Dec-08-2018 18:36:53:393][3C54] 0x01 -> osm_get_port_by_mad_addr: ERR 7504: Lid is out of range: 0
[Dec-08-2018 18:36:53:393][3C54] 0x01 -> trap_rcv_process_request: ERR 3809: Failed to find source physical port for trap
[Dec-08-2018 18:36:53:393][3C54] 0x02 -> log_notice: Reporting Generic Notice type:4 num:144 (CapabilityMask, NodeDescription, Link [Width|Speed] Enabled, SM priority changed) from LID:1 GID:fe80::16:35ff:ffbf:bb5
[Dec-08-2018 18:36:53:399][2AA4] 0x02 -> osm_ucast_mgr_process: minhop tables configured on all switches
[Dec-08-2018 18:36:53:399][2AA4] 0x02 -> SUBNET UP
[Dec-08-2018 18:36:53:517][3C54] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:401b:ffff::ffff:ffff
[Dec-08-2018 18:36:53:517][4574] 0x02 -> log_notice: Reporting Generic Notice type:3 num:67 (Mcast group deleted) from LID:1 GID:ff12:401b:ffff::ffff:ffff



so i have to get a second computer with the card to try...
 
Back
Top Bottom