Recommendations for a 10gbps NIC?

katie279

katie279

katie279

Associate
Joined
20 Jul 2007
Posts
2,160
Location
A sunnier or damper area than Ron-ski....
Hi all,

Looking to install some 10gbps NiCs in two PCs, any recommendations? (Seems to be a bit of a minefield of overheating chips and underperforming elements from the searches I've done, so any solid recommendations much appreciated!)
 
What medium are you thinking about using?

RJ45 - Copper

Intel X540-T1 (These are older cards now)
Intel X540-T2 (These are older cards now)
Intel X550-T1
Intel X550-T2

I know there is an Intel X7XX but I can't think of a model number off the top of my head)

SFP - OM3/4 - Fibre

Mellanox ConnectX 4
Mellanox ConnectX 5
Intel X710-DA

These are not cheap options but they will perform trouble-free.
 

TP-Link 10 Gigabit PCI Express Network Adapter (TX401) works well. I paid £90 each for 2 of them and they have been flawless. Been running them 24/7 in my server for 2 years without a problem.​

 
Get second hand server NICs from eBay, from a reputable brand such as Broadcom, Intel, Mellanox etc, with the correct height PCI bracket and SFP module/RJ45 option. Most of mine are Intel X520's or Intel 82599.
 
That motherboard might be a minor bit of an issue.

You only have 1 full PCI 4.0 (Or is it 3.0?, just checked it's confirmed PCIe 3.0) x16 slot that's fully wired according to the specs. Usually reserved for your GPU.

The second PCI 3.0 x16 slot is only a x4 wired slot. AND it shares bandwidth with your m.2 NVME drives as well if they are installed. So this is unlikely able to get the most from a 10gb network card. You can certainly fit one in, but full 10gb bandwidth is unlikely.

Your remaining slots are x1 slots (electrically wired, or physically actually a x1 slot), so wouldn't be able to saturate the 10gb card either.

It might not be worthwhile jumping to 10gb network with this motherboard. Not unless if you want to swap out your GPU to another slot.

For reference, I've got a 10gb card in a PCI 2.0 x4 Slot (Which should be more than 2GByte/second but as it's a shared channel it only nets me aroud 600-700MByte/second). Still OK. But not 10gb (950MByte+/second)
 
Last edited:
Meddling-Monk said:
It might not be worthwhile jumping to 10gb network with this motherboard. Not unless if you want to swap out your GPU to another slot.

For reference, I've got a 10gb card in a PCI 2.0 x4 Slot (Which should be more than 2GByte/second but as it's a shared channel it only nets me aroud 600-700MByte/second). Still OK. But not 10gb (950MByte+/second)
Click to expand...

Which is still a significant upgrade and would easily saturate a SATA SSD, for relatively small cost its worth upgrading.
 
Last edited:
Firestar_3x said:
Which is still a significant upgrade and would easily saturate a SATA SSD, for relatively small cost its worth upgrading.
Click to expand...
Oh without a doubt it's a worthy upgrade. I've certainly enjoyed my upgrade of it. :)

The only thing is, we don't know what purpose this network upgrade is for from the OP. And we also don't know how many other devices are plugged in and sharing the bus (on the system with that motherboard). All of which can impact on the throughput of the network. It might be that they want to share the m.2 NVME, in which case they might be scratching their head as to why they're getting such a strange number for network throughput instead of standard 1gbps/2.5gbps/5gbps/10gbps numbers. Which can lead to all sorts of diagnosis procedures that may end up ripping out fully functional stuff and replaced when it's entirely unnecessary as well, so just pointing out that there'll be issues for the OP to recognise if and when they attempt it.
 
I use a Intel X520 in my server. Just be cautious, mine was a Dell part (wasn't mentioned in the listing) and would stop the machine from posting with a certain amount of ram in it. I had to do a mod to the card (Block pins 5 & 6 on the pci-e connector) to allow it to post with all the ram. I also had to modify the firmware (Just a bit flip which can be done in linux with ethtool) to allow the use of any DAC/SFP+ Modules. Was a right pain, but was quite satisfying to get it full working in the end :cry:

If getting a x520, just make sure your getting a Intel variant.
 
Sorry for bringing this thread back from the dead (I'd rather not make a new thread for the same topic).

I'm also looking into 10g networking for my upcoming NAS, especially given the dozens of TB I have to store on it. I don't see much benefit from having 2 ports, so I'm only looking at single port cards.

So far I've found the following:
Intel X540-T1 (x8 physical slot, supposedly runs at PCIe 2.0 x8 and only supports 10/1/0.1G speeds)
Marvell AQC113C (x4 physical slot, assuming it runs PCIe 3.0 x4, but have heard some mention it running PCIe 4.0 x1 and supports 10/5/2.5/1/0.1G speeds)
Intel X550-T1 (x4 physical slot, runs at PCIe 3.0 x4 and supports 10/5/2.5/1/0.1G speeds)

On the NAS side, I'm re-purposing the old PC on a Gigabyte Z97 motherboard. It has 3 physical PCIe 3.0 x16 slots, that can run at either x16/x0/x0 or x8/x8/x4 speeds (also a bunch of useless PCIe 2 x1 slots). The x8 in the first slot won't be an issue since I'll keep a GPU there for encoding (since the CPU doesn't have built-in graphics). So no problems there.

On my PC side though, I've got the Asus B650E-E in the sig. Whilst it has the same 3 physical x16 slots, they only run at x16/x0/x4 or x8/x4/x4 speeds, admittedly at faster PCIe 5.0 for the first 2 and PCIe 4.0 for the last one. Here-in lies the problem, the best I can do on this end is a x4 spare slot, since the GPU is in the first slot.

An X540 will be fine in the NAS as I can stick it in the 2nd slot running at PCIe 3.0 x8. For the the PC, ideally I need an 10g NIC that won't be hampered by only 4 PCIe lanes. Only issue is that the Intel X550 has barely any availability or even at a good price, looking like double the price of the X540. The Marvell I've heard mixed things and some say not to bother with anything other than Intel.

Is it possible to get away with an Intel X540 in the PC with only 4 PCIe 4.0 lanes? It'll still only run at PCIe 2.0 speeds... admittedly my NAS will be unlikely to saturate the 10G connection (5G would be realistic, but it doesn't look like there's much 5G options available and it's not worth it when 10GB options cost only a little more). Theoretically a PCIe 2.0 at 4 lanes does up to 2GB/s and the 10GB NIC shouldn't get above 1GB/s.

Are there any other options by now that would be reliable for only 4 PCIe 4.0 lanes?
 
Last edited:
I can't really answer your question, but whenever i see NIC's mentioned anywhere they always suggest to go for an Intel chipset.
 
Last edited:
RSR said:
What medium are you thinking about using?

RJ45 - Copper

Intel X540-T1 (These are older cards now)
Intel X540-T2 (These are older cards now)
Intel X550-T1
Intel X550-T2

I know there is an Intel X7XX but I can't think of a model number off the top of my head)

SFP - OM3/4 - Fibre

Mellanox ConnectX 4
Mellanox ConnectX 5
Intel X710-DA

These are not cheap options but they will perform trouble-free.
Click to expand...
Solid recommendations.

Can get them second hand from server refurb sites.


SFP+ will generate less heat and if you can use DAC cables, they will use less power than fibre or RJ45.
 
Last edited:
I wouldn’t go x520/82599 at this point if buying something, PCIe 2 based cards are ancient, and lack the now expected CPU offloading to handle data flow. The CX3/4 is fine on 4xPCIe 3 (or above) lanes, certainly more capable and power efficient than an x520 at this point, circa £20-30 used on the low end, and the 4 with SFP 28 is backwards compatible with SFP+ transceivers.

Older cards like the Myricom or X520/82599 based stuff that’s PCIe 2 based do make sense if you already have the hardware or it’s stupidly cheap, you’re only saving single digit watts on each card, so the payback on a £20-30 outlay isn’t worth it generally, as in a decade when you break even, you’ll have upgraded anyway.
 
RavenLunatic said:

TP-Link 10 Gigabit PCI Express Network Adapter (TX401) works well. I paid £90 each for 2 of them and they have been flawless. Been running them 24/7 in my server for 2 years without a problem.​

Click to expand...

+1 I’ve got two of these between my PC and server working really well with no issues
 
Avalon said:
I wouldn’t go x520/82599 at this point if buying something, PCIe 2 based cards are ancient, and lack the now expected CPU offloading to handle data flow. The CX3/4 is fine on 4xPCIe 3 (or above) lanes, certainly more capable and power efficient than an x520 at this point, circa £20-30 used on the low end, and the 4 with SFP 28 is backwards compatible with SFP+ transceivers.

Older cards like the Myricom or X520/82599 based stuff that’s PCIe 2 based do make sense if you already have the hardware or it’s stupidly cheap, you’re only saving single digit watts on each card, so the payback on a £20-30 outlay isn’t worth it generally, as in a decade when you break even, you’ll have upgraded anyway.
Click to expand...
Following on from this, if you use the free version of VMware ESXi 8.

X520/82599 are not supported in VMware 8, but they will work. However, CX3 have had the driver completely pulled from VMware 8.
 
I’d imagine (or at least hope) if ESXi compatibility was a requirement, it would have been stated. It’s been 3+ years since 8 rolled out and broke support for a load of older hardware, between that and the change of direction in terms of customers, Proxmox seems to get quite a bit more love in the home/SMB side now, where as historically seeing it in anything other than charity/start-up’s with zero budget was rare.
 
Regretting purchasing my 10Gtek 10Gb PCI-E NIC Network Card, Single Copper RJ45 Port, with Intel X540-BT1 Controller. It drops repeatedly (fortunately only briefly but not good during a PUBG match) and I missed that it doesn't do WoL.
 
You must log in or register to reply here.
Back
Top Bottom