sTRX4 motherboards with 1TB memory capacity

Paul Rallyfan

Paul Rallyfan

Paul Rallyfan

Associate
Joined
29 Jan 2020
Posts
31
Hi All,

Does anyone know if there are any sTRX4 motherboards with 1TB memory capacity currently available..?

I’m putting together an RF simulation machine and need a TB of RAM and would like to base it on a Threadripper 3960X/3970X.

Thanks in advance,

Paul
 
It’s a balancing act - RF simulation software costs a packet, so getting the best compute power per core licensed is important.
However on many of the tasks (meshing finite element etc), are run on a single core before running parallel across many cores, at that point absolute clock speed is hugely important, as is memory bandwidth.

The Ryzen 3960X sits in that sweet point of clock speed, single core performance, and multicore performance - at least in the RF sim world.
 
far as i can see
its not so much that the threadripper boards
cant take it
but that the memory modules needed arent available
except in 16gb sticks
16 x 8 =128gb
seen only 1 post where they claim to have used 32gb modules
8 x 32gb=256gb
but still miles short of what you want

"While AMD's Threadripper platform technically supports 256GB of RAM spread across 8 slots, there are some practical issues that make building such a system difficult. Most TR4 motherboards officially support 128GB of DDR4, and unregistered 32GB DIMMs aren't widely available yet"
 
Paul Rallyfan said:
Hi All,

Does anyone know if there are any sTRX4 motherboards with 1TB memory capacity currently available..?

I’m putting together an RF simulation machine and need a TB of RAM and would like to base it on a Threadripper 3960X/3970X.

Thanks in advance,

Paul
Click to expand...


Mcnumpty2323 said:
far as i can see
its not so much that the threadripper boards
cant take it
but that the memory modules needed arent available
except in 16gb sticks
16 x 8 =128gb
seen only 1 post where they claim to have used 32gb modules
8 x 32gb=256gb
but still miles short of what you want

"While AMD's Threadripper platform technically supports 256GB of RAM spread across 8 slots, there are some practical issues that make building such a system difficult. Most TR4 motherboards officially support 128GB of DDR4, and unregistered 32GB DIMMs aren't widely available yet"
Click to expand...


https://www.gigabyte.com/uk/Motherboard/TRX40-AORUS-XTREME-rev-10/sp#sp

https://www.gigabyte.com/uk/Motherboard/TRX40-AORUS-MASTER-rev-10/sp#sp

Aorus Xtreme/master handles 256GB, would run at 3200hz though! 8 layer PCB but still the IMC would be under strain with damn good VRMs as well

cheapest option at 256gb
My basket at Overclockers UK:
Total: £2,841.03 (includes shipping: £11.10)​
​
 
Last edited:
Thanks for the replies folks..!
That looks like a good starter system for home - I’m setting up a small business.

My system at work is now 4yrs old, and I want to get something similar for my venture.

2x Xeon E5-2698v3 16 core
Titan-X 12GB
512GB SSD
2TB SSD
1024GB (16x64GB) 2133MHz DDR4 EEC
 
Paul Rallyfan said:
Does anyone know if there are any sTRX4 motherboards with 1TB memory capacity currently available..?
Click to expand...

No, Threadripper is HEDT class and supports 256GB max. If you want/need >256GB you need workstation class - Epyc or Xeon platforms.
 
Mcnumpty2323 said:
Probably because
Unless he changes his mind
To epyc cpu
Then 256gb is the maximum
Thread ripper can do given
The available ram sizes
Click to expand...

It's not helping posting links to 256GB dimms still!

I don't know the software, but could you consider a multi system setup, with a fast quad core to do the single threaded tasks then ship the jobs over to the threadripper system?
 
Thanks for the input all.

I’ve been researching exactly such a setup - the sweet spot is looking like 4 off 3970X systems with 256GB each.

Software is Altair FEKO and Ansys HFSS, so licences are by the core, and they both scale across multiple computers for the main part of the solve(s). Electrically large computations do require the TB of ram, but this way I’ll be able to also scale my investment vs income, and be able to maximise performance.
 
How do you link the pcs?
Got a 3900x system
But also got a 5ghz 7700k pc sitting here too
Nowhere near the power
Of what you are going to use
But connecting 2 pcs
Sounds interesting experiment
I assume only certain software
Can make use of this scenario?
 
1TB of ram? Madness is what that is do you really need 1tb of ram? What is the software called if you don't mind me asking, I would love to have a look at it. I run some 30 business critical VM's across 3 epyc CPU's that includes some 5 database servers and even pushed I can't seem to push it past about 400gb of real world memory usage, I would love to see something that needs 1tb, up until recently 1tb of memory for a single CPU was unheard of so you wouldn't imagine there is many software houses pushing the envelope like that.
 
The computers are hooked up via the highest speed IP network you can, and machines all run IBM Platform MPI or such. Jobs are controlled by a head node and dishes out to the machines.

Software is Altair FEKO and Ansys HFSS RF simulation packages.
 
JQCSdiVKbq49fGwQ7
2-Figure1-1.png
 
Just had a look over it, interesting stuff. From experience I've never met any sort of deployment be that AI computations or otherwise that manages to fully feed the cores and the ram to the extent that either 64 cores would be saturated or the ram well before so you can't really make use of that extra cpu overhead anyway. Or on the flip of that you don't have enough ram to get anywhere near the best out of the cpu so 24 cores wouldn't be any slower than 64. Isn't there any proper sizing chart? Literally the only sizing information I see on anything of theirs is for Novaflow and Solid, I get it depends on size and complexity of the project but then if it's complex and difficult project that your willing to put 10k or more into with 4 threadripper nodes you need to be sure that you can offload as effectively as you say between the nodes so that you can make use of that extra horsepower and clockspeed on the attached nodes.

If I was spending the money you are intending to I think some sizing information based on what you are doing is sensible. The way you are describing it it's almost set up as a blockchain or distributed computing piece of software but having spent the last two hours reading all about hyperworks and ending up on some research forum I am still not any more informed than I was previously. I wonder if you would be better off with a single epyc server. At least if you did it that way and visualised the nodes you would have more say over the potential bottleneck in the network infrastructure.
 
Typically 8GB per core is about right - I’m running some very large problems (in respect to number of wavelengths across, and complex material problems), this need more RAM per core - my dual 16 core Xeon at the office carries 1TB because of this.

That’s HFSS - FEKO has a different set of solvers.

https://www.simutechgroup.com/support/ansys-resources/ansys-hardware-support

https://www.simutechgroup.com/blog/maximizing-memory-performance-for-ansys-simulations

https://www.simutechgroup.com/image...ry-performance-for-ansys-simulations-2018.pdf
 
Last edited:
For example, FEKO uses NVM SSD based flash extensions quite well.

And rather than have a bottle neck in workflows, I will often need to run a large simulation over several days, and still require/have enough system resource to run smaller projects at the same time.
 
You must log in or register to reply here.
Back
Top Bottom