AI server build query

[needhelpwithlife]

Hi there community,

I've been doing custom builds for over 15 years now. Went from basic to watercooling (soft tube/ hard tube) every inch and back to basics. So I'm very comfortable with the general building of computers.

However for my business, I need to build my first Threadripper build. Now I assume there's no trick to it, same process, different platform. But if anyone has any suggestions or advice that I need to be aware of, and also if you could just go over the spec, making sure I've not made any blunders, that would be greatly appreciated.

Specs are as follow. If you have any questions please feel free to ask. Even if it's just confirming that it's a solid build and I've not made any mistakes, that's also appreciated.

Purpose : Build server + Local AI for inference and training

Component	Part
GPUs	2x NVIDIA RTX Pro 6000 Blackwell Workstation Edition 96GB GDDR7 (192GB total VRAM)
CPU	AMD Threadripper 9970X
Motherboard	ASUS Pro WS TRX50-SAGE WIFI A
RAM	4x 32GB Samsung DDR5-4800 ECC RDIMM (128GB, quad-channel)
PSU	Seasonic PRIME PX 2200W (ATX 3.1, 80+ Platinum)
CPU Cooler	SilverStone XE360-TR5 360mm AIO
Storage	Samsung 990 Pro 8TB (models) + 2x 990 Pro 2TB (OS/scratch)
Case	Corsair 9000D RGB Airflow
Fans	Noctua NF-A12x25 (loadout TBD)

Key assumptions made and reasoning

RAM : 4800 vs 5600. Availability and price. For my general use case it seems like a waste of money. Not running VMs.
RAM : 128GB vs 256GB. For inference and training, I won't need 256GB. I don't see the use case for it.


Again, if any of my assumptions are wrong. Please lets discuss.

 

[jigger]

It’s a bit difficult to give advice with out knowing what you intend to run, but I’m assuming you’re not spending the thick end of £20k on graphics cards for nothing, but you’ll probably want at least 256gb.

 

[needhelpwithlife]

I have said just above the table : Purpose : Build server + Local AI for inference and training

Local AI, inference and training. VRAM heavy
Dev server.

Neither of which are particularly RAM heavy. Going from 128GB to 256GB is a £4k difference, and I won't be using it.

 

[aaronyuri]

What kind of AI models? What do you plan on developing?

It's a lot of capex to throw at a local box that will be heavily outdated in 2-3 years. Have you considered opex models like bare metal rentals or cloud instances?

 

[needhelpwithlife]

What kind of AI models? What do you plan on developing?

It's a lot of capex to throw at a local box that will be heavily outdated in 2-3 years. Have you considered opex models like bare metal rentals or cloud instances?

Multiple use cases. Different models for different purposes.

When you say heavily outdated, I feel that you're not really in touch with the current market. I sold a 4090 for the exact price I paid for it 2 years before it. RTX 6000 ADA is still selling for £5-6000, 4 years old, 48GB memory

A cloud instance for my use case would run through $60-80 per day. And in three years, I would have paid that price, and I would own nothing.

I am not asking for the business use case for it, I know what I need for my business. I'm just asking for build advice.

 

[aaronyuri]

Multiple use cases. Different models for different purposes.

When you say heavily outdated, I feel that you're not really in touch with the current market. I sold a 4090 for the exact price I paid for it 2 years before it. RTX 6000 ADA is still selling for £5-6000, 4 years old, 48GB memory

A cloud instance for my use case would run through $60-80 per day. And in three years, I would have paid that price, and I would own nothing.

I am not asking for the business use case for it, I know what I need for my business. I'm just asking for build advice.

If you say so. The description of your use case means nothing that anyone can quantify or scrutinise against the components.

When you say advice, I feel that you're really here seeking validation. You've clearly invested a lot of thought in this and will divulge no meaningful information; click buy.

 

[jigger]

You’re not really explaining much. AI is quite topical, but aaronyuri makes a good point. Spending £10,000 cards this late into a product cycle when new cards are shipping to OEMs probably isn’t a great use of capital for a small business.

 

[needhelpwithlife]

If you say so. The description of your use case means nothing that anyone can quantify or scrutinise against the components.

When you say advice, I feel that you're really here seeking validation. You've clearly invested a lot of thought in this and will divulge no meaningful information; click buy.

Seeking validation from who man, you think I'm going online posting pictures of this ****. Sorry man, I have no time to get into some online squabble.

I asked the same question from another forum, and someone taught me that CEB is not a form factor, and that I can buy most E-ATX cases. Useful info.

You questioned the business sense. How does that make a difference in output quality.

Local llm, inference, creating agentic systems that can monitor and report on our systems. More VRAM, better model, high quality.

That's like me asking someone, how do I build a fast car, and someone starts questioning the destination. It's a car, you want it to be reliable and fast.

 

[needhelpwithlife]

You’re not really explaining much. AI is quite topical, but aaronyuri makes a good point. Spending £10,000 cards this late into a product cycle when new cards are shipping to OEMs probably isn’t a great use of capital for a small business.

Go online, and go buy 4090. That's last Gen. Used for £1800. The concept of "last gen" gpu has vastly changed due to the intrinsic value of memory due to AI. Unfortunately.

Me waiting 12-18 months for Rubin, only to hope that I can purchase it at MSRP, and it does not get scalped and price hiked for 12k for a year (like what happened with current gen). Which will mean it would be 24+ months till Rubin is even close to 10k.

I'm not buying a gaming RIG, that's 24 month of work that I wouldn't have had done, that I would have had to pay cloud infra fees for $40-50 per day, $1500+ per month.

 

[jigger]

No you’re not buying a gaming rig are you… Rubin shipped a quarter ago and is available now. Your competition might already have access to systems. You’re not explaining the use case other than AI things, so it’s difficult to advise. Maybe consult a systems architect firm.

 

[needhelpwithlife]

No you’re not buying a gaming rig are you… Rubin shipped a quarter ago and is available now. Your competition might already have access to systems. You’re not explaining the use case other than AI things, so it’s difficult to advise. Maybe consult a systems architect firm.

Rubin has not even been released for major data centers. And bearing in mind that datacenter release pipelines are not the same as workstation, data centers get new gpus 6-12 month ahead of workstation releases.

And the specific use case for AI is irrelevant. Local llm, inference, creating agentic systems that can monitor and report on our systems, training models for specific business cases. More VRAM => better model => higher quality output. The exact use, model, training technique, doesn't change what this threat was about. I just wanted to know if there was anything that I need to bear in mind building a threat ripper. Maybe if somone has had a similar experience with heat dissipation with similar builds. Is the AIO ****... is the CPU/Ram combo something i should be aware of. Stuff I don't know.

Not whether this is the right business decision, or to start a debate online.

Some very helpful guy sent me this link https://acecloud.ai/blog/multi-gpu-rtx-pro-6000-blackwell-workstation-build/

Cheers man. I think it's fair to say we can leave this chat

 

[jigger]

Rubin started shipping about 8 weeks ago and biggest firm probably took delivery before then… You also missed the point about renting vs buying this late in a release cycle and gulf in performance that may or may not be a significant factor.

 

[needhelpwithlife]

You are either confused or are wasting time. And I don't think you are very clued up on this.

A. Who got these shipping of Rubin? Data centers? Not workstations, different class of products, different release cycles.
B. Who cares that some data centers have more powerful GPU. I'm not running a data center. Datacenter gpu and consumer/workstation are COMPLETELY different products. You need to understand this. Not the same product. Different. They share very little in common. Use different memory types. RTX Pro 6000 is not in the same class as B200 (Blackwell Datacenter GPU), which is about 4-5 times the price. Data center gpu are released up to a year prior to consumer/workstation product
C. Buying this late in release cycle, I already responded that it could take 18-24 month before Rubin is the current price of Blackwell (for consumer, again, i'm not running a data center)
D. Gulf in performance. It's going from 4nm to 3nm, and historically that mean 15-30% boost in throughput. Throughput != Better models. VRAM does. Thus the value of GPUs not dropping

I'm not going to respond anymore. Or check this thread. You've made it a waste of time, you had 0 intention of actually helping.

 

[jigger]

Rubin shipped some time ago, Nvidia are already talking about and taking orders for it’s successor Rubin Ultra which I believe is due to start shipping in around 6 months. Blackwell is very late into its cycle now and is very over priced.