Took the plunge - First Home Server

BogEyes

BogEyes

BogEyes

Associate
Joined
7 Jan 2006
Posts
1,566
Location
somewhere between venus and mars
So it seems I've reached middle age.

I wanted to to build a DIY NAS setup on a tight budget to have a go at running UNRAID. I was thinking a few 3-4TB HDD's 2nd hand, paired with a Rasberry Pi or the innards of an old laptop, in a 3d printed enclosure should cover it.

So naturally, one week later, I've pulled the trigger on a LGA 2011-3 Dual Xeon Motherboard, x2 Xeon E52650's, 3x 8TB SAS Drives, 64GB of DDR4 and x2 Tesla V100's.

I need to get several other essential components such as adaptor boards and pci-e riser cables, and I suspect this is going to be the tricky bit.

It's going to be a custom build rather than plug and plug, and so fabricating my own enclosure and cooling solution is going to be on the cards.

Please roast and berate my decision making, or feel free to offer some advice to someone who a week ago didn't know the difference between a router and a modem.
 
@BongoHunter Haha, yes, the server PSU's are something I am looking into, thankfully it seems possible to pick them up quite cheaply! Also, I have a bunch of 12v projects on the go, which it transpires they are perfect for, so I might see if I can get a discount on 3-5 of them at once.

The motherboard side of things has been the bit which has taken the longest to decipher tbh. SLi and Crossfire made sense to me back in the day, but two CPU's has taken me several days to wrap my head around :cry: My research had me settled on a SuperMicro X10DRH-iT (or similar), but after a number of declined offers etc, I've wound up with an ASUS Z10PE-D8 WS. Which I'm glad about, because it looks cool. Though, it has pained me to spend over £100 on a motherboard over 10 years old! I wonder what they retailed for?!

@Armageus I'm fairly certain that what I've purchased so far is all compatible with each other, i've double and triple checked, but I'll run what I've got past you...I'm still in the return window for most of it so please let me know if I've made a boob!

ASUS Z10PE-D8 WS (private seller, sold as fully working, so got my fingers crossed on this tbh)
2x Intel Xeon E5 2650V4 (1 year warranty)
3x Toshiba HDEPK41DAB51 8TB 7200RPM SAS-12Gbps 512E SIE 3.5" (1 year warranty)
4x 16GB 2133MHZ Ecc Ram (I'm not sure what format this is yet as I'm waiting on delivery. I've taken a gamble on this because I'd like to go for LDIMM, but not the end of the world if not as it was for a price it would have been daft not to get it.
2x Nvida Tesla V100 16GB SMX2's without heatsinks, just just the bare board.

As it stands I am aware I need a HBA Board (and associated SAS to SATA cables), but in trying not to make any mis-steps, I'm hesistant to start sourcing these until I'm completely sure of what I need.

I feel that I got the SAS drives for a price that justified the decision, £6 per TB compared to the £10~ SATA based CMR drives (from private sellers), I've nearly doubled my potentional storage space. I feel that the 8TB drives are going to suit me better than 4tb (or less) long term, when considering an 8TB drive doesn't use 2x more energy than a 4TB drive. Also, I didn't really fancy helium drives (I'd read it's hit or miss with regards to leaking after several years).

Yeah energy usage is on my radar, and it's sort of a factor in why I've gone down this route. I'll be trying my best to reduce power consumption to a minimum. The V100's are the passively cooled ones and I don't even have the heatskinks for them! And of course I'll be taking a risk on SMX2 to PCI-E adaptors.

What is the actual purpose, as the hardware doesn't seem ideal for anything in particular?
Click to expand...

Fair question, and difficult to answer briefly, but I'll try my best (It's a common story we're all familiar with).

TLDR: Unhappy with life, looking for change, I believe that taking the bull by the horns and going all in with building my first Home Network, MAY allow me at the very least to have a stepping stone towards achieving my dreams and aspirations. I want to be able to utitilse LLM's without daily limits.

I've chosen the above compoents based on my specific wants and needs, which I am trying to capture all in one go;

- To be able to run a local large language model when I don't have access to the interent or have been 'locked out for the day'.
- To be able to disconnect from all of the digitial services I am literally throwing money at week in, week out, for the sake of conveniece. Music Streaming, Film/Television subscriptions, Cloud Storage, Subscription based LLM's, gone. I would like a replacement for all of these services, so that I can tailor and expand my preferences as my knowledge and interests grow.
- To be able to 'back up' and protect, all of my own, and immediate families digital lives, should anything happen, there is redundancy and reducing the chance of memories lost to data decay and corruption.
- To be able to run various 3D Software Design programs in a lag free/smooth enviroment, as some of the things I am designing and making are beginning to become a bit of a faff as the models get bigger. It feels like being on dial up again at times.
- 3D printing, see above really, I'm slicing stuff left right and centre and I've been looking a iMac Pros etc.
- Home Security, I've had an ongoing home CCTV project for several years which I've struggled to get off the ground.
- Distributed Computing Projects such as folding at home sound great to me, and I would like to do 'my bit'.
- Scaleable, I want to build something that I can add more System Memory, more Local Storage and more VRAM too as I go. I want something that can grow with me, at least for, lets say, the next 5 years.

@jigger

Unless you know this system makes sense for the use case and will perform the intended tasks well, I’d maybe consider un-pulling that trigger because you could find yourself with an assortment of headaches that require some serious attention to resolve.
Click to expand...

Hopefully my above post answers your questions, but I appreciate the candidness.

I am aware of the power, energy, physical and practical constraints of the system I want to build.

However, I can weld (tig, mig and arc), I can braze (oxy-acy), I can solder, I can fabricate metal, I can work with wood (love me a pre War #8 Record), I can rebuild petrol and diesel engines, I can plumb, plaster and concrete.

I feel that I have the practical skills and capacity to learn, in order to be able to throw myself at building a completly bespoke Home Server System for myself.

And yeah, if it all fails I'll put the whole thing up for sale in the Members Market :D

In terms of cost, so far, all in all I've spent circa £1000, and based on my current estimates I've got about £200-300 worth of stuff left to buy (Power suppies, HBA Board, SMX2 to PCI-E Single Card Adaptors, and as many old Copper heatpipe and Vaporchamber heatsinks I can get my hands on!

I was costing up 2x 2nd hand 3090's, then 4090's, and I just thought, actually, I can get similar performance with more scaleability for potentially less money.

Yes, it's going to be a proper project.
 
Last edited:
@decto

Thanks for the encouraging and useful reply.

You make a great point on not trying to set it all up at once, I can see that you're right, and the sensible approach.

I'm going to have to learn several new areas of 'computing' that I've never really touched before. The picture I painted above (with regards to what I want the whole 'system' to do (I don't even know how to descibe what I am envisaging, I don't know the lingo!), is more of a vision further down the road, and yeah, I'm taking just taking the first steps into what seems like a complety different domain.

The first iteration of trying to get up and running will absolutely be a seperate NAS utilising some low TB WD Purples I have for my CCTV project. Once I understand the software side, I'll try try to increase the capacity, it was what initally drew me to Unraid with regards to just needing the one large Parity Drive.

I ideally need to source an appropriapte PSU (I want to get something that can comfortably power 4x V100's for future proofing (they are only 16GB HBM2), and then have a seperatate PSU for everything else. The sole reason for choosing this board, and considering the Supermicros, was the scalabilty with regards to having as many PCI-E lanes as possible. Maybe one day, having the equivelant of Gemini Pro (70b model?) running on VRAM alone would be awesome, and fits completely with my desire to stop heamoraghing money little and often, without losing access to something I am thoroughly enjoying
decto said:
Those V100's will take a lot of taming, you're probably best with water cooling blocks and managing the heat in a loop with at least 2 x 240mm radiator than trying to create a custom air cooling solution that properly fits the GPU core and VRM's with the correct tolerances.
I've experimented with a couple of P40's and even with blower fans turned down a little for inference you don't want to be sat next to them.
Click to expand...

If I'm honest, this is the side of things that I'm sort of genuinely excited about, in a non computing sense. I've been learning about fluid dynamics and thermo dynamics due to other interesting (to me) projects that I'm trying to build. I suppose it touches on why maybe I never really 'understood' servers and entriprise computing. It all just looked like boring grey plastic trays with no idea what it actually does. I've realised that actually, although, depending on your notion of what effeciency means, server racks and data centres are incredibly effecient.

However, I'm only one person, I don't need a great big boring plastic server tray, designed to live in a stack, and to function as best as possible while space is at an absolute premium. I have plenty of space to be able to design, test, and hone in on an efficiency based solution for cooling a sever myself in novel and wonderdful ways. Space now being a premium will allow me to get creative with what a Server should actually look like. Yeah, I can't change the hardware itself, but I can change (to a point), the order in which the pieces have been put together. The goals of a Server Rack designer from Dell isn't going to have the same set of constraints or freedom as me, and vice verser.

My first GPU was an x1900xtx 256mb which was released not long after I joined this forum, I still have it, and the original blower fan. How cool woould it be to incorporate that into design somewhere. I don't know, not everything is the best was of doing things because that's just how they're done.

I haven't yet taken a dive into the Home Server side of the forum, I am hoping I might encounter and meet others who have novel ways of making a bespoke server that works primarily for them.

Thanks for information regarding the memory, the plan is to increase the amount, it will just be a case of trying to snipe anything appropriate I see come up for sale.

Once I've got enough hardware I am planning to do some bench testing to make sure the Motherboard and Xeons (one at a time, and then both together) are working correctly. Though, I only currently have my 4070, so I'm unsure how problematic that will or won't be. Once I'm happy with the board, cpu's and ram. It will be a case of getting the tape measure out, opening some 3d software, and getting creative!

Maybe this could be a build thread...expect irregular updates :cry:
 
@Armageus

You've made excellent points, and you're right to question nearly all of my logic.

You've taken time out of your day to read and respond to me, so I think it's only fair to paint as clear a picture as possible, which I am guilty of not doing so far.

I haven't successfully communicated my current hardware landscape and my proposed destination (which I seem to have mistakenly called a 'Home Server').

- 1x system will be as follows, 4-8x 16GB V100's, with as much DDR4 Server Ram as required, this the horspower to enable be to run models on VRAM alone ideally, and learn all about how large language models work.

- 1x 5800x3d Am4 based system, that doesn't really enter this equation as it is my daily driver, however I have trying to figure out if I could use the VRAM of the V100's and the single core CPU performance of the 5800x3d for use in 3d software applications. I suppose, what do you call multiple computers working together?

- 1x 10500T based system which was initially bought to be an NVR running Blue Iris. However I have recently discovered Home Assistant (which is a catalyst for this project), and I am re assessing my totally current hardware and this is currently a very capable system.

- 3x R1505 Ryzen DDR4 based thin clients. Being put to various uses. The brains of a NAS running UNraid, a router running OpenWRT for SQM (I get terrible bufferbloat playing iRacing), Adguard.

3x Atom based DDR3 based thin clients. Same as above, various lightweight uses. I want to set up some wind speed and temperature sensors and I think these could be quite capable for such tasks. I was thinking these could be useful for remote storage backups.

1x AIO 10th gen i3, screen displaying live view CCTV, the system itself running a network node for a Blockchain I want to help secure.

I also then have an assortment of older laptops and other hardware bits and bobs that I think I can repurpose. Currently it's all just destined to be e-waste, but i'd like to try and make use of it.

I wouldn't be posting if I didn't think I would be challenged by people like yourself (and all of you that have responded so far), who are far more knowledgeable and experienced than me. Through this sort of engagement on this forum I have be learning computing over 20 years, just at a very slow pace, and mostly mainstream pc building and overclocking.

Please, rip me apart. Just be nice about it and offer alternatives for me to consider (as you all have been doing)

@mid_gen

I did look at other options, but I couldn't find anything that was going to be as flexible as what I want. I was extremely tempted by an iMac Pro if I'm honest. But, where's the fun in that.

I'll take a look into the services you've mentioned, the only one I've heard of is Tailscale, but I'll have a deep dive this evening..

@Vestas

I agree, if you know of any cheap DDR4 please let me know!

@jigger

Yup, the toothpaste is out of the tube.
 
jigger said:
I’d just point out that if you intend to use the video cards memory as a pool you will run into problems. NVlink will be a nightmare and at best you can run two pairs of cards with a 16x gen3 link before you have to deal with the DMI between chips.

For 8 cards you’ll need two systems with high speed networking. Really you want a Nvidia certified system with NVlink and all the joys that entails.
Click to expand...
Thanks for this, I'll take a look into the NVlink and DMI between cards you mentioned (I don't know what either are currently).

My current viewpoint is that running 4x, and potentially 8x, V100's on the one board should be possible. It that not the case?

What is an Nvidia certified system?
 
Last edited:
Thanks for the few positive comments and also the mostly negative ones.

I appreciate when people use language such as 'nonsense' in such a throwaway manner, helps me to filter out who is worth sharing and engaging with, and who is not :)

I want to play around with 'proper' pc hardware from older servers. When I learnt how to work on vehicles I done so on my 1980's 30 year old first car, the principles and the mechanics of the combustion engine was no different to me 2nd car which was a 20 year newer petrol engine.

Buying a load of ex enterprise hardware far less than the cost of one 2nd hand 4090 by itself to as a basis to learn and apply to he basic theory is worth £1000 to me. That's all there is it it.

If you want to help me along the way as I problem solve, or have useful suggestions or alternatives that fit with my overall ethos of learning, please contribute.

Otherwise, I'll get more wisdom from talking to the dog.

If anyone thinks this project is solely about 'AI', which is a daft name for large language models I might add, they've completely misunderstood the broader pictures.

I'll drop the crux of my total project goal here just to fan the flames even more...I'd like to see how much compute it is possible to run using renewable energy sources.
 
@Vestas

Could you please let me know what the combined memory bandwidth of a 128GB AMD Strix holo is, it sounds absolutely fantastic!
 
Thanks Rroff, it's becoming clearer by the day, but it still appears mumbo jumbo at first glance.

I've spent most of this evening just salivating over the bare V100's, things of absolute beauty!!!

I'd like to get up and running with two gpu's so that I can benchmark and optimise an appropriate cooling solution, run them both at the same time with different prototypes sort of thing. I think I may have to source an OEM heatsink to be fair, just to reverse engineer the dimensions of the mating surfaces, as it looks to me like the mirrored chip in the middle is slightly recessed. I'll be spending a fair bit of time designing a clamping solution that's for sure.
 
Changing the tune. Thanks everyone has read and responded, warts and all. Everyone has encouraged me to go and read things I didn’t know existed last week.

I think part of why I might have rustled a few feathers is that I’ve just done a poor job of separating what I want to learn long term, from what I expect to achieve immediately.

To reset expectations a bit for anyone who is still interested:

Short Term.

- Accumulate enough hardware to start designing an enclosure
- Start a build thread
- Source one OEM heatsink, reverse engineer how it mounts to the V100, and build some prototypes
- Learn fluid and thermal dynamics to an A-Level understanding, and then put a wanted ad up in the members market for old heatsinks
- 3D Print a better testing bench for my worktop
- Check the Motherboard, CPU and Ram posts and is stable with my 4070

Mid term.

- 2× V100's
- 1x Xeon
- 4x DIMMs
- No attempts at VRAM pooling, NVLink, or scaling beyond what the board supports by default.
- Get a stable platform I can benchmark, stress, cool, and understand.

Longer term.

- Completely wide open, other than just trying to learn more about computers and how they work and what they can do beyond play games.

To make it clear, I fully accept;
- V100s, Xeons and DDR4 are legacy hardware in AI terms
- Support/drivers/new models will lock me out sooner, rather than later
-This will never compete with hosted and subscription based models

Where I would appreciate advice and help:

Known HBAs that will be suitable for Unraid with SAS-12Gbps drives
Practical cooling approaches from people who have used enterprise gpus outside of a rack chassis
Any known quirks with the Asus Z10PE-D8 WS, maybe something I should watch out for that isn't obvious in the manual.
BIOS settings worth changing (or not changing) for initial system stability

I mean this in as genuine a way as I can possibly convey; This isn’t about trying to build something that is going to be better than anything I can pay for, it won't be. It’s about understanding hardware, software, and an entire area of computers I have no previous experience of. Rather than moping about because I feel that i've missed the boat with regards to ever having a career in engineering, design or creativity. I'm just going to have a go at home, and see how I get on learning something new that interests me.

If the bloody thing ends up being loud, power hungry and just too old to do anything useful with, so be it – but I’d rather learn that from building it and it failing, than not just trying at all and going to the pub.

And for anyone who is still reading, I will start a build thread once parts are 'on the bench' :)
 
Been a bit sidetracked and doing a lot more research off the back of many of the points you've all made.

@Demon thanks for the HBA recommendation. Re: the Ali blower fan setup, without a GPU shroud I'm not sure it would work. However, I do like the idea of going the watercooled route now. After getting some more accurate measurements of the V100 die, I think it might be the best option.

@Vestas the Strix Holo and DGX Spark are interesting no doubt. It seems they could do everything I need and more. However, for me, buying something ready made that just works, although sensible, doesn't appeal to my inner 'i'll build one in the shed instead' mentality. If I needed to get up and running tomorrow, this would definitely be the route to take, so thanks for all the extra info and linked vids.

re: 2x V100's
This is the problem for me. You just shouldn't use these because not only are they not going to do what you hope, they're going to cost you more money.
Click to expand...

You're right, you're right. 2x 16GB V100's will not do what I hope...*cue evil laugh*

@decto

After your post, I've decided against the PCI-E route. There are too many compromises, and although I enjoy a good 'hack', I think that unless I'm utlising NVlink to get the full bandwidth available to the V100's, there is no point to the project at all. It was your comment regarding training that was the spanner in the works for me, that has made me re-think my entire plan, so thank you :) Those Ali waterblocks don't look like they cool the VRM's to me, also I've never watercooled any PC parts before. However, I do think it's going to be the way forward.

@Rroff

It was those adaptors I had been looking at. But between yourself and decto, i've gone off the idea completely. Re: The SXM2 mounting, yeah it's only because I have calibrated torque wrenches that I feel comfortable going down this route.

oh, and @Vestas

I hope we didn't overcook you ;)
Click to expand...

Not at all, I think you all have helped me see the holes in my original plan, and have sent me off down a more small engineering project pathway, rather than a, lets just buy all these different odds and sods and hope it will work.

My new plan it to just see if I can fabricate a waterblock for one v100 as a small CNC project and, everything else is on hold until then.
 
Yeah, I'm toying around with a couple of different mounting solutions using the 3d printer atm.

I do think water cooling is the right direction to go, as although many CPU heatsinks (Air) on paper should be able to cool a power limited V100, all of that leverage un settles me.

I've never watercooled until using an AIO CPU cooler recently, and I don't think that really counts being a closed system. I'm trying to fully wrap my head around how waterblocks and AIO's work.

Also, if anyone has any idea on what temp the VRM stacks get to on the V100 that would be really useful information.
 
@Cyber-Mav

I would like to utilise and experiment with NV-Link rather than depending upon PCI-E or Unified memory for LLM learning purposes.

What are you (and @Armageus), suggesting exactly?
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom