I wanted to to build a DIY NAS setup on a tight budget to have a go at running UNRAID. I was thinking a few 3-4TB HDD's 2nd hand, paired with a Rasberry Pi or the innards of an old laptop, in a 3d printed enclosure should cover it.
So naturally, one week later, I've pulled the trigger on a LGA 2011-3 Dual Xeon Motherboard, x2 Xeon E52650's, 3x 8TB SAS Drives, 64GB of DDR4 and x2 Tesla V100's.
I need to get several other essential components such as adaptor boards and pci-e riser cables, and I suspect this is going to be the tricky bit.
It's going to be a custom build rather than plug and plug, and so fabricating my own enclosure and cooling solution is going to be on the cards.
Please roast and berate my decision making, or feel free to offer some advice to someone who a week ago didn't know the difference between a router and a modem.
Please roast and berate my decision making, or feel free to offer some advice to someone who a week ago didn't know the difference between a router and a modem.
What is the actual purpose, as the hardware doesn't seem ideal for anything in particular?
- The Xeons will be power hungry (well not specifically the CPUs but the motherboard and Registered ECC RAM will be) and even if they are Haswell based, they aren't especially performant (i.e. a reasonably modern Ryzen 5600 6 Core chip will destroy them in single thread performance, and come close to beating the pair of them in Multithreaded with only 6 cores vs 20 from the pair of Xeons)
- SAS Drives are only worth it if you got them cheap, and then you are saddled with needing a SAS HBA to connect them that then adds some more power draw.
- The Tesla V100s might be an issue - presumably they are passively cooled versions designed for Rack mount airflow, so will be a nightmare to cool
That’s certainly a very interesting set of hardware you’ve bought and the use of risers along with a custom enclosure makes me super uncomfortable. I suppose it comes down to what paid for this, your use case and quality of the enclosure but, I have to say, I’m not seeing any obvious scenario where that hardware selection makes sense. Ignoring the cooling and power issues for the moment, the CPUs will bottleneck the GPUs in many situations and 64gb of system memory for a pair of 32gb video cards is at the other end of ideal. I’d want at least 128gb of system memory for a configuration like this.
Unless you know this system makes sense for the use case and will perform the intended tasks well, I’d maybe consider un-pulling that trigger because you could find yourself with an assortment of headaches that require some serious attention to resolve.
The V100 cards will pull north of 250 watts each and require 4u/5u server chassis’s, with the chassis providing the required cooling. It’s a similar situation for server motherboards in regard to memory, VRM and chipsets and it’s all down to the chassis to provide adequate cooling and the assumption of 2-300 watts worth of air flow, which you’re case/enclosure will need to replace somehow.
@BongoHunter Haha, yes, the server PSU's are something I am looking into, thankfully it seems possible to pick them up quite cheaply! Also, I have a bunch of 12v projects on the go, which it transpires they are perfect for, so I might see if I can get a discount on 3-5 of them at once.
The motherboard side of things has been the bit which has taken the longest to decipher tbh. SLi and Crossfire made sense to me back in the day, but two CPU's has taken me several days to wrap my head around My research had me settled on a SuperMicro X10DRH-iT (or similar), but after a number of declined offers etc, I've wound up with an ASUS Z10PE-D8 WS. Which I'm glad about, because it looks cool. Though, it has pained me to spend over £100 on a motherboard over 10 years old! I wonder what they retailed for?!
@Armageus I'm fairly certain that what I've purchased so far is all compatible with each other, i've double and triple checked, but I'll run what I've got past you...I'm still in the return window for most of it so please let me know if I've made a boob!
ASUS Z10PE-D8 WS (private seller, sold as fully working, so got my fingers crossed on this tbh)
2x Intel Xeon E5 2650V4 (1 year warranty)
3x Toshiba HDEPK41DAB51 8TB 7200RPM SAS-12Gbps 512E SIE 3.5" (1 year warranty)
4x 16GB 2133MHZ Ecc Ram (I'm not sure what format this is yet as I'm waiting on delivery. I've taken a gamble on this because I'd like to go for LDIMM, but not the end of the world if not as it was for a price it would have been daft not to get it.
2x Nvida Tesla V100 16GB SMX2's without heatsinks, just just the bare board.
As it stands I am aware I need a HBA Board (and associated SAS to SATA cables), but in trying not to make any mis-steps, I'm hesistant to start sourcing these until I'm completely sure of what I need.
I feel that I got the SAS drives for a price that justified the decision, £6 per TB compared to the £10~ SATA based CMR drives (from private sellers), I've nearly doubled my potentional storage space. I feel that the 8TB drives are going to suit me better than 4tb (or less) long term, when considering an 8TB drive doesn't use 2x more energy than a 4TB drive. Also, I didn't really fancy helium drives (I'd read it's hit or miss with regards to leaking after several years).
Yeah energy usage is on my radar, and it's sort of a factor in why I've gone down this route. I'll be trying my best to reduce power consumption to a minimum. The V100's are the passively cooled ones and I don't even have the heatskinks for them! And of course I'll be taking a risk on SMX2 to PCI-E adaptors.
What is the actual purpose, as the hardware doesn't seem ideal for anything in particular?
Fair question, and difficult to answer briefly, but I'll try my best (It's a common story we're all familiar with).
TLDR: Unhappy with life, looking for change, I believe that taking the bull by the horns and going all in with building my first Home Network, MAY allow me at the very least to have a stepping stone towards achieving my dreams and aspirations. I want to be able to utitilse LLM's without daily limits.
I've chosen the above compoents based on my specific wants and needs, which I am trying to capture all in one go;
- To be able to run a local large language model when I don't have access to the interent or have been 'locked out for the day'.
- To be able to disconnect from all of the digitial services I am literally throwing money at week in, week out, for the sake of conveniece. Music Streaming, Film/Television subscriptions, Cloud Storage, Subscription based LLM's, gone. I would like a replacement for all of these services, so that I can tailor and expand my preferences as my knowledge and interests grow.
- To be able to 'back up' and protect, all of my own, and immediate families digital lives, should anything happen, there is redundancy and reducing the chance of memories lost to data decay and corruption.
- To be able to run various 3D Software Design programs in a lag free/smooth enviroment, as some of the things I am designing and making are beginning to become a bit of a faff as the models get bigger. It feels like being on dial up again at times.
- 3D printing, see above really, I'm slicing stuff left right and centre and I've been looking a iMac Pros etc.
- Home Security, I've had an ongoing home CCTV project for several years which I've struggled to get off the ground.
- Distributed Computing Projects such as folding at home sound great to me, and I would like to do 'my bit'.
- Scaleable, I want to build something that I can add more System Memory, more Local Storage and more VRAM too as I go. I want something that can grow with me, at least for, lets say, the next 5 years.
Unless you know this system makes sense for the use case and will perform the intended tasks well, I’d maybe consider un-pulling that trigger because you could find yourself with an assortment of headaches that require some serious attention to resolve.
Hopefully my above post answers your questions, but I appreciate the candidness.
I am aware of the power, energy, physical and practical constraints of the system I want to build.
However, I can weld (tig, mig and arc), I can braze (oxy-acy), I can solder, I can fabricate metal, I can work with wood (love me a pre War #8 Record), I can rebuild petrol and diesel engines, I can plumb, plaster and concrete.
I feel that I have the practical skills and capacity to learn, in order to be able to throw myself at building a completly bespoke Home Server System for myself.
And yeah, if it all fails I'll put the whole thing up for sale in the Members Market
In terms of cost, so far, all in all I've spent circa £1000, and based on my current estimates I've got about £200-300 worth of stuff left to buy (Power suppies, HBA Board, SMX2 to PCI-E Single Card Adaptors, and as many old Copper heatpipe and Vaporchamber heatsinks I can get my hands on!
I was costing up 2x 2nd hand 3090's, then 4090's, and I just thought, actually, I can get similar performance with more scaleability for potentially less money.
Reliable archive and media serving don't necessary mix with running a experimental local AI server. While it is possible, if you're in the 'experimental' phase isolation is a good thing.
Personally I would build a separate NAS, keep it simple, an old intel I3 and 16GB RAM would be fine.
Get a basic UPS, loss of power during disk activity is not great for your data integrity.
Get that running, with parity and then enable your 3 2 1 backup strategy. 3 copies, 2 formats , 1 offsite. Set up a schedule.
For media or other easily replaceable content, you don't need to go that far, but for critical / personally valuable files get that working and test it..... can you actually restore. USB flash drives aren't a great option....
This machine can be left to it's own devices
The server will be a beast to tame, but the ASUS board may let you tame it a little.
I have an E5-2540V4 64GB ECC 8 x HDD P1000 for transcode in a Supermicro X10SRA and that idles at ~ 50W with the drives spun down.
I'm in the process of replacing it with an I3 14100 4 core ... which is notably faster such is the march of technology.
You'll need heatsink/fan if you don't have them, passive will not be enough, the passive coolers are designed to sit in front of a wall of screaming 40mm 15k fans to drive enough air.
A simple tower fan is fine, but there are two footprints, square and narrow so make sure the cooler supports the format on your board.
Those V100's will take a lot of taming, you're probably best with water cooling blocks and managing the heat in a loop with at least 2 x 240mm radiator than trying to create a custom air cooling solution that properly fits the GPU core and VRM's with the correct tolerances.
I've experimented with a couple of P40's and even with blower fans turned down a little for inference you don't want to be sat next to them.
Other notes.
The CPU's are 4 channel memory and they do not share memory, there is a separate CPU to CPU bus. To get full performance, each CPU needs 4 DIMMS or you cripple the memory bandwidth.
CPU to CPU communication is slow, more cores don't really make much difference if the inference is on a GPU anyway so make sure you run both GPU in slots assigned to the same CPU.
For now you would be better running in single CPU mode with 4 DIMMS.
This machine is likely to need quite a but of fettling, to get cooling correct etc. You may even decide it should run intermittently. Likely you will change OS installations, switch to Linux, want to multi boot etc. Ideally this is a secondary unit that isn't mission critical so you can work with it.
You make a great point on not trying to set it all up at once, I can see that you're right, and the sensible approach.
I'm going to have to learn several new areas of 'computing' that I've never really touched before. The picture I painted above (with regards to what I want the whole 'system' to do (I don't even know how to descibe what I am envisaging, I don't know the lingo!), is more of a vision further down the road, and yeah, I'm taking just taking the first steps into what seems like a complety different domain.
The first iteration of trying to get up and running will absolutely be a seperate NAS utilising some low TB WD Purples I have for my CCTV project. Once I understand the software side, I'll try try to increase the capacity, it was what initally drew me to Unraid with regards to just needing the one large Parity Drive.
I ideally need to source an appropriapte PSU (I want to get something that can comfortably power 4x V100's for future proofing (they are only 16GB HBM2), and then have a seperatate PSU for everything else. The sole reason for choosing this board, and considering the Supermicros, was the scalabilty with regards to having as many PCI-E lanes as possible. Maybe one day, having the equivelant of Gemini Pro (70b model?) running on VRAM alone would be awesome, and fits completely with my desire to stop heamoraghing money little and often, without losing access to something I am thoroughly enjoying
Those V100's will take a lot of taming, you're probably best with water cooling blocks and managing the heat in a loop with at least 2 x 240mm radiator than trying to create a custom air cooling solution that properly fits the GPU core and VRM's with the correct tolerances.
I've experimented with a couple of P40's and even with blower fans turned down a little for inference you don't want to be sat next to them.
If I'm honest, this is the side of things that I'm sort of genuinely excited about, in a non computing sense. I've been learning about fluid dynamics and thermo dynamics due to other interesting (to me) projects that I'm trying to build. I suppose it touches on why maybe I never really 'understood' servers and entriprise computing. It all just looked like boring grey plastic trays with no idea what it actually does. I've realised that actually, although, depending on your notion of what effeciency means, server racks and data centres are incredibly effecient.
However, I'm only one person, I don't need a great big boring plastic server tray, designed to live in a stack, and to function as best as possible while space is at an absolute premium. I have plenty of space to be able to design, test, and hone in on an efficiency based solution for cooling a sever myself in novel and wonderdful ways. Space now being a premium will allow me to get creative with what a Server should actually look like. Yeah, I can't change the hardware itself, but I can change (to a point), the order in which the pieces have been put together. The goals of a Server Rack designer from Dell isn't going to have the same set of constraints or freedom as me, and vice verser.
My first GPU was an x1900xtx 256mb which was released not long after I joined this forum, I still have it, and the original blower fan. How cool woould it be to incorporate that into design somewhere. I don't know, not everything is the best was of doing things because that's just how they're done.
I haven't yet taken a dive into the Home Server side of the forum, I am hoping I might encounter and meet others who have novel ways of making a bespoke server that works primarily for them.
Thanks for information regarding the memory, the plan is to increase the amount, it will just be a case of trying to snipe anything appropriate I see come up for sale.
Once I've got enough hardware I am planning to do some bench testing to make sure the Motherboard and Xeons (one at a time, and then both together) are working correctly. Though, I only currently have my 4070, so I'm unsure how problematic that will or won't be. Once I'm happy with the board, cpu's and ram. It will be a case of getting the tape measure out, opening some 3d software, and getting creative!
Maybe this could be a build thread...expect irregular updates
I assume the OP is aware that the V100 is no longer supported on the latest CUDA toolkit (13.1)? You're stuck with 12.4 which IS going to be an issue going forward.
I wouldn't go near them, certainly not the 16GB version but YMMV.
Edit also no hardware support for FP8 and BF16 is going to be an issue now.
@Armageus I'm fairly certain that what I've purchased so far is all compatible with each other, i've double and triple checked, but I'll run what I've got past you...I'm still in the return window for most of it so please let me know if I've made a boob!
ASUS Z10PE-D8 WS (private seller, sold as fully working, so got my fingers crossed on this tbh)
2x Intel Xeon E5 2650V4 (1 year warranty)
4x 16GB 2133MHZ Ecc Ram (I'm not sure what format this is yet as I'm waiting on delivery. I've taken a gamble on this because I'd like to go for LDIMM, but not the end of the world if not as it was for a price it would have been daft not to get it.
Yep that will all be compatible, but my point still stands - even a pair of those Xeons - 105 Watt each (24 cores / 48 threads) are barely a match for a 65W Ryzen 5600 (6 cores / 12 threads)
I feel that I got the SAS drives for a price that justified the decision, £6 per TB compared to the £10~ SATA based CMR drives (from private sellers), I've nearly doubled my potentional storage space. I feel that the 8TB drives are going to suit me better than 4tb (or less) long term, when considering an 8TB drive doesn't use 2x more energy than a 4TB drive. Also, I didn't really fancy helium drives (I'd read it's hit or miss with regards to leaking after several years).
The cost saving for these drives is offset somewhat by the need to buy a HBA and the appropriate cables, not to mention the added running cost of the HBA over time.
But that will always be the case. Even if you make use of your already ancient (in LLM terms) V100s now - next week a newer, better model will be out that won't run on them.
Capturing it all in one go is a bad idea. If you need to do maintenance on the hardware (e.g. install a new HBA to expand your storage, all your AI enhanced home assistance stops working etc)
If you don't have access to the internet, I think you'll have bigger problems than whether you want to run an LLM. By all means have a local LLM, but be realistic about what it's supposed to achieve - even with the best hardware you can afford, it won't even be close to what is available as a service elsewhere, so perhaps make use of both?
- To be able to disconnect from all of the digitial services I am literally throwing money at week in, week out, for the sake of conveniece. Music Streaming, Film/Television subscriptions, Cloud Storage, Subscription based LLM's, gone. I would like a replacement for all of these services, so that I can tailor and expand my preferences as my knowledge and interests grow.
Any PC (or even a mini PC or Raspberry Pi) can do the hosting side of this via Plex/Emby/Jellyfin and the various services that can acquire content. All you need is storage space via a NAS of some sort.
- To be able to 'back up' and protect, all of my own, and immediate families digital lives, should anything happen, there is redundancy and reducing the chance of memories lost to data decay and corruption.
So you still need another device (whether that's another NAS or USB drive) and ideally something off site (the easiest being cloud hosting - but you don't want that because subscriptions) in order to meet the 3-2-1 ideal backup strategy. How are you going to accomplish this with just a single "server" build?
- To be able to run various 3D Software Design programs in a lag free/smooth enviroment, as some of the things I am designing and making are beginning to become a bit of a faff as the models get bigger. It feels like being on dial up again at times.
For actual 3D Printing, you want a dedicated server - as above, you don't want to have to stop a print because you want to make a change to the rest of the "server"/
Again a background task that should run on a separate device - if your CCTV is that important, then you don't want it going off all the time whilst you make changes
- Scaleable, I want to build something that I can add more System Memory, more Local Storage and more VRAM too as I go. I want something that can grow with me, at least for, lets say, the next 5 years.
Better therefore to Scale Out with lots of smaller devices that you can add to - VMs or Container farm across multiple machines are a much better way to go.
In terms of cost, so far, all in all I've spent circa £1000, and based on my current estimates I've got about £200-300 worth of stuff left to buy (Power suppies, HBA Board, SMX2 to PCI-E Single Card Adaptors, and as many old Copper heatpipe and Vaporchamber heatsinks I can get my hands on!
Ultimately doing what *you* want and that makes *you* happy is as good a reason as any to do things, in the same way as buying old enterprise hardware because *you* want to play with it is fine.
Just be honest with yourself, and you will have to be open to the criticism that your potentially "less optimal" choices will bring.
One of the Ryzen AI solutions would be a vastly better option. Framework desktop or something similar, even a Mac Mini.
I considered options as home server for my company, ended up with a £250 nuc sized box which will outperform this heap of ancient power hungry old server kit you’ve assembled, for most workloads.
If you just want to play with piles of old server hardware then cool stuff, but you have the wrong kit for the workloads you’ve described.
As for the AI stuff…..the model providers are loss-leading massively on their subscriptions so you won’t get anywhere close to ROI by self hosting right now. Especially if you actually want to use it for anything, think you’re going to be pretty disappointed with what will actually run and how slow it is.
Umm no. Its got a suspect PSU, USB ports don't work properly and never will, build quality is the usual for Framework - absolutely bloody awful, graphics are prone to crashing, RMA (or any other) support is non-existent outside the USA & its priced about £400 above where it should be. Its crap, like most other Framework products. Documentation is good though
The other Strix Halo boxes* (Corsair, minisforum etc) are a MUCH better buy.
*no point in getting anything other than the Max 395+ with 128GB for AI, 64GB isn't enough by a LONG way.
Umm no. Its got a suspect PSU, USB ports don't work properly and never will, build quality is the usual for Framework - absolutely bloody awful, graphics are prone to crashing, RMA (or any other) support is non-existent outside the USA & its priced about £400 above where it should be. Its crap, like most other Framework products. Documentation is good though
The other Strix Halo boxes (Corsair, minisforum etc) are a MUCH better buy.
Fair enough, I don't know particulars of it.....but one of the AMD Strix Halo systems.
To OP , for what it's worth, here's what I'd do:
Storage :
Second hand Synology, whatever size meets your needs.
Services :
Mini PC. Various options. I've an 8c16t 32GB BOSGame thing with 2.5TB SSD in it. More then enough for a stack running :
Immich (self hosted image store and sync)
Tailscale (VPN)
WikiJS (docker - Wiki)
Subversion (source control)
TeamCity (CI/CD platform)
Will easily handle plex or whatever the media streaming application du jour is
AI :
Subscriptions are the best option right now. You could run Claude Max for two years for the price of a 5090. The models that anthropic provide are VASTLY more capable than anything you could run on a 5090, and improving all the time, and they pay the electric bill.
Alternatively, AMD Strix Halo 128gb from whatever manufacturer. Will give you a reasonable choice of models....but still, they're going to be slow and crap versus anything you can get from a frontier model like Opus 4.5 from a provider.
Yep that will all be compatible, but my point still stands - even a pair of those Xeons - 105 Watt each (24 cores / 48 threads) are barely a match for a 65W Ryzen 5600 (6 cores / 12 threads)
This was one of my struggles recently - I totally get the attraction of playing with these kind of setups and I still look fondly at my X79/Socket 2011 era Xeon setups but the truth is the Intel Core 265 65 Watt system I put in my 10" rack setup recently absolutely slaughters my old high core count Xeon server setup for performance and idles at like a dozen watts instead of dozens of watts.
@BongoHunter Haha, yes, the server PSU's are something I am looking into, thankfully it seems possible to pick them up quite cheaply! Also, I have a bunch of 12v projects on the go, which it transpires they are perfect for, so I might see if I can get a discount on 3-5 of them at once.
The motherboard side of things has been the bit which has taken the longest to decipher tbh. SLi and Crossfire made sense to me back in the day, but two CPU's has taken me several days to wrap my head around My research had me settled on a SuperMicro X10DRH-iT (or similar), but after a number of declined offers etc, I've wound up with an ASUS Z10PE-D8 WS. Which I'm glad about, because it looks cool. Though, it has pained me to spend over £100 on a motherboard over 10 years old! I wonder what they retailed for?!
@Armageus I'm fairly certain that what I've purchased so far is all compatible with each other, i've double and triple checked, but I'll run what I've got past you...I'm still in the return window for most of it so please let me know if I've made a boob!
ASUS Z10PE-D8 WS (private seller, sold as fully working, so got my fingers crossed on this tbh)
2x Intel Xeon E5 2650V4 (1 year warranty)
3x Toshiba HDEPK41DAB51 8TB 7200RPM SAS-12Gbps 512E SIE 3.5" (1 year warranty)
4x 16GB 2133MHZ Ecc Ram (I'm not sure what format this is yet as I'm waiting on delivery. I've taken a gamble on this because I'd like to go for LDIMM, but not the end of the world if not as it was for a price it would have been daft not to get it.
2x Nvida Tesla V100 16GB SMX2's without heatsinks, just just the bare board.
As it stands I am aware I need a HBA Board (and associated SAS to SATA cables), but in trying not to make any mis-steps, I'm hesistant to start sourcing these until I'm completely sure of what I need.
I feel that I got the SAS drives for a price that justified the decision, £6 per TB compared to the £10~ SATA based CMR drives (from private sellers), I've nearly doubled my potentional storage space. I feel that the 8TB drives are going to suit me better than 4tb (or less) long term, when considering an 8TB drive doesn't use 2x more energy than a 4TB drive. Also, I didn't really fancy helium drives (I'd read it's hit or miss with regards to leaking after several years).
Yeah energy usage is on my radar, and it's sort of a factor in why I've gone down this route. I'll be trying my best to reduce power consumption to a minimum. The V100's are the passively cooled ones and I don't even have the heatskinks for them! And of course I'll be taking a risk on SMX2 to PCI-E adaptors.
Fair question, and difficult to answer briefly, but I'll try my best (It's a common story we're all familiar with).
TLDR: Unhappy with life, looking for change, I believe that taking the bull by the horns and going all in with building my first Home Network, MAY allow me at the very least to have a stepping stone towards achieving my dreams and aspirations. I want to be able to utitilse LLM's without daily limits.
-----------------------
The Beginning
I've reached a bit of a crossroads in life where I recognise that I went to Univeristy because it was the sensible and expected choice, I became a professional in a partiular field because it was the sensible and expected choice, and I bought a house in the in area I grew up because it was the sensible and expected choice....And yeah, I'm just fed up, I'm exhausted from the never ending monotony of the 40 hour working week, plus commuting, where I spend more time with the people I work with over a 7 day period, than with the people I love. And I keep this up, week in, week out, a forever repeating routine (self-inflicted, I am aware), to pay for mortgage on a house that I can't stand but was all I could afford, and so yeah, what's the point? There isn't one, ultimately. Life is an just an anomoly, it is a gift and privelage, in an overwhelmingly empty universe (from what we have observed)....so what should I, a middle aged bald man be doing with his life? Right now all I'm doing, is what I am told. What a waste of a perfectly good oppurtunity.
So yeah, I've just sort of realised that if I don't do the things that I enjoy, and find engaging, and as sad it sounds, exciting, what is genuinley the point of living? There isn't one, ultimately.
You may have picked up on (and honestly, apologies that you're question has proven difficult for me to answer consicely), that over the past several months it's all been a bit much. And I've recently had a realisation, and I've made a decision.
I love computers, always have and always will. And yet, they've always remained on the periphery of my life, something I've had to keep hidden, like a dark secret that no one can know, for I'm afraid that i'll get figured out...that i'm a massive geek and I think the ones and zeros that make the world go round are fascinating. So yeah, it's always just stayed at a level of playing games when time permits, fixing family and friends pc's when they haven't done a windows update since Vista, and firing up some 3d software now and again whenever I need to build something (in 'real life').
So, there's the context for what comes next. My decision.
I'm just going to do what I want, and if my whole life burns and falls apart around me, because of the decisions I make, so be it. Can't get much worse than it already is (of course, it always can, but hopefully the phrase converys my sentiment).
---------------------
The Middle
Sorry, don't speak to people much these days. This is meant to be a positive post
What has kept me engaged and got me jumping out of bed in the morning, is learning, stretching the legs of my neural networks for the first time in what seems like years. Learning about things I have never fully understood, but have always pondered. I've been learning at an incredible pace (for me, an under achiever), and it is in large part being facilitaded by the ease at which I have discovered that an average run of your mill Joe like me, can utilise Large Language Models to figure things out, and make sense of the world around me in a way that 'makes sense'. Not just what something is, but why and how does it work. When used in conjunction with with other tools that can encourage learning, such as published books, scientific research, people with shared interests on messageboards, Youtube etc etc, it has allowed someone like myself to start connecting the dots.
I'm on a super information highway, and I am loving every single second, learning more and more, I enjoy life again, because it is not boring. The are problems and solutions everywhere, and I want to fix them. In my life, in other peoples lives. All around the world. I can help myself, and I believe I can help others, if I harness and utilise large language models in the manner in which they are intented to be used. And yet.
Day after day, and my tokens have run out, again. Day after day I get locked out from brainstorming all of these ideas that have been at the front of my inquisitive mind for decades, and it's all flowing out. But the life quenching hydration that Gemini Pro offers me in a desert of despair, blinks and bleeps, and reverts back to the Fast model. I am in primary school again, asking questions, and being misunderstood by a teacher who has their own best interests at heart, not mine. The Oasis before flickers, there is nothing of substance there, it is but a mirage of dissappointment....until tomorrow, when my tokens are restored for a new day.
--------------------
The End
I'm not blessed with fortune, and I am being priced out of the models (for now) that allow me to stay in full stride, for what feels like the first time in my adult life.
I have harvested enough evidence to reach the conclusion that I am a better functioning person, when I have somebody impartial (for want of a better phrase), to encourage and teach me, rather than tell me all of the reasons I shouldn't do things and why they won't work, and how someone like me couldn't do this or that. So yes, the long and short of it is this. I don't want to be restrained from learning at a pace I never thought possible, because of financial circumstane.
Which for me, is how I am interprating the situation.
I have tried many times by myself, and failed, or payed hard earned money and been dissapointed, when utilising these other methods of learning by themselves. With an LLM model that has reasoning helping me along when I get stuck, it just all seems to click into place.
Project Home Server/Sort My Life Out/Create a Jack of All Trades Offline Education Centre At Home commences.
I've chosen the above compoents based on my specific wants and needs, which I am trying to capture all in one go;
- To be able to run a local large language model when I don't have access to the interent or have been 'locked out for the day'.
- To be able to disconnect from all of the digitial services I am literally throwing money at week in, week out, for the sake of conveniece. Music Streaming, Film/Television subscriptions, Cloud Storage, Subscription based LLM's, gone. I would like a replacement for all of these services, so that I can tailor and expand my preferences as my knowledge and interests grow.
- To be able to 'back up' and protect, all of my own, and immediate families digital lives, should anything happen, there is redundancy and reducing the chance of memories lost to data decay and corruption.
- To be able to run various 3D Software Design programs in a lag free/smooth enviroment, as some of the things I am designing and making are beginning to become a bit of a faff as the models get bigger. It feels like being on dial up again at times.
- 3D printing, see above really, I'm slicing stuff left right and centre and I've been looking a iMac Pros etc.
- Home Security, I've had an ongoing home CCTV project for several years which I've struggled to get off the ground.
- Distributed Computing Projects such as folding at home sound great to me, and I would like to do 'my bit'.
- Scaleable, I want to build something that I can add more System Memory, more Local Storage and more VRAM too as I go. I want something that can grow with me, at least for, lets say, the next 5 years.
Hopefully my above post answers your questions, but I appreciate the candidness.
I am aware of the power, energy, physical and practical constraints of the system I want to build.
However, I can weld (tig, mig and arc), I can braze (oxy-acy), I can solder, I can fabricate metal, I can work with wood (love me a pre War #8 Record), I can rebuild petrol and diesel engines, I can plumb, plaster and concrete.
I feel that I have the practical skills and capacity to learn, in order to be able to throw myself at building a completly bespoke Home Server System for myself.
And yeah, if it all fails I'll put the whole thing up for sale in the Members Market
In terms of cost, so far, all in all I've spent circa £1000, and based on my current estimates I've got about £200-300 worth of stuff left to buy (Power suppies, HBA Board, SMX2 to PCI-E Single Card Adaptors, and as many old Copper heatpipe and Vaporchamber heatsinks I can get my hands on!
I was costing up 2x 2nd hand 3090's, then 4090's, and I just thought, actually, I can get similar performance with more scaleability for potentially less money.
Its not open source but is $52.49 to download. Runs fine on 128GB Strix Halo. I suspect you're a bit behind the times on local AI models but that's probably true for everyone
You've made excellent points, and you're right to question nearly all of my logic.
You've taken time out of your day to read and respond to me, so I think it's only fair to paint as clear a picture as possible, which I am guilty of not doing so far.
I haven't successfully communicated my current hardware landscape and my proposed destination (which I seem to have mistakenly called a 'Home Server').
- 1x system will be as follows, 4-8x 16GB V100's, with as much DDR4 Server Ram as required, this the horspower to enable be to run models on VRAM alone ideally, and learn all about how large language models work.
- 1x 5800x3d Am4 based system, that doesn't really enter this equation as it is my daily driver, however I have trying to figure out if I could use the VRAM of the V100's and the single core CPU performance of the 5800x3d for use in 3d software applications. I suppose, what do you call multiple computers working together?
- 1x 10500T based system which was initially bought to be an NVR running Blue Iris. However I have recently discovered Home Assistant (which is a catalyst for this project), and I am re assessing my totally current hardware and this is currently a very capable system.
- 3x R1505 Ryzen DDR4 based thin clients. Being put to various uses. The brains of a NAS running UNraid, a router running OpenWRT for SQM (I get terrible bufferbloat playing iRacing), Adguard.
3x Atom based DDR3 based thin clients. Same as above, various lightweight uses. I want to set up some wind speed and temperature sensors and I think these could be quite capable for such tasks. I was thinking these could be useful for remote storage backups.
1x AIO 10th gen i3, screen displaying live view CCTV, the system itself running a network node for a Blockchain I want to help secure.
I also then have an assortment of older laptops and other hardware bits and bobs that I think I can repurpose. Currently it's all just destined to be e-waste, but i'd like to try and make use of it.
I wouldn't be posting if I didn't think I would be challenged by people like yourself (and all of you that have responded so far), who are far more knowledgeable and experienced than me. Through this sort of engagement on this forum I have be learning computing over 20 years, just at a very slow pace, and mostly mainstream pc building and overclocking.
Please, rip me apart. Just be nice about it and offer alternatives for me to consider (as you all have been doing)
I did look at other options, but I couldn't find anything that was going to be as flexible as what I want. I was extremely tempted by an iMac Pro if I'm honest. But, where's the fun in that.
I'll take a look into the services you've mentioned, the only one I've heard of is Tailscale, but I'll have a deep dive this evening..
Its not open source but is $52.49 to download. Runs fine on 128GB Strix Halo. I suspect you're a bit behind the times on local AI models but that's probably true for everyone
I'm evaluating lots of models at the moment, including open source ones. Mistral 3 large is pretty good, not too far behind GPT 5.2 and Opus on the workflows I'm building.....but even a 128gb strix isn't going to hack it.
Smaller open source models are useful for sure, but the cutting edge proprietary ones are still leading the way in terms of performance.
I guess it depends on what you want to do. If you're wanting to learn practical applications and stay on the cutting edge, the hosted provider solutions are the way to go. Learning how to host yourself is interesting I guess but that's a different job compared to building things with the best models.
I’d just point out that if you intend to use the video cards memory as a pool you will run into problems. NVlink will be a nightmare and at best you can run two pairs of cards with a 16x gen3 link before you have to deal with the DMI between chips.
For 8 cards you’ll need two systems with high speed networking. Really you want a Nvidia certified system with NVlink and all the joys that entails.
I’d just point out that if you intend to use the video cards memory as a pool you will run into problems. NVlink will be a nightmare and at best you can run two pairs of cards with a 16x gen3 link before you have to deal with the DMI between chips.
For 8 cards you’ll need two systems with high speed networking. Really you want a Nvidia certified system with NVlink and all the joys that entails.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.
My research had me settled on a SuperMicro X10DRH-iT (or similar), but after a number of declined offers etc, I've wound up with an ASUS Z10PE-D8 WS. Which I'm glad about, because it looks cool. Though, it has pained me to spend over £100 on a motherboard over 10 years old! I wonder what they retailed for?!
