Advice on homelab/homeprod and containers/setup

[TechMinerUK]

Hey all,

Recently I've been thinking of how I can move my homelab/homeprod setup forward as whilst I revamped it a little while ago and it's not really as "good" as what I wanted it to be.

To cut a long story short, I'm looking to move more stuff back "on-prem" at home where possible and not silly, so local file storage (NextCloud), local picture storage (Immich) and local media (Emby) along with my home automation stuff.

Right now I have the following hardware

• 3x Minisforum MS-01 (64GB RAM, 12600H, 1TB local storage) - Used as a Hyper-V cluster and DR host
• 2x QNAP TS932PX (1x with 4x 12TB disks for media/file storage and 4x 2TB SSDs for VM storage and the other with 2x 24TB HDDs, 2x 6TB HDDs and 4x 1.8TB HDDs for VM storage)
• 1x Beelink (N100, 16GB RAM, 512GB SSD) - Emby physical server
• 1x GMKTec (N100, 16GB RAM, 256GB SSD) - Zigbee2MQTT

Currently in terms of VMs I have:

• 3x Windows Server DCs (1 on each host)
• 1x Windows Server CA
• 1x Windows Server (Veeam Backup & Replication)
• 1x Windows Server (File server)
• 10x Ubuntu (NextCloud, Sonarr/Radarr, UptimeKuma, Postfix/Email notifications, VPN, Ansible [Not yet used], UniFi controller, Public Reverse proxy, Internal reverse proxy, Docker [Not yet used])
• 1x Home Assistant OS

Overall I feel as though the above is overkill which is fine but the idea initially was a lower power setup that didn't lack "grunt" when it came to spinning up a VM but also wasn't excessive to the point of being a pain to manage etc however whilst the MS-01s are great they are hot and feel overkill and not really super for the task at hand which is continuous operation whilst sipping power. Likewise the above setup has various SPOFs such as the 10G switches and NAS'.

I do want to keep the hosts running Hyper-V, not because I don't want to learn but because I can't find a suitable hypervisor that supports FDE in a polished format (I know Proxmox can "work" with a custom Debian build + a Clevis/Tang network unlock setup or using a light SSH to input it on reboot but that’s not ideal for this setup).

I've read that Docker/Podman could potentially be a way to move to smaller hosts e.g. having multiple Docker containers on a VM rather than a VM per role however I've ended up with my head swimming after all the reading on how to securely configure containers with networking and rootless operation (Which seems to break many containers by default)

Ultimately I was potentially thinking of something like so:

• 2-3x N300 Mini PCs (32 or 64GB RAM each with larger SSDs)
• 2x NAS (Similar config as now but one directly hosting SMB rather than iSCSI for storage) or some sort of custom build server using a Jonsbo case
• A 2.5G switch (rather than the 10G ones) for connection between the hosts and storage

In terms of the way forward, for the Windows VMs there isn't really much I can do to streamline these other than bop the third DC on the head and remove the need for a file server by moving that to its own dedicated box (Direct on the NAS or a custom TrueNAS setup) but for the NextCloud, Immich, etc I think I may have potential room to consolidate these in a singular VM using Docker but does anyone have any suggestions on how to securely achieve this (I will profess that I am a complete amateur when it comes to Docker and I'm very paranoid when it comes to security)

Any help, advice, suggestions would be appreciated

 

[BongoHunter]

I would be surprised if a 16 core 32 thread Ryzen with 128GB of RAM couldnt run nearly all of that in a server tbh.

I would go for 1 NAS with a 10GB NIC to the server (no switch). But make the NAS slightly oversized so you could at least run a couple of VMs on it if needed. DIY NAS isn't too challenging these days, just get a case with lots of 5.25 bays so you can add hot swap caddies so disk failures are easy to deal with.

If you need to backup the NAS as well then I would look at streaming it off to an S3 bucket (very cheap if you just writing it there and only pulling back if you have a catastrophic failure with the NAS)

Any other networking just do over 1GB with a cheap low power switch?