Just to say that I will give you an update on this but it might take me another week or so. Although I failed miserably with the linux piece on IOMMU I hopefully won't fail so badly with this. To be honest I still have no idea how to do what I needed to do in linux and how anybody likes linux is still totally beyond me and in 15 years I have come across more unix machines than linux boxes in businesses. That being said had I spent a lot of time in linux I might be able to easier solve some of these issues.
Home Lab Threadripper Build Thread.
- Thread starter Vince
- Start date
-
- Tags
- threadripper
Just to say that I will give you an update on this but it might take me another week or so. Although I failed miserably with the linux piece on IOMMU I hopefully won't fail so badly with this. To be honest I still have no idea how to do what I needed to do in linux and how anybody likes linux is still totally beyond me and in 15 years I have come across more unix machines than linux boxes in businesses. That being said had I spent a lot of time in linux I might be able to easier solve some of these issues.
Little update. That fella posted an updated video so I have purchased an Asus board to test which will be with me tomorrow. Very happy in this Instance to be proved wrong and am glad that there are solutions out there that just work, this Taichi had me totally convinced that the problem ran deeper than the hardware but I have been wrong before and it looks like I am again. I went for the prime and if that's no good it will go back and I'll work my way up the stack until I get to the rog.
Now I just have to deal with the lack of stuff on the board I have ordered. It will be here tomorrow so I guess we know if the prime is an option in a day or so. Dunno what to do with my couple of month old Taichi but may just use this as an excuse to build another TR machine for the wife.
Now I just have to deal with the lack of stuff on the board I have ordered. It will be here tomorrow so I guess we know if the prime is an option in a day or so. Dunno what to do with my couple of month old Taichi but may just use this as an excuse to build another TR machine for the wife.
Last edited:
A bit deflating but at least there is a solution, also if you can get away with another build with the missus then why not? Mine would kill me
A bit deflating but at least there is a solution, also if you can get away with another build with the missus then why not? Mine would kill me
There is indeed but what I would find interesting is if the prime has issues that the rog doesn't. I would be amazed if the only two boards that are marketed more towards the professional market are not up to scratch with the ROG when it comes to using them for more professional type applications. I mean this is for professional use more than anything and boards with gaming focus, gaming in the title etc are not really something I want to be buying if i'm doing it through the company, which was the case with the Taichi even if it isn't with the Prime. All the other stuff i'm now having to buy to get my lab up and running 100% is just a tad annoying but I guess if you want to be on the bleeding edge then you have to suck it up.
One issue I do have apart from outrageous prices, which is superficial to say the least is that I don't like the look of the rog at all really and that includes the strix. It just does nothing for me in terms of aesthetics where as the prime seems much more clean. I just hope it works.
Soldato
Subbed.
There's a 4 node RAC 12c oracle build to throw at a suitably endowed ESXi setup on threadripper one day.
BTW - did you ever get further with tuning the ram or still 3066? I'd want it to be home-lab and next to no compromises on the gaming front (tall order, I know
).
There's a 4 node RAC 12c oracle build to throw at a suitably endowed ESXi setup on threadripper one day.
BTW - did you ever get further with tuning the ram or still 3066? I'd want it to be home-lab and next to no compromises on the gaming front (tall order, I know
).Subbed.
There's a 4 node RAC 12c oracle build to throw at a suitably endowed ESXi setup on threadripper one day.
BTW - did you ever get further with tuning the ram or still 3066? I'd want it to be home-lab and next to no compromises on the gaming front (tall order, I know
Ram which is 3466 will do 3200 in the Taichi with cas 14 timings and 1.1 soc voltage, it was stable but felt like I was trying to squeeze every last bit out of it to get it there, it wasn't doing much for temps either so I did go back to 3066. To be honest others have had a more plug and play experience with the same ram in other boards so later today I should be able to compare how the Prime fares. Gotta be honest the last thing I wanted to do was tear down the whole build to replace the board but tonight that is what will happen.
In terms of gaming I can't say you would really be compromising with memory at 3000+, no game out there today really touches a 1950x threadripper cpu for more than about 10% so to some extent you are constrained to its single core performance and while it's not quite up there with the 7700k and 8700k in terms of gaming performance it's certainly no slouch, you do see gains at 3200 there is no doubt but they get smaller and smaller as you go past 3000. I have my 1950X paired with a vega 56 (running 64 air bios, overclocked and undervolted to 1680/1100) and so far for the majority of games I just stick it at 4k with the majority of the eye candy turned up and away she goes. Project cars 2, 4k with all the bells and whistles sits nicely pinned at pretty much 60fps, PUBG I have to reign it back a bit to 2k with medium settings and ultra draw and I sit somewhere around the 80 to 100 frames. I have installed a few games and it does what you would expect it to do, goes more than fast enough with masses left in the tank.
Couple of pics of the swap over:
Bitter sweet for me as the ASRock is the nicer board but the prime installed esxi 6.5 update 1 rollup straight off the bat. I can see all my devices, connect to my nfs datastores and importantly sata works and I can use sata devices as datastores.
Bitter sweet for me as the ASRock is the nicer board but the prime installed esxi 6.5 update 1 rollup straight off the bat. I can see all my devices, connect to my nfs datastores and importantly sata works and I can use sata devices as datastores.
Last edited:
Nope, not a single issue, 100% plug and play.
Glad to hear that !!!! Congratulations Vince we celebrate your aport
did you had a guideline whit your experience?
i´ll keep on eye on the progress ... still i´m waiting the release of EPYC 8C /16T with supermicro mobo for my configuration
Thanks Vince thanks
did you had a guideline whit your experience?
i´ll keep on eye on the progress ... still i´m waiting the release of EPYC 8C /16T with supermicro mobo for my configuration
Thanks Vince thanks
In terms of guidelines I would say, and this is a guess right now until I dig deeper, but if your buying a threadripper board for ESXi look for boards that don't have the full compliment of 8 sata ports. The one common theme seems to be that the boards with 6 ports, so all the asus boards, seem to not have the issue with sata. At first I thought it was down to the number of controllers exposed to esxi as the device ID 0x7901. I don't think however after last nights investigations that this is the case. Both the ASRock Taichi and the Asus Prime expose two devices both with the same ID to esxi, that device is the AMD FCH Sata Controller. In terms of the devices in ESXi I cannot find any difference between the boards so right now I am thinking that there are some firmware/bios issues with at least the ASRock that cannot be resolved with simply poking it.
I can now push forward with further testing. If you guys want I can do a video or two of, well... whatever you want to see. I am happy to set up a VM with GPU pass through running a game or two? Also happy to just load up the platform and put it under a ton of stress if that's what you want to see? Basically from here on out Ill be guided by you good folks in here. Tell me what you want me to test and ill build it up and put the results in here or in a video on youtube. In terms of what I need to achieve I am pretty much there now, my requirements are met. I am not giving up on the Taichi though and think I am going to build up another threadripper machine which will mainly be for the wife but will also serve as a place to continue testing that board as ASRock/VMWare push out updates.
So to clarify, and it pains me to say it because I really liked the ASRock but right now its just a no go for ESXi. What I can say for sure is that the X399-a prime is a very good, relatively low cost candidate when it comes to building a workstation with ESXi in mind.
Last edited:
Glad to see that you eventually got it working!
By the way, may I request that you try the following:
1) Setting up a couple VMs and stressing the ESXi host to 100% CPU usage for some time? (Some motherboards boards are having issues with ESXi PSODing when the CPU is pushed hard)
2) Keeping ESXi running continuously 24/7? (Some motherboards are having issues keeping ESXi running 24/7 and PSOD after a couple hours even when fully idle)
Not a problem, I will build and clone 5 or 6 windows VM's and have each one running an instance of prime or something, I think an hour or two should give us a decent real life scenario while proving stability?
In terms of leaving it running ESXi, ill do an extended test overnight and through tomorrow and see how it goes.
Cheers lads. Feels good that it is now running, so far so good on overall stability, its been on now for a couple of hours. I forgot to leave it overnight but have been starting to build up a little test environment, so far we have a 2012 server, a 2016 server, one windows 10 machine. 8 cores on the 2012 server have been running prime now for about an hour and everything is ticking along nicely.
I'm going to build up another 2008r2 server then I will stress it all some more
These arrived:
Not sure if I actually need them anymore, also they could be faster so not sure if I am going to keep them for this build. Might put them in the ASRock board.
I'm going to build up another 2008r2 server then I will stress it all some more
These arrived:
Not sure if I actually need them anymore, also they could be faster so not sure if I am going to keep them for this build. Might put them in the ASRock board.
Last edited:
Looks like its going pretty nicely thus far! Do keep us updated on how the stress tests went!Cheers lads. Feels good that it is now running, so far so good on overall stability, its been on now for a couple of hours. I forgot to leave it overnight but have been starting to build up a little test environment, so far we have a 2012 server, a 2016 server, one windows 10 machine. 8 cores on the 2012 server have been running prime now for about an hour and everything is ticking along nicely.
I'm going to build up another 2008r2 server then I will stress it all some more
These arrived:
Not sure if I actually need them anymore, also they could be faster so not sure if I am going to keep them for this build. Might put them in the ASRock board.
Stress test for several hours at load and 24 hour (almost) went absolutely fine. The board appears to be stable. I am not sure it is the board for me just yet as I have seen the board in the link below is about to drop, also asus have some of the worst customer service I have ever had the misfortune of having to use:
https://www.gigabyte.com/Motherboard/X399-DESIGNARE-EX-rev-10#kf
Am very tempted to buy that one and give it a go. Feature wise etc it is exactly what I am looking for. Did notice a small issue building a windows 10 machine which I am looking into now. Particularly that it wouldn't accept more than 2 cores but it could be something to do with how I built it so am going back over the win 10 gpu passthrough later on this week.