Any low hanging fruit left in this 128G OC @ 1900 FCLK?

[Cenuij]

I've pretty much reached the limit of my knowledge about about tuning RAM on a Ryzen system.

For search engines:

Corsair Vengeance RGB Pro CMW128GX4M4D3600C18
Hynix M-die (MJR) H5ANAG8NMJR-TFC

It's 128G Vengeance Pro 3600 C18-22-22-42 kit, it's Hynix parts but at that density there wasn't really a lot of options in stock without importing and praying.



In any case I understand Hynix has the capacity for higher frequency but depending on the die/bin can suck for tighter timings.

I think I can probably thank my SoC silicon, I pretty much just slapped this stuff in - set the SoC voltage to 1.15v, set the FCLK to 1900 and the RAM to 3800. Booted first go, nice.

It posts at 18-20-20-36 with 1.36v but fails hard at C16 at any frequency (this was expected after reading about Hynix M die).

Are there any more easy sub timings I should tighten up? I really have no clue what to look at next, thinking there's probably not much room for improvements now?

 

[opethdisciple]

Import your timings via typhoon burner in to the dram calc and see what it tells you.

 

[MrPils]

If you cant get your primary timings down (tCL, tRDRD, tRP, tRAS) the next most impactful are your controller to dram latencies. These are in two sets, the first and most important are tRRDS, tRRDL and tFAW (Set tFAW to 16 first then tune down the other two timings independently and go down one at a time). The next set are less impactful, these are tWTRS and tWTRL (4/8 should be your target but I suspect 4/12 may be the limit).

You can try reducing tRDRDSCL and tRWRWSCL to reduce same channel latencies, these can usually go as low as 2 but having quad ranks may throw that out the window.

Finally dial back your tRFC as that's way high, keep it a multiple of your tRC value (you should be able to start at 840 and go down 56 at a time) but don't go gung-ho with it as you can cause OS corruption if you try and boot into windows with it too low. Mid to Upper 600's is where Hynix usually lands.

Do take note though that tuning 128gb of memory is something nobody has much experience of. Its hard on the memory controller running 4 x 32gb sticks already so you won't reach the same kind of timings as others have running 2 x 16gb of the same chips. Also thoroughly testing 128gb of memory takes a long time....a very long time. This will not be a quick tune, you should be prepared for a lot of post failures and stress testing that takes many hours at a time.

Also according to your screenshot your SOC voltage is at 1.05v...

 

[Cenuij]

If you cant get your primary timings down (tCL, tRDRD, tRP, tRAS) the next most impactful are your controller to dram latencies. These are in two sets, the first and most important are tRRDS, tRRDL and tFAW (Set tFAW to 16 first then tune down the other two timings independently and go down one at a time). The next set are less impactful, these are tWTRS and tWTRL (4/8 should be your target but I suspect 4/12 may be the limit).

You can try reducing tRDRDSCL and tRWRWSCL to reduce same channel latencies, these can usually go as low as 2 but having quad ranks may throw that out the window.

Finally dial back your tRFC as that's way high, keep it a multiple of your tRC value (you should be able to start at 840 and go down 56 at a time) but don't go gung-ho with it as you can cause OS corruption if you try and boot into windows with it too low. Mid to Upper 600's is where Hynix usually lands.

Thanks great advice, need to run a few burn/stability test now and I think i'll call it a day. Wasn't expecting much out of these sticks but 1900 FCLK was a pleasant surprise.

Do take note though that tuning 128gb of memory is something nobody has much experience of. Its hard on the memory controller running 4 x 32gb sticks already so you won't reach the same kind of timings as others have running 2 x 16gb of the same chips. Also thoroughly testing 128gb of memory takes a long time....a very long time. This will not be a quick tune, you should be prepared for a lot of post failures and stress testing that takes many hours at a time.

Yes! had the same issue last time around with 4x16G

Also according to your screenshot your SOC voltage is at 1.05v...

Ah, so the interesting thing about that is after having to set 1.15v VSOC I noticed there was a fair amount of vdroop, I've applied an amount of VSOC LLC and it seems very happy now at 1.05v - sensors report 1.03v but I'll ged the multimeter onto the provided board contacts just to check the LLC is not spiking.

Here's where I am with no burn/stability tests so far:

 

[MrPils]

Thanks great advice, need to run a few burn/stability test now and I think i'll call it a day. Wasn't expecting much out of these sticks but 1900 FCLK was a pleasant surprise.

Yes! had the same issue last time around with 4x16G

Ah, so the interesting thing about that is after having to set 1.15v VSOC I noticed there was a fair amount of vdroop, I've applied an amount of VSOC LLC and it seems very happy now at 1.05v - sensors report 1.03v but I'll ged the multimeter onto the provided board contacts just to check the LLC is not spiking.

Here's where I am with no burn/stability tests so far:

You've managed a nice drop in latency there, nearly 10% well done . It's latency improvements you'll feel most in usage, so that's where you want to spend most of your efforts really. Its the rarer use cases where bandwidth comes to the fore...though 4 x 32gb is a rare use case already


I've only experience with Samsung B-Die, so the below rules are based on that. It would be interesting to see if they hold true for Hynix too:

tWR can always be set the same as tWCL, lower requires testing.

tRTP can always be set to the same as tWTRL, lower requires testing.

tRDWR and tWRRD are set as a pair on AMD. It looks like you're at a slightly more relaxed setting from the 8/3 it defaults to with Samsung. Most of the time 9/1 seems fastest but I wouldn't try going that low, there's definitely a loose relationship with tCL and tWCL too.

Timings ending in DD refer to latency between Different Dimms. These can usually drop to 6 on Dual Rank B-Die or 1 on Single Rank B-Die

Timings ending in SD refer to latency between ranks on the Same Dimm. These can usually drop to 4 on Dual Rank B-Die or 1 on Single Rank B-Die

Timings ending in SC refer to latency between dimms in the Same Channel - you already have this at 1 though so it can't go lower. Included just for information only

 

[Cenuij]

You've managed a nice drop in latency there, nearly 10% well done . It's latency improvements you'll feel most in usage, so that's where you want to spend most of your efforts really. Its the rarer use cases where bandwidth comes to the fore...though 4 x 32gb is a rare use case already


I've only experience with Samsung B-Die, so the below rules are based on that. It would be interesting to see if they hold true for Hynix too:

tWR can always be set the same as tWCL, lower requires testing.

tRTP can always be set to the same as tWTRL, lower requires testing.

tRDWR and tWRRD are set as a pair on AMD. It looks like you're at a slightly more relaxed setting from the 8/3 it defaults to with Samsung. Most of the time 9/1 seems fastest but I wouldn't try going that low, there's definitely a loose relationship with tCL and tWCL too.

Timings ending in DD refer to latency between Different Dimms. These can usually drop to 6 on Dual Rank B-Die or 1 on Single Rank B-Die

Timings ending in SD refer to latency between ranks on the Same Dimm. These can usually drop to 4 on Dual Rank B-Die or 1 on Single Rank B-Die

Timings ending in SC refer to latency between dimms in the Same Channel - you already have this at 1 though so it can't go lower. Included just for information only

Cool cheers, trying to elimate some instability which I feel has been there since I enabled PBO2, so I've got the current timings locked in a profilie I'll run for a few days. I'll check out these additional timings and report back in a few days.

 

[MrPils]

Cool cheers, trying to elimate some instability which I feel has been there since I enabled PBO2, so I've got the current timings locked in a profilie I'll run for a few days. I'll check out these additional timings and report back in a few days.

You're welcome

If you find instability with PBO enabled look at giving vSOC a small bump (up to 100mv). If that doesn't help leave vSOC at its raised value and try another 50mv VDDG CCD and CLDO VPP voltages (individually and in that order, max 950mv). If that stabilises it then you can slowly reduce vSOC back down until you find your lowest stable voltage. CLDO VPP = VDDG CCD is usually the most stable, they like to be the same.

 

[pc-guy]

You're welcome

If you find instability with PBO enabled look at giving vSOC a small bump (up to 100mv). If that doesn't help leave vSOC at its raised value and try another 50mv VDDG CCD and CLDO VPP voltages (individually and in that order, max 950mv). If that stabilises it then you can slowly reduce vSOC back down until you find your lowest stable voltage. CLDO VPP = VDDG CCD is usually the most stable, they like to be the same.

dude you sound like a bit of a Guru, any idea how I can get my Micron stuff to run at 1900 fclk

tried the timing given in the DRAM calculator - wont post. doesnt seem want to post anything beyond 1800MHz. below is the current timing which has been tightened from the XMP 3600MHz rated.

 

[MrPils]

dude you sound like a bit of a Guru, any idea how I can get my Micron stuff to run at 1900 fclk

tried the timing given in the DRAM calculator - wont post. doesnt seem want to post anything beyond 1800MHz. below is the current timing which has been tightened from the XMP 3600MHz rated.

Set your memory voltage manually, leave the timings all at motherboard auto (not even running XMP) and actually get to 1900/3800 first. Once you are there you can start tuning down the timings afterwards. This is the problem with the calculator - it tells people to run before they can walk .

With Micron sticks you'll ordinarily find you do the most of the work with the secondary timings, tCL will usually go 2 or 3 steps lower than TRCDRD, tRP will usually go one step lower than TRCDRD. This means as you lower tCL and its stable you should then lower the tRCDRW/RW and tRP timings one at a time afterwards so that tCL stays within 3 and tRP remains one lower than the tRCDs.

Next part is maths on your final achieved primary timings - tRAS you shouldnt go below tCL+tRCDRD. tRC dont go below tRAS + tCL, make tRFC a multiple of tRC (be careful TRFC though it can cause weird corruption, Micron will want to be above 500, maybe close to 600).

The next most impactful are your controller to dram latencies. These are in two sets, the first and most important are tRRDS, tRRDL and tFAW (Set tFAW to 16 first then tune down the other two timings independently and go down one at a time - aim for 4/4 but expect to not get there). The next set are less impactful, these are tWTRS and tWTRL (4/8 should be your target but I suspect 4/12 or 5/14 may be the limit). - see where that gets you to and quote me back with your results I'll see if I can offer any more pointers. Almost all of your gain will be primary timings and the dram latency timings, doing the rest will be a large time investment compared to the achieved gain.

 

[pc-guy]

Set your memory voltage manually, leave the timings all at motherboard auto (not even running XMP) and actually get to 1900/3800 first. Once you are there you can start tuning down the timings afterwards. This is the problem with the calculator - it tells people to run before they can walk .

With Micron sticks you'll ordinarily find you do the most of the work with the secondary timings, tCL will usually go 2 or 3 steps lower than TRCDRD, tRP will usually go one step lower than TRCDRD. This means as you lower tCL and its stable you should then lower the tRCDRW/RW and tRP timings one at a time afterwards so that tCL stays within 3 and tRP remains one lower than the tRCDs.

Next part is maths on your final achieved primary timings - tRAS you shouldnt go below tCL+tRCDRD. tRC dont go below tRAS + tCL, make tRFC a multiple of tRC (be careful TRFC though it can cause weird corruption, Micron will want to be above 500, maybe close to 600).

The next most impactful are your controller to dram latencies. These are in two sets, the first and most important are tRRDS, tRRDL and tFAW (Set tFAW to 16 first then tune down the other two timings independently and go down one at a time - aim for 4/4 but expect to not get there). The next set are less impactful, these are tWTRS and tWTRL (4/8 should be your target but I suspect 4/12 or 5/14 may be the limit). - see where that gets you to and quote me back with your results I'll see if I can offer any more pointers. Almost all of your gain will be primary timings and the dram latency timings, doing the rest will be a large time investment compared to the achieved gain.

Thanks for the heads up. Will give it a shot over the weekend when I got spare time.

 

[Cenuij]

So just to complete this after a year or so of running the tuned/stable XMP on these sticks - here's where things ended up after benching and burn. This equates to about a 5-6% performance uplift in e.g. linpack type workloads versus the non tuned XMP profile.

Thanks again @MrPils

 

[Cenuij]

tWR can always be set the same as tWCL, lower requires testing.

tRTP can always be set to the same as tWTRL, lower requires testing.

tRDWR and tWRRD are set as a pair on AMD. It looks like you're at a slightly more relaxed setting from the 8/3 it defaults to with Samsung. Most of the time 9/1 seems fastest but I wouldn't try going that low, there's definitely a loose relationship with tCL and tWCL too.

Timings ending in DD refer to latency between Different Dimms. These can usually drop to 6 on Dual Rank B-Die or 1 on Single Rank B-Die

Timings ending in SD refer to latency between ranks on the Same Dimm. These can usually drop to 4 on Dual Rank B-Die or 1 on Single Rank B-Die

For the timings ending in DD|SD anything lower than the values in the previous post and the system wouldn't post
The other values had an amount of leeway but on longer burn tests I saw a few errors and felt I was starting to see minimum returns from the time being invested in more benching and burn tests

 

[caminashell]

Hello,

Thank you for the information above over the years - it helped me understand some things, and try to stabilise my system a bit more than just PnP.

I recently purchased this kit for my workstation and got quite a few crashes to desktop and to BSOD.

After using Cenuij settings there but at 3600 MT/s for the time being, I seem to get errors when testing using TestMem5 extreme config. Mostly #6, #1, #0.
Interestingly when I completely removed XMP and left everything at defaults. I still got errors in TestMem5. I really don't know if its just normal or what.

I am very unfamiliar with memory, and didn't expect to have so many issues moving from 32GB 3200 (4x8GB) to 128GB 3600 (4x32GB).

Also tried following the "tWR can always be set the same as tWCL" 26>18 etc suggestions but the BIOS ended up resetting itself after several cycles. I guess it didn't like it.

Also tried matching VOC as much as I could but got beeps and freezes on boot to I have left it at 1.200.

What worries me is that with all this instability and crashing etc, I am damaging my OS/files and possible drive?

I really don't overclock and tend to run very safe settings as I prefer stability and reliability.

I have same as you, R9 5900X, Gigabyte Aero 570S, RTX 2080ti.