Expected Temperatures of a Watercooled 4090FE?

[Mysterae_]

I've noticed that the temperatures of my watercooled 4090FE (Watercool Heatkiller block) have been increasing lately to higher than I would expect for a watercooled part:

Even the coolant which goes to an external MoRa3 to be cooled is exceeding 42C (not shown above) which I think is also too high but is an indicator that it is carrying heat.

Compare that to before, playing the same game and settings, even with a higher core frequency and power draw. The image below was before I monitored the water temperature but I know it wasn't exceeding 40C as the D5 Next overtemp alarm didn't go off.

Are these typical temperatures for a watercooled 4090? Do you think the 4090 needs to be remounted/repasted?

 

[Mysterae_]

I like the display
Coolant temperature does seem high
Assuming your ambient temperature isn't
Also very high?

Is that cpu and gpu in the loop?
Only got a 3080ti and a 5950x in mine
But coolant temperature this time of year on mine
Is just over 30c
Though I have a voltage curve on gpu
And lowered it's power a bit
Is yours flat out?
Or done similar to voltage and power?

The CPU (13700k) is indeed in the loop and is auto OC'd by the BIOS but always has been, but games consume no more than 100W power. The 4090 is at stock settings.

Ambient temperatures have increased slightly, but I position the external radiator so the heat goes outside which increases the water in/out delta by quite a bit!

 

[Mysterae_]

At 45°C it doesn't seem too outrageous.

If the coolant is heating up along with the GPU then the mounting is fine! It's the dissipation that could be an issue?!

Have you checked the radiator/s or intakes for dust buildup? Changed the fan speeds at all?

So 45C isn't that bad?! You could have a bath in it at that temperature!

The radiator is as clear as a whistle as it's only just been rebuilt a month or so ago. I increased the radiator fans by way of a fan curve associated with the coolant temp when I first saw the coolant get over 40C.

When I positioned the radiator at the window to exhaust it's heat outside you can feel how warm the tube is to the radiator and how cold it is coming from it. Yet the 4090 seems to be getting hotter.

I'm wondering if pump speed is a factor.

 

[Mysterae_]

Yeah given that it's only 9c here outside
That should help considerably
The temperature difference between both photos
Definitely says something is up
Shame no flow rate to also compare

As keenan said check for dust in radiators
But with external rad that should be pretty easy to see
How longs the block been on the gpu?
And what sort of thermal paste did you use?
And no sign of clogging in micro fins?
Pump speed looks over 3k rpm so no issue there

Yeah, shame about no flow rate; the D5 Next gives the option of water temp or water flow rate, not both.

The block has been fitted since mid March going by my build log (which I was going to conclude before this), so a bit weird for it to start playing up now.
I used Artcic MX-6 for both the GPU and CPU.
The GPU is easy to check for blocks in the microfins, the CPU (EK Velocity2) not so easy!

 

[Mysterae_]

What's your current pump speed? Maybe increase it a bit and see if the temps come down..

Says 3285rpm on first photo for pump?

If my memory serves me right (not at the PC right now) the pump speed of 3285rpm (7W) is about 60% according to the D5 Next.

 

[Mysterae_]

We actually didn't..

Also, is it one loop or two separate loops for CPU/GPU?

I just wrongly assumed that folk clicked the link in my sig .

It's a simple two block loop, in and out to external radiator (4x180 fans) and single D5 pump & reservoir:

With two T30's blowing down cooling the case internals:

It still comes back to the large difference
Between those 2 photos and gpu temperature
In them
If its same circumstances for both of them
Why such a big temperature difference in just over a month?
If they built it today
And said why is my coolant 45c
Then fair enough maybe we could say that's normal for what you have

This is what it boils down to. The big jump in both coolant and GPU temperatures from March to now with nothing changed in regards to the watercooling.

I think I'll do some testing with Furmark so that I can have a stable load and tweak various things before stripping down and repasting.

 

[Mysterae_]

Sounds like a flow issue, 42c fluid temp is too high.

Hard to make out but looks like you got a heatkiller block on the FE ?

Going from the first pic. 30 degree difference between fluid temp and core is a concern. Check if your fins are blocked or thermal paste pump out.

I’m using a heatkiller strix and mora420 myself, delta between fluid and core temp is around 14-15c. About 7-8c difference between the core and hot spot.

Thermal paste pump out is the thing I'm concerned about and why I was thinking about a re-paste. A hotspot temperature of 75C on a watercooled 4090 seems a bit high.

It does sound like there's an issue then?

 

[Mysterae_]

Thanks for the help guys, now that I know there's an issue and I'm not going mad I can start to investigate it. Just when I was really starting to use the computer it looks like I need to strip it down.

Possible causes in my order of likelihood:

1. Blockage in the CPU block (EK Velocity2)
2. Blockage in the GPU block (Watercool Heatkiller)
3. Blocked filter mesh in the reservoir (AquaComputer Utilitube)
4. Thermal paste pump-out on the GPU block
5. Pump failure and not actually circulating the water
6. Blockage in the radiator
7. Poltergeist

Let's hope it's not number 7.

 

[Mysterae_]

Well number 7 is a new one
Not seen that before lol

2 choices
Do it in order of likelihood
Or do it in order of easiest/difficulty I guess

The joys of custom watercooling unfortunately

It'll be quite easy to tell if there's a blockage in either block; disconnect the quick disconnects, drain the water then connect a pressure tester with an open loop. Just like I did when checking for leaks, sort of:

With the loop being open, if the pressure builds up and doesn't immediately vacate, then I've a problem. Tomorrows job, yay.

 

[Mysterae_]

So last night I fired up a few benchmarks (Cinebench R23 and Furmark) to warm up the system to see how each individual waterblock was performing. It wasn't good.

Then I started playing a game and very quickly the temperatures of the 4090 got way out of control - like 86C hotspot temperature! That's not good at all.

I suspect it's a flow issue with the water, wherever the impediment may be.

So no gaming for me until I find it.

 

[Mysterae_]

That's the computer all stripped, cleaned and rebuilt. While it warms up I'll post some pictures of what I did and tested.

TLDR - I think it's thermal paste pump out on the GPU.

Stripped the reservoir to clean the filter of the Utilitube which was a little dirty from small bits of the EPDM hose that acts as the umbilicals between the computer and the radiator. This wouldn't cause much of an issue as the mesh is just between the reservoir contents and the pump.

Tested the flow of the pump, this is at 100% and perfectly fine.

The micro fins of the EK Velocity 2 CPU block look clear except for what looks to be a very small hair.

The micro fins of the Watercool Heatkiller for the GPU look fine also, a tiny bit of crud but noting I'd think would cause a problem. The gunk around where the o-ring would sit has what I presume silicone to protect the o-ring.

Here's a close up of the 4090's die just after removing the waterblock. Does this look like thermal paste "pump out" to any of you?

An image of the waterblock after removing the 4090.

It certainly looks like thermal paste pump out from the few images I've found from searching. This was a brand new tube of Artic MX-6 too.

The kicker is it's the same stuff I've just used on the reassembly .

Anyway, gave both block's contact plates a nice ultrasonic bath in some isopropanol alcohol...

Checked the fins were clean using my smallest feeler gauge

Pressure tested both waterblocks before refitting them. Better than factory!

Final pressure test before adding water.

How things are at present, with just an idle workload. I'll start some GPU and CPU workloads up now to bed things in.

So, thermal paste pump out on the GPU we think?

 

[Mysterae_]

it looks like the gpu paste has gone off, looks super watery which isnt right, and super thin layer too, either pump out or gone off looks to be the issue.

also getting rid of the debris in both blocks will help flow and cooling too

In all my time watercooling I've never seen that pattern of paste when removing a block. It was a brand new tube of MX-6 too.

There wasn't that much debris in the blocks to be honest, I expected something drastic!

Good Lord, seems a complete strip down was inevitable!

How are the temps looking afterwards?

Indeed it was, only way to be sure!

Temps are great now but I have a feeling it won't last....

This will happen again. Normal thermal pastes are just not good for 4090 die. It will rise back eventually. Also that hotspot/core delta was horrendous.
You have 3 options:
-liquid metal(not worth it imho)
-Thermalright TFX(the most viscous thermal paste on the market)
-Honeywell PTM7950 - this is the stuff 4090fe uses originally. Its not pumping out on mine for the last year - I've had issues even with TFX, but significantly less than any other paste(I've tried multiple pastes over the last few years, both on 3090 and 4090, all are s***t after few weeks/months.)
Rule of thumb is that if hotspot/core delta is >10 degrees under load then it indicates a problem with the paste/mount.

Thanks for this post, and yes it's inevitable that it will happen again as I used the same MX-6 thermal paste. It's only a matter of time.

I'm going to order some Honeywell PTM7950 that you mention in preparation for the repasting. I've used liquid metal in a previous build and it was a pain, it's not going anywhere near the 4090!

That’s why I stop using thermal Grizzly Kryonaut. Performance was good, but wouldn’t last long.
Using the MX6 now, but both the 4080 and the 7800x3D are relatively “cool”.
Maybe time for those sheets from Thermal Grizzly or similar?

Interesting that you're using MX-6 too, it'll be interesting to see how you get on but as you say it may not affect you.

 

[Mysterae_]

Not sure how I ended up here, but this is a splendid thread

I had never heard of pump out!

However you arrive, you're welcome @dlockers! I'm just having break from all the house diy stuff but normal service will resume!

I had heard of pump out but never experienced it, part of me thought it may have been a myth - an excuse for crap or badly applied thermal paste. I know differently now!

 

[Mysterae_]

Yeah gpu thermal paste looks terrible
Would say that's main issue
Though coolant temp seemed high before too
It may be worth ordering
Some of the Pastes mentioned
Just in case since you're rebuilding it with
Same paste as first time

Never used a reservoir with a filter
Does make me curious if it affects flow
At all

Yeah thermal sheet may be an idea
Think there's a few manufacturers making those now

You do bring a valid point - why was the coolant temp so high?

Something was restricting the flow in the computer like it had never done before. I even thought that I'd mixed up the in and out quick disconnects as that would affect flow tremendously, but the tubing going in to the computer was a lot cooler than coming out. I have some thermal images of this and as soon as I can get my thermal camera connected I'll share some images of this.

As for the reservoir and it's filter, I don't think it impedes the flow.

 

[Mysterae_]

I guess it's sort the main obvious issue first
Then see how coolant temp is
And look at that
Yeah in my head at least
If the filter impeded flow
The level in the reservoir would start rising
If coolant was leaving pump faster
Than returning to it

It's a shame I didn't integrate a flow sensor in to the circuit, I even have 2 spare from a previous build (Aqua Computer MPS Flow 200). I didn't want to fit one because of space constraints or having to use an Aquaero.

Thinking of scenarios, perhaps a poorly connected quick disconnect (or something else) impeded flow and the increasing temperatures exasperated the GPU core temperatures leading to the pump out. Totally surmising here.

Yeah its not the first time for me, i have a friend that had the same issue with his loop but his cpu paste had gone off, his 5900x was getting to mid 80-low 90's and wondered why, i kept saying you need to repaste and eventually he did, new mx4 beforehand and looked watery, but he used a bit too much, silver 5 to the rescue and temps maxing around 69c.

even the tiniest amount of debris is something you really dont want in blocks as it can promote growth eventually, though coolants are very good these days to stop that.

It's weird that you hear very little about something but as soon as I searched for pump out on 4090 I get loads of results! It seems it's known even on non-watercooled 4090s due to their high power load (+400W).

 

[Mysterae_]

Finally managed to get some images downloaded from my thermal camera that I took before stripping the computer down. It picked a good time to require an update...

The is the rear of the computer, the inlet/feed is the quick disconnect at the bottom and the outlet is the quick disconnect at the top. The outlet is much warmer than the feed so at least I didn't mix them up - that would have been hilarious.

The image below is the reservoir and pump showing the warm water going in and out of the pump to the top of the MoRa3 radiator. The outlet of the radiator is the shiny thing just at the bottom of the pump. The water is much cooler so the radiator is doing it's job.

Below is the Heatkiller block for the 4090 - the top tube is the inlet to the block and the bottom tube is the outlet. There is definitely some heat absorption by the coolant.

This leads me to believe there was a flow restriction which meant the coolant hung around the components for longer than it should leading to a high coolant temperature. With the MoRa3 radiator being so damn efficient causing the delta of the coolant to be so large, it may have caused the thermal paste "pump out" on the GPU, each temperature cycle making it worse. Sounds plausible?

 

[Mysterae_]

Definitely thermal paste pump out, though i've never had this happen that does not look right. You've applied the same stuff back too!

Hope its not another strip down a few months down the line again man.

Indeed! I think it's inevitable that the 4090 will need to be repasted with something more appropriate. At least it's just the graphics card that will need it so not a total strip down.

Temperatures are back to where they should be - a benchmark run of FurMark 2 at 2560x1440 with a 438W load - core 47C and hotspot 54C. It's a shame the FurMark knocks out my secondary screen while it runs, means I can't see the stats on the fly.

I wonder how long it will last!

 

[Mysterae_]

Themal camera very nice tool for analysing the loop

You have just about everything very nice build
I would definitely think of adding
A flow meter too
Temperatures tell half the story
When my cpu block was clogged up
Surprisingly little effect on Temperature
Was only because I added a 3080ti with active backplate
And couldn't figure out
Which ports were in and out on active backplate
By looking at the flow
I realised something else was up with my loop
As tested all 6 possible in/out combinations on the 3080ti

It's amazing how useful a thermal camera can be, from diagnosing car issues, around the house to even PC cooling .

I'm considering adding one of the flow meters I mentioned earlier, depends if it can be connected to the D5 Next without losing water temperature. I wonder if AIDA64 would pick it up if just connected via USB. It'll be tricky to fit though with not a lot of space available.

It can be frustrating when something changes with your computer with no obvious reason as to why.

 

[Mysterae_]

Not totally sure if aida64 would pick it up
Just using usb
Only a guess but can't see why not
Other devices use usb to report stuff
I use a tablet or any old android device
I have Laying around for aida64
But I have sensor wire for temperature to motherboard not usb
And just using a standard spinning wheel for flow
Lowered coolant level in distro too
So it's below the coolant return point
Gives me a visual on the returning coolant speed/flow
Might add a digital flow meter at some point though

Could you add flow meter on the external tubing to the rad?
Might need to lengthen the wires
But have seen someone in here forget who
They have all the wiring for fans and pump etc extended
To a mora
But they have used some sort of quick disconnect wiring plugs
So can detach the wiring and coolant tubing from the pc
If that makes sense lol

In this build I wanted as few connections between the PC and the radiator as my last build had 5 (fans, Aquabus, power, quick disconnects x2) so went with only the quick disconnects. I gave in and added a USB connection between them to provide me with the coolant temp and pump and radiator fan speeds - that has proved to be invaluable here.

D5 next has a header for flow sensor, no?
Just make sure you use old type of sensor. New Aquacomputer flow sensor(flow2/next) makes clicking noise and was the loudest part of the loop, was deiving me mad(that was after adjusting the propeller, out of the box I've heard it from another floor)

Yes and no - from the D5 Next manual:

5.3. Connector “Fan” for fan or flow sensor -
Depending on configuration, this connector can either be used as a PWM regulated fan output with speed signal processing or to connect a flow sensor. Simultaneous use of both functions is not possible!

I'm already using this connector to control the 4x180 PWM fans of the MoRa3. Shame! Can't even connect it to the "Bus" connector as it's not on the list of compatible Aquabus devices. If I really wanted to have it I could reinstate my Aquaero controller but that would be a challenge to find somewhere to fit it .

 

[Mysterae_]

All this has made me want to repaste my 4090, temps have been creeping into the mid/high 50s for a while now. I think I'll just liquid metal as I've done with all my past gpus, never went off or pumped out.

Liquid metal will definitely make a difference but I think it was too big a risk for me. My 4090 stands vertically just like the PS5 that had many problems with liquid metal leaking out.

Good luck to you should you decide to do it!