Jump to content

Welcome to ExtremeHW

Welcome to ExtremeHW,聽register to take part in our community, don't worry this is a simple FREE process that requires minimal information for you to signup.

Registered users can:聽

  • Start new topics and reply to others.
  • Show off your PC using our Rig Creator feature.
  • Subscribe to topics and forums to get updates.
  • Get your own profile page to customize.
  • Send personal messages to other members.
  • Take advantage of site exclusive features.
  • Upgrade to Premiumto unlock additional sites features.
IGNORED

Modern GPUs - Cooling, Power & Mounting


J7SC_Orion

Recommended Posts

Hello GPU aficionados聽馃檪

I am starting this thread to look at modern GPUs and some of the things to look out for with cooling, power and also mounting...聽

Given the various 12VHPWR issues which have crept up, as well as 'cracked' PCBs and such, I figured a closer look is warranted...

  • Respect 2

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

Posted (edited)

I start with cooling a 4090 (original air cooling, then water). The key variable I am interested in post is the delta between GPU and Hotspot. This parameter can be quite telling, also if it deteriorates over time when running the same app and oc. More to come later, and your comments are welcome.聽

4090 clocks shown with core voltages (1.00, 1.05, 1.075, 1.10) range from聽 2775 MHz to 3240 MHz, PL up to 650W.聽

聽聽

4090_air_water_u.thumb.jpg.a27d6fa2ca186d57a130cf614d11f0d2.jpg

4090coreclockDelta_U.thumb.jpg.cfa83fcc424e6f3fb0be74572ba74c1f.jpg

聽聽

Apart from keeping overall temps in check for a 600W + power limit (even if one usually sets it well below that), it is also a question of keeping the temp delta between GPU and hotspot relatively low - typically less than 20 degrees delta for a water-cooled GPU.

Another parameter to watch out for with the newer power hungry GPUs is how long 'heat soak' can be delayed....the bigger the cooling surface of the rads, the liquid volume in a loop and also the number (and type) of fans, the longer heat soak can be delayed. The 4090 system above has a total rad space area of 1320 mm x 63mm 'triple core' copper / brass rads.聽 Below is a comparison between a single core aluminum 360 AIO rad and a triple-core copper/brass rad.聽 One guess which one cools better...

aio_full_rad_U.thumb.jpg.9f37daa61b37c5ca2f4cdd0adb585e2b.jpg

聽聽

When it gets to cooling modern GPUs with gobs of power, the GPU itself (rather than just rads, pumps etc) need attention. Obviously, there is the water-block itself, but also the thermal paste for the die - I typically use Gelid GC Extreme for CPU and GPU dies, but there are other good pastes out there. For VRAM, I have switched to thermal putty a while back and so far, the experience has been great - even after 2.5 years plus, identical temps show in stress tests at the same adjusted ambient temp. The only really draw-back of thermal putty is that it is a bit messy to clean up - but I rarely take a GPU apart after water-blocking it; I have some water-blocked cards in regular use for more than four years without disassembly.聽

For VRM components, I also either use thermal putty, or Thermalright thermal pads. Finally, I add thermal putty to the back of the GPU and then mount an extra heat-sink on the back - it does help with power-hungry GPUs that generate a lot of heat over a small area.

Collage pic of 2080 Tis聽 top left, 6900XT bottom left, 3090 top right, 4090 bottom right:

GPU_temps_wc_N_hrz.thumb.jpg.29ced9070dc2a139ecac3457b69dc682.jpg

Edited by J7SC_Orion
  • Thanks 1
  • Respect 2

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

The fewer connection points between PSU and GPU, the better, in theory. Every time you add a connection (right angle adapters, cable extensions, splitters), you add contact resistance. None of the contacts will have the exact same resistance either, and differences can compound as connection points are added, leading to current imbalances between wires. It is possible that using a squid splitter that takes 3-4 PCIe cables from the PSU, and combines them into a 12VHPWR connector, would have lower resistance than a single direct PSU cable, since the current is shared across more wires and contacts for most of the path, but I haven't seen anyone test this.聽

Use an anti-sag support to relieve any strain on the PCB / slot.

  • Thanks 1
  • Respect 2
Link to comment
Share on other sites

Posted (edited)

Of dimples, dongles and power arcs...

Power connections for GPUs with a stock vbios of 600W or more (never mind 700+ W transient spikes) are a major recurring problem theme. Before touching in the 12VHPWR saga (also already touched on by @T.Sharp聽), it is worth pointing out that melt problems can arise even with traditional connectors. Below is pic of the 3x8pin PCIe of my 3090 Strix with a custom vbios that could peak at ~ 600W...there was a loose connection and some arcing / minor melting which must have occurred after the original mounting and before other parts tugged and obscured the view of it. Fortunately, each single connector had 200W at most, and it all still works great (including folded electrical tape to fix the area around the arrow).

3090pwrmelt.jpg.28ef49acc9f75c9f3a588a42183dc6e3.jpg

The 12VHPWR introduced last year is a whole different ballgame...NVidia and also select Intel cards use that connector, but it聽with the highest-power cards such as RTX 4090s that potential weaknesses have become real problem cases - and initially at least, a lot of 'guesstimzations' as to what the underlying issue actually was (or issues were).

By now, vendors have already installed modified 12VHWPR connectors (for ~ three months now) without any official announcements It is a running change, mostly for RTX4K cards. This new 12VHPWR hybrid connector comes with 4 sense pins that are shorter now. Sense pins are the 4 smaller ones on top in the pic below of my Giga-G-OC.聽

聽聽

4090_connector.jpg.87cc5b24a246c5f0ac4b19ff98810355.jpg

This all gets down to one major point: Making absolutely certain that the connector is 100% flush mount, and secured as such. The original, longer sense pins in the early 12VHPWR connectors could be exposed in anything but a perfect flush mount and start arcing...after all, that connector carries 600W+ in a much smaller package than for example the 3x8 (6+2) pin standard connectors used in the 3090 above.

So even a slight bump of the cable could cause issues, but things got even worse with certain 180 degree adapters - a fair number melted power connections (and the odd fire) were the result...

Now, there will be a new connector design, the so-called 12V-2x6 connector. That is incidentally backward-compatible with the 12VHPWR ones.聽Still, most 12VHPWR connectors have functioned fine as long as there is no tension on the cable and it is a perfectly flush mount and secured to stay that way. Also, it is probably wise not to add superfluous bits such as adapters for cosmetic reasons (each additional connection point is another potential point of failure). In addition, especially with the early 'dimple' designs, the fewer times the cable is mounted / unmounted into / from the GPU connector the better, because eventually, that can create a bit of the dreaded 'play' in the connection via the dimples wearing and affecting the flush mount.

Apart from the power capabilities, most 4090 cards are also humongous in terms of width (and weight per upcoming post) when air-cooled, but their PCB is actually among the narrowest I have worked with (the rest of the wide air-cooler is extra metal and overhang beyond the PCB). The difference is almost comical when prepping one of these monsters for water-cooling. In terms of depth, these air-cooled cards are basically 4 slot cards.

聽聽

4090card_box_width_u.thumb.jpg.11599ff6308dedb949c00d9fbd7055e8.jpg

聽聽

Per second post above that showed various 4090 GPU and hotspot temp deltas, I find it highly advisable to water-cool any GPU that can pull more than 400W (also keeping in mind transient spikes). When it gets to the latest narrow-PCB, high-power designs by NVidia, it is also worth keeping in mind that right below the already 'sensitive' 12VHPWR connector are major VRM components that run very hot in their own right and also affect the power connection region right above it. On my 4090, I placed thermal putty behind the VRM and the power connector area where it meets the backplate. Said backplate has an additional heatsink, and 2x 80 fans above it.

聽聽

cablemod_detail.thumb.jpg.bc9e292702c0a04f94074c3bb3bf2c84.jpg

back_heatsink.jpg.6a7980adc85bf894823e7457e4fc21ef.jpg

聽聽

Originally, I ran the 4090 with the 4-into-1 dongle聽 (aka the squid, on the right in the pic below) that came in the GPU retail box...the dongle worked fine but it could only be a temporary solution given the unique build (dual mobo) and related power cable routing. The dongle got quite warm at a sustained 640W (ie. ROG Furmark/Vulkan) at the joint where the four 'arms' flowed into one, though it never got outright hot.

The 12VPWR x 2x8 pin PCI cable on the left in the pic below has not been used yet (it was supplied by Seasonic for my PX1300W Platinum PSU). The white CM 12VHPWR x 4x8 pin PCI is also custom made for my PSU and that is what I have been using since I converted the card to water-cooling and installed a custom vbios for ~ 670 W. That cable has never been taken out so that is still the 1st and only mount of the cable, and only the 2nd mount for the GPU connector itself. The white CM cable has been problem-free since early December '22. As you can tell from the pics below, there is no stress on it at all, and I can easily check re. the flush mount without touching it.聽

聽聽

12VHPWRcomp_700.jpg.22efa1ea1338f9ffaa2fbee20adce3c9.jpg聽 聽 聽12vhpwr_PSU4x8_sm.jpg.c2c247edd8552853e9e55ecd24f1c7d0.jpg

C1.thumb.jpg.520025f7d9f00fc254d93545cb258ca8.jpg

聽聽

4090extrablue.thumb.jpg.07357bb57563ac5c1f887aff5138f696.jpg

All in all, it is a no brainer hi-po GPUs require extra attention re. their new power connections, and of course cooling.聽

On cooling, there is also a bit of a quirk with 4090s: Like the 3090, the 4090 has 24 GB of GDDR6X, but unlike the 3090 which uses double-sided 1 GB VRAM, the 4090 uses single-sided 2 GB VRAM. The VRAM on the 4090 is faster from the get-go and can be over-clocked quite well - but for real high OC speeds, the 4090 VRAM should be in the mid 50 C range or higher; it likes a bit of fuzzy warmth聽馃憖

Edited by J7SC_Orion
  • Thanks 3

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

18 hours ago, TonyBombassolo said:

FWIW I had a cablemod connector melt at around 465w.

...that's why I hammered away in this thread at caution and vigilance re. adapter flush seating. The 12VHPWR 'saga' started with several of the 4-in-1 dongle cables melting on 4090s last fall; then came some cables and also special adapters - especially the CM 180 degree ones for Asus (which has the connector flipped around on their cards).聽

  • Thanks 1

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

1 hour ago, J7SC_Orion said:

...that's why I hammered away in this thread at caution and vigilance re. adapter flush seating. The 12VHPWR 'saga' started with several of the 4-in-1 dongle cables melting on 4090s last fall; then came some cables and also special adapters - especially the CM 180 degree ones for Asus (which has the connector flipped around on their cards).聽

Yeah, this led me to think that they should have thought about a different mechanism for securing the connector against side to side tension that can just enough make it come out and have problems.

  • Thanks 1
  • Respect 1

Showcase

 Share

CPU: AMD Ryzen 9 5900X
GPU: Nvidia RTX 3080 Ti Founders Edition
RAM: G.Skill Trident Z Neo 32GB DDR4-3600 (@ 3733 14-8-14-14-21-35 1T GDM)
MOTHERBOARD: ASUS Crosshair VIII Dark Hero
SSD/NVME: x2 Samsung 970 Evo Plus 2TB
SSD/NVME 2: Crucial MX500 1TB
PSU: Corsair RM1000x
MONITOR: LG 48" C1
Full Rig Info

Owned

 Share

CPU: Intel Core i7-4790k
GPU: ASUS GTX 980 Strix
RAM: Corsair 32GB DDR3-2400
MOTHERBOARD: ASUS Z97 Deluxe
SSD/NVME: Crucial M550 512GB
HDD: 2x WD Black 3TB
PSU: Corsair AX750
CASE: Lian Li O11 Dynamic
Full Rig Info
Link to comment
Share on other sites

Content Creator
269 195

I'll lump into this because I've had some interesting things with my 4090. I have the Strix OC model and honestly just run it stock all the time. That combined with my 360 + MORA420 is enough to keep it comfortably around 40-45c under any stress test with the hotspot temp being around 5-7c higher than that. That being said, if I max my pump out to 100% from 60% I go down to 37-38c. So I definitely think these modern, denser-finned coldplates are a lot more restrictive. (I have a 7950X3D in the same loop as the 4090, both with Heatkiller Pro blocks on them). In the future I want to move my pump/res external so I can get it away from my head and turn up the speed while not being able to hear that awful coilwhine.

For what it's worth, I also have the Cablemod cable - but I removed their shield thing as I found it made it harder to verify if the connector was 100% seated.

image.png

Edited by Alex
  • Thanks 1
  • Respect 1

Owned

 Share

CPU: AMD Ryzen 9 - 7950X3D
MOTHERBOARD: Asus X670E Hero (pls EVGA make an AMD mobo)
RAM: G.Skill 64GB @ 6000Mhz
GPU: RTX 4090 Strix OC
SSD/NVME: 2x SN850X 4TB | 1x 970 Pro 0.5TB
MONITOR: Something OLED, 120hz and 42-48"
WC RADIATOR: MO-RA3 420 | HeatKiller 360S
WC CPU BLOCK: Heatkiller IV Pro
Full Rig Info

Owned

 Share

AUDIO: Topping DX7 Pro
AUDIO 2: Sennheiser HD800S
AUDIO 3: Sennheiser HD650
AUDIO 4: Sennheiser HD 4.50 BTNC
AUDIO 5: Edifier RT1700BT
AUDIO 6: Custom XLR balanced cables
Full Rig Info

CA$40000

Owned

 Share

CPU: F22C | 257hp 251nm
CPU COOLER: K&N FIPK Intake | AEM filter
GPU: Yokohama AD08R 255/40r17
CPU COOLER 2: Titan 7 R-10s 17x9.5+51
AUDIO: Alpine UTE73BT + Modifry DCI
MOTHERBOARD: 脰hlins Road & Track DFV Coilovers
OPERATING SYSTEM: Hondata
CASE: Recaro Pole Position
Full Rig Info
Link to comment
Share on other sites

16 hours ago, Alex said:

I'll lump into this because I've had some interesting things with my 4090. I have the Strix OC model and honestly just run it stock all the time. That combined with my 360 + MORA420 is enough to keep it comfortably around 40-45c under any stress test with the hotspot temp being around 5-7c higher than that. That being said, if I max my pump out to 100% from 60% I go down to 37-38c. So I definitely think these modern, denser-finned coldplates are a lot more restrictive. (I have a 7950X3D in the same loop as the 4090, both with Heatkiller Pro blocks on them). In the future I want to move my pump/res external so I can get it away from my head and turn up the speed while not being able to hear that awful coilwhine.

For what it's worth, I also have the Cablemod cable - but I removed their shield thing as I found it made it harder to verify if the connector was 100% seated.

image.png

That is a very nice build !聽聽

One quick question: I cannot quite make out how the GPU +w-block is supported (I am working on a post about weight of new-gen GPUs that for this series); how are you supporting that GPU/w-block combo ?聽

As to your point about moving some bits 'out of the case' for external positioning, it adds gobs of flexibility - my earlier project-insanity taught me that聽馃お...Just one example would be my previous 'primary' system (now serves as a secondary work+play setup)...it has two mobos (front and back) on a heavily modded TT Core P5, plus 5x 360mm x 62mm rads, 4x D5 pumps, plus an AIO etc. That was the last time I tried to stuff everything on/in one case. The problem is also the sheer and awkward-to-move weight (well over 100 pounds).聽 Here is an early pic: of that build on the left, followed by an update how it looks now聽 (a w-cooled 3090 Strix joined the 2x 2080 Ti for some productivity tasks)...

C2_OrcaNew.thumb.jpg.147d41c8c9ce35e417b8fa1be161087f.jpg

聽聽

My current 'primary' work+play build is also a dual mobo one (2x聽 X570 regular ATX), but side-by-side this time per pic below, using a TT Core P8. I already used Koolance QD4 quick-disconnects on the previous build (btw, the horizontal GPU cooler tubes on the front above are actually cooper pipes...).聽

Learning from my prior projects, this current primary setup has 5x D5 pumps and all the rads (6 of them with a combined total of 2520mm x 63 mm) on a separate little, wheeled 'table'. The 4x QD4s also pictured below make it easy to update / move / clean etc, noting that the reservoirs stayed on the case, high up and in the back.聽 The 'cooling table' is only 2 feet away, yet I hear only the faintest whoosh of air, not actually any of the 40+ push-pull 120mm fans (fixed at 1800 rpm) or D5 pumps (fixed at 70%).

With your MORA 420, you are pretty much on the path of a 'segmented' build anyway - it is very liberating to be able to build more outside a regular 'case' and customize any which way one sees fit. At the end of the day, the foot print is also better at the personal work space.

C1_workcorner_ver2.thumb.jpg.235e5b1b795d1017419300596076b8e4.jpg

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

Content Creator
269 195
11 hours ago, J7SC_Orion said:

That is a very nice build !聽聽

One quick question: I cannot quite make out how the GPU +w-block is supported (I am working on a post about weight of new-gen GPUs that for this series); how are you supporting that GPU/w-block combo ?聽

Nothing really, the soft tubing helps of course but the whole thing is an enclosed block. There is zero flex because of the waterblock design but obviously as you can see it is tilted slightly down. Doesn't really bother me though.

11 hours ago, J7SC_Orion said:

As to your point about moving some bits 'out of the case' for external positioning, it adds gobs of flexibility - my earlier project-insanity taught me that聽馃お...Just one example would be my previous 'primary' system (now serves as a secondary work+play setup)...it has two mobos (front and back) on a heavily modded TT Core P5, plus 5x 360mm x 62mm rads, 4x D5 pumps, plus an AIO etc. That was the last time I tried to stuff everything on/in one case. The problem is also the sheer and awkward-to-move weight (well over 100 pounds).聽 Here is an early pic: of that build on the left, followed by an update how it looks now聽 (a w-cooled 3090 Strix joined the 2x 2080 Ti for some productivity tasks)...

External cooling is a huge bonus IMO. Makes the inside of the case cleaner, easier and honestly the whole thing was MUCH less of a hassle to put together. It's not like my desktop moves around tons anyway, and having QDCs on the external loop makes it as easy as unplugging a cable.

11 hours ago, J7SC_Orion said:

聽My current 'primary' work+play build is also a dual mobo one (2x聽 X570 regular ATX), but side-by-side this time per pic below, using a TT Core P8. I already used Koolance QD4 quick-disconnects on the previous build (btw, the horizontal GPU cooler tubes on the front above are actually cooper pipes...).

Learning from my prior projects, this current primary setup has 5x D5 pumps and all the rads (6 of them with a combined total of 2520mm x 63 mm) on a separate little, wheeled 'table'. The 4x QD4s also pictured below make it easy to update / move / clean etc, noting that the reservoirs stayed on the case, high up and in the back.聽 The 'cooling table' is only 2 feet away, yet I hear only the faintest whoosh of air, not actually any of the 40+ push-pull 120mm fans (fixed at 1800 rpm) or D5 pumps (fixed at 70%).

With your MORA 420, you are pretty much on the path of a 'segmented' build anyway - it is very liberating to be able to build more outside a regular 'case' and customize any which way one sees fit. At the end of the day, the foot print is also better at the personal work space.

In the future I want to get an L-shaped desk (currently have the IKEA Alex+Karlby combo, want to get another Karlby counter-top and create an L shape from it). In where the two desks join I was thinking of making a "cool box". Essentially some spray-painted black or white plywood to house as much watercooling components as I can.

The idea is to try fit at least 2x MORA420s with their own pumps and a single res in the middle. Then have an HVAC filter for an intake to try eliminate dust build-up... only worry is static pressure but that's for when I actually build it 馃槃

  • Respect 1

Owned

 Share

CPU: AMD Ryzen 9 - 7950X3D
MOTHERBOARD: Asus X670E Hero (pls EVGA make an AMD mobo)
RAM: G.Skill 64GB @ 6000Mhz
GPU: RTX 4090 Strix OC
SSD/NVME: 2x SN850X 4TB | 1x 970 Pro 0.5TB
MONITOR: Something OLED, 120hz and 42-48"
WC RADIATOR: MO-RA3 420 | HeatKiller 360S
WC CPU BLOCK: Heatkiller IV Pro
Full Rig Info

Owned

 Share

AUDIO: Topping DX7 Pro
AUDIO 2: Sennheiser HD800S
AUDIO 3: Sennheiser HD650
AUDIO 4: Sennheiser HD 4.50 BTNC
AUDIO 5: Edifier RT1700BT
AUDIO 6: Custom XLR balanced cables
Full Rig Info

CA$40000

Owned

 Share

CPU: F22C | 257hp 251nm
CPU COOLER: K&N FIPK Intake | AEM filter
GPU: Yokohama AD08R 255/40r17
CPU COOLER 2: Titan 7 R-10s 17x9.5+51
AUDIO: Alpine UTE73BT + Modifry DCI
MOTHERBOARD: 脰hlins Road & Track DFV Coilovers
OPERATING SYSTEM: Hondata
CASE: Recaro Pole Position
Full Rig Info
Link to comment
Share on other sites

A 'weighty affair', or 'honey, does this make me look fat...?'馃

Beyond cooling and physical size of modern GPUs discussed (and fyi, updated) in the earlier posts on modern GPUs, it is time to look at weight and mounting orientation...

Top-tier GPUs used to be a lot lighter, whether factory-air-cooled or using water-cooling...take for example the Antec 302 cases on the top right in the next pic, two of which had quad GPUs in them, mounted vertically (terrible idea re. heat build-up and retention, but that was all the room in the 'office' I had, apart from being just plain silly)...

QuadsGPU_1.thumb.jpg.f028c334d53cbfd57c026d7a9cd640ae.jpg

聽 聽

...even the then-new聽 high end GPUs (ie. 980s) such as the EVGA Classifieds came with a cold plate, depicted below on the top right. That in turn meant that a universal water-block (Swiftech shown below) combined with 120mm fans blowing down on strategic bits of the cold plate was all that was really needed for top performance.

By the time the 2080 Ti rolled around, things really started to change...a single聽factory-water-blocked 2080 Ti was about as heavy as the previous heaviest card I used which was a water-blocked dual-GPU AMD 8990 (both pictured lower left). In fact, the 2x 2080 Ti project was the last time I build anything with 'normal horizontal mounting orientation' (mobo vertical, GPU horizontal, per build-log photo lower right). FYI, I ended up supporting those two 2080 Tis with custom-cut pieces from the super-dense foam from the boxes GPUs typically come with - useful stuff, that聽馃憤聽(as long as it is really dense).

聽 聽

QuadsGPU_2.thumb.jpg.c04846e9adce9b5783fefe0186b3675a.jpg

聽 聽

Nowadays, the weight problem is even worse, IMO:聽Four of the EVGA Classified GPUs shown above with cold-plates and universal water-coolers weigh only a bit more combined than聽one water-blocked 4090...which with its 600+ W capability need much more massive cooling areas (including metal). Apart from the sheer maximum wattage, there is also the fact that the heat is much more concentrated in a smaller are of the die, but using more layers in the PCB. In addition, the聽 overall VRAM capacity is also much higher these days which on its own produces additional heat, never mind more heat per VRAM chip.聽 The worst VRAM offender I have in use was/is the 3090 with double-sided 24 x 1 GB VRAM...the ones on the back of the 3090 PCB were ready for barbequing until I added an additional metal heatsink and a few other cooling tricks... Finally, the VRMs also produce more heat since they provide more power output.聽

The increased weight and size has also led to a new type of problem according to several folks, including serious commercial repair operators such as Krisfix.de / YouTube . There is a marked increase of latest-gen cards 'dying' due to cracked and torn layer/s in the PCBs (especially some long, air-cooled 4090s) around the areas highlighted with the red circles below.. I am fortunate to never have experienced that, but it does not have to be an outright, visible crack / tear with a card that no longer works at all - sometimes, it can also be dropped PCI lanes, such as a 16x lane PCI card only showing 8x or even 4x in software monitoring even when there is nothing else competing for the lanes, and following verified initial 16x functionality.聽

Weight_1b.thumb.jpg.e4cc20bcdc0c4814a2259fc71bcb290e.jpg

聽 聽聽

...it is not only the weight itself, but also the fact that the PCI 'fingers' of the card are offset to one side with the new ultra long & heavy cards and their cooler overhang when water-cooled - that creates a lever force effect that is responsible for much of the tearing. OEM custom ordered system with these cards are even more prone to this kind of damage, due to '''shipping'''聽鈽狅笍.聽

As @T.Sharp聽already pointed in an earlier post, at the very least, it is wise to add a support bracket of some kind at the right side if the card is mounted the traditional, horizontal way. In any case,聽I switched over to vertical mounting over the last four-plus years. This does mean using PCIe 4.0 risers, though, or mount the mobo horizontally. On risers, I ran several tests with both a 6900XT and 3090 Strix using quality PCI 4.0 risers (LinkUp) on a temporary test-bench below before water-cooling and permanent / final install.聽 I found no reduction in fps and score at all compared to a direct PCI slot mount of the same card in the same conditions. Finally, with my latest dual-mobo build (on the bottom in the pic below), there is zero weight and zero lever / twisting forces on the PCI / risers.聽

vertical_1.thumb.jpg.4b47573b954cff11b4e075c883377bc9.jpg

聽 聽

...at the end of the day, there is no such thing as a 'free lunch', at least when it gets to top-tier GPUs in each generation. While there is no doubt that GPU efficiency has increased greatly over the last few generations, it is also true that overall power consumption is still going up - which is not surprising if you consider this table...

GPU_gen_data.jpg.eccf598e85ff5add67e6d1656c13b18d.jpg

聽聽

聽 聽

  • Respect 2

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

I'm also a big fan of thermal putty as a pad replacement. It removes all the guesswork and allows for the best possible contact and pressure on the core. A very common issue that people run into when upgrading to high performance pads, is that they are more stiff / harder to compress than OEM. Even if you get the proper thickness, they can prevent optimal core contact.

Good putty matches or outperforms high end pads, so there's really no downsides AFAIC. Sure it leaves a bit of residue, but high end pads do as well. IME, it's easier to remove and clean after it has been on a card for a while, because it will stiffen up a bit and come off in complete pieces. Depending on the age, you may be able to reuse it too. You can knead and mix it back together to soften and even out the consistency.聽

Not too long ago, the only decent choice was TG-PP10, but it has since been discontinued. Fortunately there are quite a few options on Aliexpress that outperform PP10 now. Snarks Domain on YT has done some nice testing of different putties. Based on his results, I snagged some U6 Pro and it has been great. More gooey than TG-PP10, which I'm not super fond of, but once it bakes for a while, it becomes more cohesive. (I take my cards apart a lot for experiments. lol)

Just today I got in a shipment of CX-H1300 because I wanted to try one of the cheaper less hyped products. The consistency is more like a non-Newtonian fluid than U6, and it isn't nearly as gooey or sticky, which is a plus. It's very dense like U6 Pro and I expect it will perform about the same. Will update when I test it on my 6700 XT.

Here's how the paste print looks with perfect contact. The unevenness is caused by the warped die. It may not look that bad, but putting a razor blade on it shows that it's far from flat 馃ゲ

KPx_U6.jpeg

EDIT : One other great product I forgot to mention, is PTM7950. It's a paste alternative that comes in a 0.2mm sheet that you cut to size. It starts to melt above 45c, allowing the excess to flow out and creating a very thin bondline. The big benefit is that it doesn't suffer from pump-out like regular paste, and can maintain its performance for a very long time. It will actually improve over time as the bond gets thinner from heat cycling.

With air cooling, you can reach ~95% of the performance in just a few minutes of Furmark with the fans turned down. In my testing against KPx paste (which performs identically to Kryonaut Extreme in Luumi's tests), the PTM runs about 2c cooler after giving it time to fully seat. With water cooling, your results may vary, since the block may never get hot enough to allow the PTM to achieve optimal bondline thickness. You could try turning the pump and fans way down, but I'm sure some people wouldn't be fond of letting their blocks and coolant get above 45c.

If you're interested in PTM, I got legit stuff from "passionate girl store" on aliexpress. It's much cheaper than other sources.

Here's an Nvidia engineer discussing the PTM used on the 4090 FE :

Edited by T.Sharp
  • Thanks 1
  • Respect 1
Link to comment
Share on other sites

Posted (edited)
2 hours ago, T.Sharp said:

I'm also a big fan of thermal putty as a pad replacement. It removes all the guesswork and allows for the best possible contact and pressure on the core. A very common issue that people run into when upgrading to high performance pads, is that they are more stiff / harder to compress than OEM. Even if you get the proper thickness, they can prevent optimal core contact.

Good putty matches or outperforms high end pads, so there's really no downsides AFAIC. Sure it leaves a bit of residue, but high end pads do as well. IME, it's easier to remove and clean after it has been on a card for a while, because it will stiffen up a bit and come off in complete pieces. Depending on the age, you may be able to reuse it too. You can knead and mix it back together to soften and even out the consistency.聽

Not too long ago, the only decent choice was TG-PP10, but it has since been discontinued. Fortunately there are quite a few options on Aliexpress that outperform PP10 now. Snarks Domain on YT has done some nice testing of different putties. Based on his results, I snagged some U6 Pro and it has been great. More gooey than TG-PP10, which I'm not super fond of, but once it bakes for a while, it becomes more cohesive. (I take my cards apart a lot for experiments. lol)

Just today I got in a shipment of CX-H1300 because I wanted to try one of the cheaper less hyped products. The consistency is more like a non-Newtonian fluid than U6, and it isn't nearly as gooey or sticky, which is a plus. It's very dense like U6 Pro and I expect it will perform about the same. Will update when I test it on my 6700 XT.

Here's how the paste print looks with perfect contact. The unevenness is caused by the warped die. It may not look that bad, but putting a razor blade on it shows that it's far from flat 馃ゲ

KPx_U6.jpeg

EDIT : One other great product I forgot to mention, is PTM7950. It's a paste alternative that comes in a 0.2mm sheet that you cut to size. It starts to melt above 45c, allowing the excess to flow out and creating a very thin bondline. The big benefit is that it doesn't suffer from pump-out like regular paste, and can maintain its performance for a very long time. It will actually improve over time as the bond gets thinner from heat cycling.

With air cooling, you can reach ~95% of the performance in just a few minutes of Furmark with the fans turned down. In my testing against KPx paste (which performs identically to Kryonaut Extreme in Luumi's tests), the PTM runs about 2c cooler after giving it time to fully seat. With water cooling, your results may vary, since the block may never get hot enough to allow the PTM to achieve optimal bondline thickness. You could try turning the pump and fans way down, but I'm sure some people wouldn't be fond of letting their blocks and coolant get above 45c.

If you're interested in PTM, I got legit stuff from "passionate girl store" on aliexpress. It's much cheaper than other sources.

Here's an Nvidia engineer discussing the PTM used on the 4090 FE :

I put away two unopened vacuum-sealed TG -PP10 jars in another vacuum-sealed back...they are residing at the bottom drawer of our fridge. While it might not last, given the rated shelf-life of unused TT-PP10, the way it is packed and stored, it just might - we'll see. In any case, good to see some of the alternative thermal putties you showed / mentioned聽馃憤聽since TG-PP10 has been discontinued, tx for that !

43 minutes ago, Barefooter said:

I've seen putty used before, but never tried it myself.

It seems like it would work better than regular pads, especially if you have irregularities.

I'll have to pick some up and give it a try one of these days.

聽 聽

Yea, thermal putty is great stuff - not least as it 'conforms' to the available space, and ends up providing additional cooling to surrounding bits. It is of course harder to clean up for disassembly, but so far on my three late-gen GPUs where I used thermal putty, it is working perfectly. Even after two+ years of use, temps are 'steady as she goes'. The only 'problem' is the VRAM on the 4090...as pointed out above, that actually likes a bit of warmth (~ 55 C) for best overclocking results. That said, my card's VRAM oc quite well even below that and for top bench runs, I can always run memtest_vulkan just before the bench聽馃き

Edited by J7SC_Orion
  • Respect 1

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

1 hour ago, J7SC_Orion said:

I put away two unopened vacuum-sealed TG -PP10 jars in another vacuum-sealed back...they are residing at the bottom drawer of our fridge. While it might not last, given the rated shelf-life of unused TT-PP10, the way it is packed and stored, it just might - we'll see. In any case, good to see some of the alternative thermal putties you showed / mentioned聽馃憤聽since TG-PP10 has been discontinued, tx for that !

I expect it will last for years in the fridge. The expiration date seems more like a formality than anything, probably a requirement for such products being sold to OEMs. Not to mention, I'm sure TG would be happy for you to throw your stock away after 12 months and order more聽馃槃

But... if you're all about having the best performance, you should try some UX Pro. It's the best stuff around, although not cheap.

Screenshot2023-08-03184809.thumb.png.b99eb672a5fbd5812c3e8d02e8a2aaef.png

(the results here may be skewed a bit, as he explains)

Edited by T.Sharp
  • Respect 1
Link to comment
Share on other sites

3 hours ago, T.Sharp said:

I expect it will last for years in the fridge. The expiration date seems more like a formality than anything, probably a requirement for such products being sold to OEMs. Not to mention, I'm sure TG would be happy for you to throw your stock away after 12 months and order more聽馃槃

But... if you're all about having the best performance, you should try some UX Pro. It's the best stuff around, although not cheap.

Screenshot2023-08-03184809.thumb.png.b99eb672a5fbd5812c3e8d02e8a2aaef.png

(the results here may be skewed a bit, as he explains)

Thanks for that 馃檪-聽booked-marked the vid for future reference as that UX Pro Ultra looks good. I might add some work-related cards in the near future, but for my regular machines, I probably wait for RTX 5K in late 2024 / early 2025 (unless AMD comes up with something extremely tempting before then). But whatever the timing, there will be thermal putty involved...

As already alluded to, the 2 GB GDDR6X chips on the 4090 are weird in that they actually like some heat for top oc. My th.putty application for that card works almost a little too good. But in any case, all GPUs I have with th.putty (including two from early 2021) are performing great over time > no net change with the same oc, load and standardized ambient temps.

FYI, whether th.putty or th.pads (such as Thermalright Odyssey 16.8 W/mK), I always add a little bit of thermal paste on top of either - could be overkill but seems to work fine...聽 聽In addition, I add th.putty onto the back of the PCB at strategic spots as well as the back of the GPU chip itself before mounting the metal backplate which in turn gets an additional big heatsink (pics above posts). With the hot summer weather and the power-slurping GPUs, I am glad that my equipment stays relatively cool.

  • Respect 1

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

59 minutes ago, J7SC_Orion said:

As already alluded to, the 2 GB GDDR6X chips on the 4090 are weird in that they actually like some heat for top oc. My th.putty application for that card works almost a little too good. But in any case, all GPUs I have with th.putty (including two from early 2021) are performing great over time > no net change with the same oc, load and standardized ambient temps.

Interesting that the 4090 likes some heat, I wonder what could cause that. I've heard of cold bugging with LN2, but at ambient temps that's surprising. Either way, if it's water cooled, temps are already gonna be more than good enough with TG-PP10. UX Pro would be more important for air cooled 3090's and whatnot. Anything besides air cooled GDDR6X cards, It would be a waste of money. The other options like U6 Pro, ZT-PY6, CX-H1300, are plenty good enough.

  • Thanks 1
Link to comment
Share on other sites

2 hours ago, T.Sharp said:

Interesting that the 4090 likes some heat, I wonder what could cause that. I've heard of cold bugging with LN2, but at ambient temps that's surprising. Either way, if it's water cooled, temps are already gonna be more than good enough with TG-PP10. UX Pro would be more important for air cooled 3090's and whatnot. Anything besides air cooled GDDR6X cards, It would be a waste of money. The other options like U6 Pro, ZT-PY6, CX-H1300, are plenty good enough.

...the VRAM 'idiosyncrasy' is kind of weird for sure. The 3090 and 4090 both have 24 GB of GDDR6X, but with the former, it is 24 x 1 GB chips (double-sided) while the 4090 has 12 x 2 x GB chips. The 2 GB chips are faster right out of the box but as you mentioned, in LN2, you can get a cold-bug. In an extensively water-cooled setup, they work just fine even when cold (at ambient) but there is a whole lot more additional OC performance to be had once VRAM is in the mid-50 C range. Below is a Superposition 8K run with normal temps per blue box -聽 with the temps at 55 C and a bit more, it would have been > 12,200 MHz. At the same time, the core will start to downclock more at such higher temps due to the temperature link in the boost algorithm. This is why some LN2ers actually have heaters wedged onto the 2 GB GDDR6X chips聽馃き

Superpos8K_J7Leaderboard.jpg.2f1b9b8860e4cd3bb6ad47012cc653ca.jpg

I am looking forward to try alternatives to the TG-PP10 you mentioned when the time comes - not so much for the temp differentials but because I just prefer thermal putty and my supply of TG-PP10 will run out sooner or later. Thermal putty is near-foolproof regarding applications (unless one is insanely generous with it) to conform to whatever space there is. It also tends to cool the surrounding area a bit more - when I applied it behind the 12VHPWR area of the PCB, it also helps cool really hot VRM components near the 12 VHPWR; the putty there bridges nicely to the backplate with its own heatsink and fans. That whole area can see spikes > 700 W and sustained PL of > 600 W, all in a tiny, finnicky package...馃お

  • Thanks 1
  • Respect 1

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

Posted (edited)

... concerning 'heavy' 4090s, another video released yesterday. BTW, I would love to have one of those machines which can lift / replace GPU and VRAM dies for Christmas.

聽 聽

Edited by J7SC_Orion

Owned

 Share

CPU: CPU: ><.......7950X3D - Aorus X670E Master - 48GB DDR5 7200 (8000) TridentZ SK Hynix - Giga-G-OC/Galax RTX 4090 670W - LG 48 OLED - 4TB NVMEs >< .......5950X - Asus CH 8 Dark Hero - 32GB CL13 DDR4 4000 - AMD R 6900XT 500W - Philips BDM40 4K VA - 2TB NVME & 3TB SSDs >> - <<.......4.4 TR 2950X - MSI X399 Creation - 32 GB CL 14 3866 - Asus RTX 3090 Strix OC/KPin 520W and 2x RTX 2080 Ti Gigabyte XTR WF WB 380W - LG 55 IPS HDR - 1TB NVME & 4TB SSDs
Full Rig Info
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...

Important Information

This Website may place and access certain Cookies on your computer. ExtremeHW uses Cookies to improve your experience of using the Website and to improve our range of products and services. ExtremeHW has carefully chosen these Cookies and has taken steps to ensure that your privacy is protected and respected at all times. All Cookies used by this Website are used in accordance with current UK and EU Cookie Law. For more information please see our Privacy Policy