G5 Quad w/rebuilt LCS will not stop overheating

[ScorpionLips]

Back in December I bought G5 quad off craigslist

At a glance, it was a mint unit - clean as hell, hadn't leaked a drop of coolant. Got it home, tore it apart - LCS, CPU cards, logic board and PSU were all immaculate.
After a fresh install of Leopard, I installed iStat and started monitoring CPU temps. Right off the bat, noticed CPU B was getting way hotter (by like 20-25C) than CPU A.

I'd heard flushing/rebuildling the dual Delphi systems was a nightmare, plus I wasn't really sure if I cared, since it wasn't getting hot enough to trigger a shutdown.
After 3 or 4 days of use, the machine started shutting down and I got the dreaded overtemp LED. I decided at this point to go ahead with CircuitBored's New Blood Mod guide, and ordered all my parts.

First thing I noticed after tearing the glue and hoses off was that each loop was about 50% air. Again, it had leaked zero dexcool, so I'm assuming this was just from 16+ years of evaporative loss. Did a pretty thorough flush with a siphon pump and a couple gallons of distilled water and vinegar mixed 50/50. Flushed both loops in both directions. Didn't get a ton of dexcool gunk out, but the system didn't feel restricted/blocked in any way when I was pumping vinegar through it, so I called it good after about 30mins per loop.
During refill/reassembly I followed CircuitBored's guide to a T, and didn't deviate from it in any way.

I got the machine back together and lo and behold it booted, but still overheated. Thermal calibration came back with two CPU fails - a "maximum temperature exceeded" message for both. Tried calibration both with and without the air baffle, made no difference. I pulled the CPUs and repasted them with different compound, no difference. Sans calibration, the system would boot into Leopard fine, but would overheat/restart about 30 seconds after getting to the desktop.

At this point, I noticed the new coolant didn't really look like it was moving/circulating at all. I couldn't tell if the pumps were spinning since the red XSPC EC6 is pretty opaque. So, after sitting on the project for a few months, I tore it all back apart.
With the CPUs out and LCS drained, I connected the pumps to the logic board with the tops removed, and powered the machine on. Sure enough, they turned over, and the motors were pretty strong. I figured at this point the circulation issue had to be from me not getting all the Dexcool sediment out.

Over the course of two nights, I did a vinegar flush, followed by a citric acid flush, of both loops. With each flush, I spent about 2hrs per loop, so like 8hrs total. I went through about 2 gallons of vinegar, and as much citric acid as I'd normally use on a car radiator. Like the first time, I didn't get a ton of gunk out, but, by the end, the water coming out was 100% crystal clear. And again, I felt no resistance at all.

During the second reassembly, I went the extra mile and replaced the o-rings that hold the CPU dies to the copper cooling blocks, which others have said are superfluous to the design. Figured it couldn't hurt.

And, upon reboot, it overheats even worse than before. The machine chimes, goes to the Open Firmware boot menu, and then freezes at the folder icon, before the ASD DVD can even load, and the overtemp LED comes on.

Is there anything I could possibly be missing here? Anyone?

At this point, the only thing I can think of is maybe the CPUs are just f*cked from being overheated over and over for years. I have two new 970MP boards on the way, and if those don't work, I'm pretty much out of ideas

 

[herd]

If it is only one CPU that overheats, what happens if you swap the positions of the CPUs? Does the high temperature follow the CPU or stay with the water block? You could also stick a thermocouple on the CPU (on the back side maybe?) and see if it's actually overheating or the machine just thinks that it is.

I have never owned or worked on a G5, so I'm no expert on these. The tenfourfox blog also has an article on maintaining the water cooling setup. Good luck and let us know how it goes...

 

[CircuitBored]

Welcome to the MLA! Sorry to hear that you're having trouble with your Quad. It is undeniably one of the most divisive models of Mac ever made but they are a great testament to that weird moment in Apple's history.

I second @herd's recommendation of swapping the CPU cards but this is likely just going to be an eliminatory step. I doubt it will fix anything.

One important thing to note with all G5s is the importance of the backside fan. This is an axial fan which actually sits behind the fan that blows over the HDD cage. You can't actually see it with the machine assembled. It is responsible for pushing air around the back of the logic board (hence the name) and when/if it fails it has disastrous consequences for the logic board. I strongly recommend that you check that this fan is working and that the backside of the logic board is not clogged up with dust. Sadly this means removing pretty much every component of the machine. It's a really big job but it's not actually very difficult, provided you are fastidious about keeping track of all the screws and bolts.

While you have the logic board out of the machine you should inspect it thoroughly and replace the thermal paste on the various heatsinks that hide back there. If you've ever noticed that the G5's logic board appears to have very few chips on it then the reason for that is simple: Apple decided to put everything on the "wrong" side.

If that procedure proves unfruitful then you might want to consider getting your hands on a replacement power supply. It may sound like an odd step but I was actually having fairly similar trouble with my Quad when I first got my hands on it and a PSU replacement sorted it right out. The logic board just goes a bit bonkers if it doesn't get enough juice.

Hopefully this was helpful. I wish you the best of luck with your endeavours! Quads are getting harder and harder to come by and working examples even more so. Please do keep us updated on your progress.

 

[ScorpionLips]

If it is only one CPU that overheats, what happens if you swap the positions of the CPUs? Does the high temperature follow the CPU or stay with the water block? You could also stick a thermocouple on the CPU (on the back side maybe?) and see if it's actually overheating or the machine just thinks that it is.

I have never owned or worked on a G5, so I'm no expert on these. The tenfourfox blog also has an article on maintaining the water cooling setup. Good luck and let us know how it goes...

In the beginning, when it was only one CPU that ran hot, and the machine wasn't overheating/restarting, I swapped the CPU cards and the problem stayed with the block. I've swapped them back and forth a few different times.
After the first LCS rebuild, I couldn't get into Leopard long enough to open iStat or Terminal to check CPU temps. All I knew at that point, is that the machine still overheated, and that both CPUs failed thermal calibration. So, sll I can assume is that now both CPUs are overheating. I'll probably try a thermocouple at some point

One thing I haven't tried yet is lapping the CPU blocks with 1500-2000 grit sandpaper. Not sure how big of a difference it would make.
Also, I could be crazy, but, I feel like the position of the CPU block O-rings does nothing to hold the blocks tighter to the CPUs. So maybe I'll try re-removing them

 

[ScorpionLips]

Lapped the blocks with 1500 grit sandpaper, removed the pointless o-rings, repasted CPUs with different compound, spent 20 or 30 minutes trying to burp any remaining air out of the LCS (there was none), started 'er up, no difference. Removing logic board now to repaste northbridge/check backside fan

 

[ScorpionLips]

Got the logic board out, repasted the northbridge as well as the ServerWorks chip (whatever that thing is) that's under the big black heatsink at the top front of the board. Checked and cleaned the backside fan
Got it all back together and it's the same. Bongs, then a few seconds later the overtemp and checkstop LEDs come on.

 

[ScorpionLips]

My two new 2.5ghz 970MP boards came today. I put them in to the exact same result.
I took one of the old CPUs and transplanted it into my dual core 2ghz system, it passed calibration and runs very nicely.
Im bout to say fuck this and just convert it to air cooled

 

[CircuitBored]

Im bout to say fuck this and just convert it to air cooled

If you decide to do this would you be willing to send me your LCS and original CPU cards so I can take a look at them? I'd love to get to the bottom of what the heck is going on here! Naturally I will gladly pay for them!

 

[Byrd]

I'm there right now (was it you on reddit that posted about the same issue?), there appears to be a blockage (not air) I need to investigate in both a single pump Delphi and dual pump Delphi units. I've replaced the pumps which are working hard but impeded by something in the loop, and my tolerance for cracking into the loops again taking apart the G5 is very low, so it's for another day. Like you, I'm tempted to just rip the lot out and modify a couple of tall PC heatsink assemblies for this purpose. I know the Mac doesn't give a shit if you have fans or water - as long as it stays cool.

With my recent work on the dual Delphi LCS even with crappy watercooling and the fans running on full turbine mode it doesn't shut down - temps are high but there are no red checkstop LEDs on mine, although this is in winter.

Don't reverse CPUs to the other slots, I believe this is a no-no

 

[CC_333]

I went through this myself with a Quad G5 I got several years ago, and it seems like nothing I tried would let it boot (it booted up exactly once, and it stayed relatively stable as long as I kept the performance setting under the Energy Saver system preference pane to "reduced performance" and basically avoided opening anything else.

Eventually, after replacing the CPU cards (as you have) and rebuilding the LCS (I had someone do it for me), I concluded that the logic board was somehow defective. I could've converted it to use air coolers, but I doubt it would've mattered, because my new CPUs and rebuilt LCS were supposed to fix it, and that doesn't fix it, what will?

Anyway, instead of fighting it any further, I got another Quad with a failed CPU. However, it was still booting and working normally, according to the seller, which was very promising because it meant the logic board was OK, and it only needed a new CPU and an LCS overhaul to be restored to fully working condition. To that end, I intended to transfer my presumed-working CPUs and rebuilt LCS into, but by the time I got everything worked out, I had to pack it up to move house, and subsequently lost any motivation to continue.

Anyway, with these two datapoints (both your and my logic boards overtemping and checkstopping for no reason), I'm starting to think there's a failed component on the logic board that nobody's thought of looking at yet, and reading your thread is motivating me to finally pull it out and see about fixing it once and for all.

c

 

[ScorpionLips]

If you decide to do this would you be willing to send me your LCS and original CPU cards so I can take a look at them? I'd love to get to the bottom of what the heck is going on here! Naturally I will gladly pay for them!

If I do wind up going the air cooled route, I'd be glad to send you the LCS. The cards I'd like to hang onto, since one is already sitting in my DC, and it'd be nice to have a spare...
I do have a 3rd machine (a DC 2.0 with a bad PSU) that I could yank the CPU from, if you want the LCS and two 2ghz cards

I'm there right now (was it you on reddit that posted about the same issue?), there appears to be a blockage (not air) I need to investigate in both a single pump Delphi and dual pump Delphi units. I've replaced the pumps which are working hard but impeded by something in the loop, and my tolerance for cracking into the loops again taking apart the G5 is very low, so it's for another day. Like you, I'm tempted to just rip the lot out and modify a couple of tall PC heatsink assemblies for this purpose. I know the Mac doesn't give a shit if you have fans or water - as long as it stays cool.

With my recent work on the dual Delphi LCS even with crappy watercooling and the fans running on full turbine mode it doesn't shut down - temps are high but there are no red checkstop LEDs on mine, although this is in winter.

Don't reverse CPUs to the other slots, I believe this is a no-no

Yep, that was me.

Something I noticed while moving one of the original 2.5ghz cards into my dual core, is that while the small heatsinks coming off the backside of the cards are different, the cards themselves are identical, so you can swap parts between them no problem. And the larger/main heatsinks, while very different looking, have (almost) the same bolt hole pattern as half of a quad LCS, so if you can correctly trim two of them to fit on the same logic board, and dremel out indentations where a couple center standoffs are supposed to go, they'll slap right on.

I'm actually really surprised with how well the DC 2.5 works. My iStat trial expired (lol) and I don't know how to monitor temps without 3rd party widgets, so I don't have exact CPU temps at the moment, but, it's not overheating, and the fans are running nice and slow, so...
Pretty stupid how Apple could have totally gotten away with making an air cooled quad, and it would have been more reliable long-term, but they just.... didn't.

So, what I'd like to do now (tentatively) is, track down a third 2ghz DC machine, harvest the heatsink out of it, combine that with the heatsink out of another broken DC machine I have, throw them in my quad, have a working quad 2.5 and a working DC 2.5, and just call it a day.
This is assuming the quad doesn't have a logic board issue, which would blow.

The air cooled guide on Macrumors makes the job look pretty daunting (dude uses a wire edm, takes measurements with a digital caliper etc), but if you look at a quad LCS and a DC heatsink side by side IRL, you can clearly see what needs to get chopped off and where. You could do essentially the same job with a fine tip sharpie, band saw, and a dremel

Anyway, couple screenshots of the dual 2.5

 

[Byrd]

There is a reason why Apple went water; the older G5 Apple heatsinks are adequate for light loads but are not good enough with the system under heavy load. That's why I intend to - one day - see if a hefty, off the shelf tall x86 heatsink can be adapted for the job if my experiments with water cooling upgrades fail.

All in all, a sick G5 isn't much fun to deal with. The lure of "fastest PPC" keeps me going, but goodness they're a horrible system to work on!

 

[Unknown_K]

Kind of funny but a 2.5ghz DC G5 only uses 125W max
going by this:

http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf

Which is no big deal to cool with an air cooler today.

 

[ScorpionLips]

Kind of funny but a 2.5ghz DC G5 only uses 125W max
going by this:

http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf

Which is no big deal to cool with an air cooler today.

Hell, in April 2007, less than 9 months after the G5 ceased production, you could order a Mac Pro with dual x5365s, each of which had a TDP of 150w, and they did absolutely fine air cooled.

 

[Unknown_K]

A little bit of copper and heat pipes would have gone a long way with the G5's. I mean they did fill the thing with cooling fans and ducted airflow already.

Water cooling is something I have never wanted to do in a PC because of leaks, cost, and long term issues with pumps and coolant.

 

[CircuitBored]

Which is no big deal to cool with an air cooler today.

It's not the power draw that's an issue, it's the surface area of the dies in proportion to the power draw. The 970mp is tiny compared to a modern desktop CPU. All that power in such a small package makes for a toasty experience, as we well know.

I'm sure some copper pipe magic could still yield good results but the fact remains that you are fighting against a fundamentally compromised design. I have often wondered if some sort of IHS could be designed to ease things along but there's also likely a reason Apple/IBM didn't do this themselves.

 

[ScorpionLips]

Alright, I eliminated the logic board as a potential cause

While sitting around brooding on it, I had a clever idea: remove the LCS and CPUs from my quad. Take the heatsink and 2.5ghz card from my air cooled DC, slap it in the top slot of the quad, see if it boots.
I did that, and it started right up. No overtemp LED, just a "CPU B Insert" light. I'm in ASD as I type this, and it just passed thermal calibration.
It's the LCS, 100%.
I realize this is a lot of me bumping my own thread with minor progress reports.
I'll check back in once I'm closer to having a working system

 

[CC_333]

Well, then, the solution to this problem has a simple enough solution: Get rid of the LCS and simply get another heatsink and CPU and make it air cooled!

I wonder if that would work on mine?

c

 

[CircuitBored]

I realize this is a lot of me bumping my own thread with minor progress reports.

Progress is progress! I for one am very interested in your tinkering. The more we know about G5 weirdness, the better.

 

[bkmoore773]

I'm doing the same rebuild, but have had much better luck. After the New Blood mod, I'm getting CPU A in the mid 40s, and CPU B in the mid 60s at idle. Not ideal. But it's a useable system and I'm not getting thermal shutdowns. I still cannot successfully complete thermal calibration.

Like your system, my liquid cooling system isn't working as efficiently as it should, at least on one of the CPUs. My theory is that the old coolant has left a thick layer of mineral deposits on the inside of the radiator and the heat transfer block. I may try ultrasonic cleaning of all the parts to try to break any residue off the inside surfaces. Just a thought as to why your LCS isn't working. You may have a much thicker build up. The other theory is if your CPU isn't making physical contact with the cooling blocks. That happened to me during reassembly, and now I carefully inspect the CPU before reinstallation in my computer. All four shoulder screws need to be flush with the back plate. If they're not, the pressure won't be even and the CPU won't make contact with the transfer block.