Ah yes, I forgot about that ResEdit extension! I actually already have it installed in one of my systems, but forget to use it.
In any case, I think I've got the search for the stack offset working, using Apple's rather strange method. There very well may be a better way of doing it buried in the assembly of PerformLib, probably somewhere in InitPerf, but that function is kind of a monster and without some kind of good decompilation I don't think I'd want to take a stab at figuring out everything it does.
I ended up writing an interrupt routine purely in assembly so that I could be in complete control of the stack, since otherwise it's difficult to tell the compiler not to be pushing things on to it. That's probably why my efforts at writing a C function to search the stack didn't work out so well. When the search is written in assembly it does seem to find a consistent offset to where the stored PC value is.
The strange thing, though, is that Apple's routine has one additional check in it: it goes through the stack word by word looking for one whose low byte is 0x1F. Then, upon finding it, it checks the next longword and see if it matches the address of the instruction doing the endless loop. If I put that check for 0x1F in there, it never finds the right offset, because that arrangement of bytes -- 0x1F followed by a longword of the stored PC value -- never occurs.
But that check works when linking to PerformLib. I wonder why that is? Could it have something to do with the fact that I'm not compiling with Apple's compiler? Or maybe (perhaps more likely) the "endless loop" trick is not a code path PerformLib always takes, and it does something more sensible to figure out the offset.