The 8500/8600 can sense when a composite or s-video display is connected.
The DingusPPC emulator does emulate this behaviour.
The bits are in a board register accessed at address F301E000. It's a 16-bit little-endian register.
Read the register in Open Firmware like this:
Composite video is 0x0080 for not connected. 0 for connected.
S-Video is 0x0040 for not connected. 0 for connected.
Unused bits are pulled high: FF3F.
We should be able to see which is stuck at connected.
I don't know if there's any Power Mac 8500/7500/7600/8600 schematics that can help you find the related components on the motherboard.