Funny thing is, I always thought of GNOME as being "OS X equivalent toolkit for Gnu/Linux" and KDE as being "Windows equivalent toolkit for Gnu/Linux" XFCE seems to me to be "SGI equivalent for Gnu/Linux" and is designed in such a way as to likely be the most compatible without requiring RandR -- most of what it does is just improving on X itself without adding extensions. Then there's LXQt, which does have extensions, but tries to keep them as minimal and focused as possible (and limited to Qt library support) -- it's essentially a modern, better architected alternative to LXDE.
When testing with MacX, I'd go in the order of:
Basic X
XFCE
LXQt
GNOME 2 (including MATE)
KDE Plasma
GNOME 3 (including Cinnamon)
You could also try something niche like OTDE (which is based on KDE 1), but most modern software won't be compatible with it.