I don't want to discourage you on this, but:
I'm curious - what's the use case here?
EDIT: To be clear, I don't mean what's the use case for using screen sharing/VNC on Mac OS X, that seems clear enough to me, my question is - what's the advantage/need of putting a newer VNC server on these older machines?
Just a couple weeks ago I started using VNC to drive "v2ls" - the mini that is eventually going to become vtools - from my main Windows PC on my LAN, and it's "fine". Because it's not encrypted and because VNC is quite inefficient, I would never think to send it outside my LAN. It was a mistake to try when I did back in the early-mid 2000s. Even with a 20 megabit upload, I can't image it it would be "good".
(In fact, I considered seeing about getting a mini g4 to VNC into to run the v2ls and vtools admin tools, using classic mode, since it's way more convenient than having to go to one of my vinage Macs where they're set up.)
If I need to do admin stuff on it when I'm away, I just RDP into my Windows box and then open a VNC client and that's "fine" - not really great because my upload dropped for unrelated reasons, but it does work. You could use a newer Mac to do this kind of relay, if you're more in on the ecosystem than me.
In general I haven't kept up with where Apple's own VNC/ARD implementation has gone and I also am not up on where VNC has gone in modern times. The low hanging fruit seems like it would be to use better compression on the signal and treat it as an h264 or h265 stream, but given the kinds of macs that run ~10.2-10.5 I don't know if that's necessarily viable, and the reason VNC was "usable" on a LAN in 2005 was that networking was good enough that sending not-well-compressed video but computing power wasn't really good enough to do a lot of compression on video.
At the time, RDP was way lower impact because what it did was (similar to X11) send instructions for drawing the screen rather than the entire screen picture, although RDP now mixes that with h264/h265 streaming in certian scenarios, which mostly has the impact of making video and gaming over it work.
I'm curious in general about the idea of replacing system bits, because an idea that was presented for new-vtools was to use something like tigerbrew or similar to set up and install newer versions of certain components for improved security on a vintage OS. I have yet to have a lot of time to look at that though.
If there's a lot of advantage to it, or if there are people who are using it this way and there's need for them to use 10.5 or older (as opposed to 10.6 even with rosetta) for some reason, then it's worth seeing if it's possible, can be integrated or can just run to the side of Apple's typical management/settings tools.