Copperweb, adventures in self-hosted proxies

[ClassicHasClass]

In a recent thread I mentioned I'd whipped up a proof-of-concept for a self-hosted proxy wherein the Mac would pull down web pages, preparse them, and then hand them to your browser of choice as a proxy. For example, you could strip JavaScript this way so that Netscape Navigator wouldn't have a snit fit, and you wouldn't be cheating by running the proxy off some larger more powerful system somewhere else. If you can't fix the browser, fix its input.

So here is that proof-of-concept:

http://www.floodgap.com/software/copperweb/

Note that this turned out to be a lot to ask from a 68K, even considering the overhead of Perl. Especially on MacOS, the browser can steal cycles from the proxy, starving itself for data (this is fortunately much better on A/UX) and the MacOS version doesn't work properly with all browsers for reasons I probably can't fix. On the whole, the whole shooting match drags things down to about two or three times slower, but it can access stuff it could never access before, so it may well be worth it.

There are both A/UX and System 7 versions. The A/UX version runs "conventionally" from CommandShell; the MacOS version uses MacPerl 4.1.8 (you can get a binary from the page too if you don't have one). Read the directions carefully as it's a bit finicky and there are quite a few drawbacks to consider.

As they say, it's not that the dog walked well. Comments (and patches especially) solicited.

 

[Bunsen]

Ooooh... have you tried it with browser6?

 

[ClassicHasClass]

No, I haven't. Can browser6 be pointed at a proxy? I didn't think it supported that.

 

[Bunsen]

No idea - I haven't tried it myself, and porter seems to be AWOL.

I just randomly stumbled onto this, designed for low-bandwidth links in the developing world:

http://www.aptivate.org/loband

Loband is a service that simplifies web pages, in order to make them download faster over slow Internet connections. Loband is designed to run on a server in a high-bandwidth area. Users in low-bandwidth areas browse to the small Loband website, and then request the site they want to view. Loband then downloads the site, reduces it in size, and then sends it on to the user. /Loband requires no software download to function, works on most websites, and is automated and open-source.

/ETA/ I'm trying it out in my current browser (FF on XP) - so far it's working really well. Instant early-90s web It choked on a form which requested a password.

 

[ClassicHasClass]

Copperweb doesn't really do that per se, though it does rip out style sheets and scripts. The trick is not to make it too complicated; it already uses a lot of CPU power.

 

[Bunsen]

Well, this is a remotely hosted service, so there's no overhead per se. Maybe someone with a working classic would like to try it. I'll repost in the relevant forum.

 

[yuhong]

This don't appear to do much more than disabling JavaScript and style sheets, and most old browsers has built-in support for disabling both, so no need for a proxy to do so. Very old browsers like Netscape 1 that don't support these technologies at all may show contents of these tags though, which can be annoying. The contents of an iframe element is fallback content I think, so stripping the elements completely would be incorrect.

 

[ClassicHasClass]

Congratulations on missing the point.

Yes, all this proof of concept does is that, but the point was to demonstrate that it's feasible, if not necessarily practical, for the Mac to preprocess its own input to a browser if you can't make the browser accept certain content. It could just as easily translate style sheets into colour tags, for example, although doing so for arbitrary content would require a lot of cleverness.

 

[yuhong]

And yea, some useful preprocessing I can think of is to convert UTF-8 to an encoding legacy browsers can handle. Only Netscape/IE 4 and later can handle UTF-8. Another example is to add a Host: header for very old browsers, and in fact I think jwz already has a script that can do exactly that.

 

[ClassicHasClass]

Exactly. The major limiting step is of course speed and multitasking, since the proxy has to compete with the browser it's feeding data to.

 

[Bunsen]

This would be ideal to run on a Radius Rocket, sending the output to the browser running on the host.

 

[ClassicHasClass]

It would. I don't remember on Rockets: they can open TCP sockets and act as servers, right?