TCP/IP for Classic Macintosh

[Dog Cow]

A few years ago I wrote a TCP/IP stack called Marina for Apple II.

I've spent the past two days looking into how to get TCP/IP on the original Mac 128K and 512K in a way that makes it accessible the greatest number of people, but also affords opportunities for application development: telnet, FTP, etc...

Hardware

The biggest problem to overcome seems to be link-layer connectivity. The historic route was to wrap IP datagrams within AppleTalk DDP packets, and route them to a DDP-> IP gateway such as Seagate, Kinetics FastPath, or Cayman Gatorbox (to name just a few routers of the day).

Classic AppleTalk DDP has an MTU of just under 600 bytes, which is about 1/3 of Ethernet's standard MTU, so we're limited on throughput...

But moreover, requiring AppleTalk now introduces additional requirements for hardware or routers. And AppleTalk itself is obsolete.

Trying to build a RS422-to-Ethernet adapter for the serial ports is one option, but again that's a hardware solution that will limit accessibility of the TCP/IP stack.

The solution then falls to using the serial ports as serial ports, and connect using SLIP or PPP. I didn't know anything about PPP until recently, and it seems to be the way to go. I installed MacPPP on my Mac SE/30 and analyzed the packets to see how the encapsulation works. I think a PPP implementation over 57600 baud serial will be accessible to the greatest number of people. That would allow for about 5 kilobytes per second throughput, about 4 times slower than LocalTalk.

There are still plenty of PPP daemons, and you can still readily get USB to serial adapters if your PPP server doesn't have RS232.

Software

Once the hardware connectivity is decided on, the next step is to write the software drivers. There is ElWhip written in C, but I think we need to write TCP in assembly to get the smallest possible code. There is already a TCP for Mac 128K and 512K, written in 1984-85, but it is written in Lisa Pascal (making it difficult to maintain) and uses AppleTalk. Thus it requires a DDP to IP gateway.

If we're not using AppleTalk DDP, that means we need to interface to the Serial Driver directly. I estimate a basic PPP implementation could be written in about 2K of assembly.

I looked at some other existing TCP/IP stacks: MacTCP and TopsTCP. The average size seems to be around 31K of code. MacTCP is designed as a RAM-based driver that is accessed using _Control calls from the application.

Let's say we write a really bare-bones, crunched TCP stack and it comes out to be 25K of code, plus the 2K for PPP.

If we're targeting a Mac 128K, that leaves about 128K - 38K - 27K = 63 Kilobytes. We also need room for buffers. Let's figure 8K for buffers. Now we're down to 55K for an application.

What can we do in 55K? Probably not a web browser. But we could probably do a small but respectable telnet. FTP would be more than possible. You could write a really simple Usenet newsreader. On a Mac 512K you could definitely do a text-mode, lynx style web browser. 

Motivation

So after this back of the envelope math, it looks like a TCP/IP with a small suite of practical applications is possible for the Mac 128K. And we already know this to be true, because Mark Sherman wrote a TCP for the Mac 128K in Lisa Pascal in 1984 with working Telnet and TFTP. So targeting a Mac 512K should be no sweat.

The last factor is motivation, or the actual applications. A web server isn't going to cut it. That gets boring after the first day or two. It's a lot of effort, but I could write a TCP for Mac 128K and 512K, and an application or two to go with it, but I don't want to do it if almost no one is using it.

Any thoughts or ideas?

 

[ScutBoy]

I couldn't help a stitch with any of the coding, but I would use a Telnet client on a 128/512k. FTP might be  handy as well.

 

[cheesestraws]

I didn't know anything about PPP until recently, and it seems to be the way to go.

Absolutely agree.  PPP is well-understood, standard, and most importantly, not going anywhere any time soon.  You're not locking anyone into long chains of vintage software either.

I've no idea what I would use this for, but it would be enormously cool.

 

[Byte Knight]

The last factor is motivation, or the actual applications. A web server isn't going to cut it. That gets boring after the first day or two. It's a lot of effort, but I could write a TCP for Mac 128K and 512K, and an application or two to go with it, but I don't want to do it if almost no one is using it.

Any thoughts or ideas?

Agreed that FTP and Telnet would be the biggest uses for TCP/IP.

 

[Dog Cow]

Thanks for the feedback; much appreciated.

But I've got some bad news, bears!  :-O  It turns out that PPP is not going to be such a walk in the park to implement as I thought it would be yesterday. This is not a show-stopper, but just some extra challenges for me the implementor to overcome.

It's story time....

So basically I spent all day trying to get my 68000 assembly PPP code to send even the most basic request packet to a ppp daemon, and failed every time. I accomplished nothing, except I gained some insight which I will share here.

Turns out that the RFCs are difficult to comprehend for someone who isn't already a serial communications guru (So, people like me, so far). What tripped me up was RFC 1662, which is how you're supposed to use PPP over a serial link. The title is "PPP in HDLC-like framing."

This RFC specifies the encapsulation for PPP packets over a serial link. Everything looked fine and dandy until I got to the FCS or CRC which is supposed to go at the end of the data transmission. This would involve a 1,024 byte lookup table needed to compute the 16-bit CRC. This would also entail additional overhead in software to iterate over every byte in the send buffer and escape certain bytes (such as the $7E flag). Well that was going to blow my whole estimate of getting a minimal PPP coded in about 2K of assembly code.

Is there another way?

Hey, I Found a Secret Shortcut

I had no idea what HDLC meant, but after some research in the Zilog Z8530 SCC manual, it turns out that the SCC has an SDLC mode which is basically the same thing. And after some more reading, I found that the SCC in SDLC mode will automatically send the flag $7E byte, will escape certain sequences in the data transmission, and will even compute the CRC in hardware, automatically! This looked really awesome! 

I just found a cool shortcut to solve most of my problems, right?

It looked like smooth sailing so far. I disassembled both FreePPP and MacPPP to see how they handled HDLC and the CRC. They were both doing it manually, in software. So is ElWhip, which has the 1,024 byte CRC table embedded in it.

Aha! I was so smug. I thought I'd found a neat shortcut that these other guys hadn't. All I need to do now is figure out how to configure the SCC for SDLC mode, and I'll get the flag byte and the CRC calculation done "for free" in hardware.

After another hour and a half or so of reviewing the Zilog SCC manual and some MacTutor articles on SCC programming, I'd put together about 2 dozen lines of assembly to put the SCC into SDLC mode. I added these lines to my PPP code, and.....

Still nothing! What went wrong? I was getting garbage data received on the other end, the ppp daemon.

The Big Revelation

I went back the RFC. I went back to the PPP source for lwip and ElWhip. How on earth are these implementations transmitting their PPP packets????

And I was met with a big revelation and disappointment.

PPP over serial is "HDLC-like". Meaning that the packet encapsulation only resembles SDLC with its flag and CRC. But it's still an asynchronous link where each 8-bit byte is wrapped in a start and stop bit(s).

To repeat: PPP over serial is not SDLC. LocalTalk is in fact SDLC, and I'd just figured out how to configure the SCC for it, as well as how to enable 115200 and 230400 baud rates.

Back to Square Zero

So I'm back where I started. I do in fact need to implement a CRC, escaping the $7E flag sequence, and some other stuff in my PPP implementation, just like everyone else. A couple times today I thought I'd have to write my own custom serial driver. I'm still undecided on that. If I wanted additional performance, I could calculate the CRC and escape the data as I push each byte out to the SCC. That would be a custom send routine.

For now, my target is to have a bare-bones and functional PPP implementation, regardless of how performant it is.

So the result is that I have some additional challenges to overcome before PPP is working. If I press on I think I could have it done before the end of this month April.

After that, IP, ICMP, and UDP are easy to implement, and could be done in probably 2 or 3 days each (when written by someone who has already written a TCP/IP stack). TCP is only as difficult as you want to make it. You can take some shortcuts and make TCP a lot simpler to implement at the expense of reduced performance. DNS is also pretty simple if you only care about resolving A records.

I looked at development docs for MacTCP, and I think I'll model the API to look like MacTCP's API.

 

[mactjaap]

Great story! I maintain MacIP.net so you can imagine I hope you will succeed! I start following this topic closely!

 

[cheesestraws]

PPP over serial is not SDLC

The useful thing about PPP, though, is partly that it is precisely not SDLC, so you can basically shove it over anything you can get bits down in an order (see also PPPoA, PPPoE, and half the dodgy VPN protocols in the world).  So this perhaps could open up all kinds of funky things to do later (I'd love to play with PPPoSCSI at some point, that sounds hilariously mad) while not requiring you to go back and radically gut your PPP implementation.

 

[Dog Cow]

Great story! I maintain MacIP.net so you can imagine I hope you will succeed! I start following this topic closely!

Your site is a great resource, by the way, and was part of the inspiration for me to work on this project.

I have good news now. Today was much more productive. Now that I understand PPP's low-level framing and data encapsulation, I rewrote my code and at 3:31pm this afternoon sent the first successful Configure-Request to a ppp daemon. It sent an Ack back to me, but I don't have the receiving code written yet to do anything with it, so that will be this week's project.

I am currently at about 1,804 bytes, so still under my 2K estimate of a few days ago.   But of course it's not complete yet.

With some hard work I might be able to have ICMP ping working in a few weeks. 68000 assembly is so much nicer than 6502 on the Apple II, by the way. 

 

[Dog Cow]

The useful thing about PPP, though, is partly that it is precisely not SDLC, so you can basically shove it over anything you can get bits down in an order (see also PPPoA, PPPoE, and half the dodgy VPN protocols in the world).

Yeah, I was bewildered by the number of variations on PPP. There's even PPP sent through UDP datagrams. But I have a much better understanding of it now, so I know where I'm headed with this.  :b&w:

 

[lisa2]

Great! Please keep working on this.  I would like to connect Lisa's running MW+ ( 1MB RAM and system 6.03 ) to a serial PPP server.  My problem is MacPPP works best with System 6.07 and better (that the stock Lisa can't run) and eats up a lot RAM ( Lisa has 1 MB RAM, but MW+ uses over 256K,  leaving less that 700K usable).  Also, the stock 5Mhz Lisa runs a little slower than a Mac, causing issues with some PPP clients internal timing. 

I need a very small, fast serial PPP client that will work with System 6 or System 5.

Something like Marina for 68K...

Thank you,

Rick 

 

[Dog Cow]

Great! Please keep working on this.  I would like to connect Lisa's running MW+ ( 1MB RAM and system 6.03 ) to a serial PPP server.  My problem is MacPPP works best with System 6.07 and better (that the stock Lisa can't run) and eats up a lot RAM ( Lisa has 1 MB RAM, but MW+ uses over 256K,  leaving less that 700K usable).

You know I was thinking about MacWorks earlier this week. I always develop my code according to Inside Macintosh, and I do not plan on accessing the SCC hardware directly. My TCP/IP should be compatible with a Lisa 2 running MacWorks, and it will certainly be compatible with System 6 and System 7. In fact, I ran it a couple evenings ago on a Power Mac G3 running OS 9.2.2 and it sent out the Config-Request packet.

I am now refactoring my code to implement the PPP Automaton in a more robust way. This morning I added support for address and control field compression, and protocol field compression. It doesn't take a whole lot more code to implement these common options, so I'm going to do it.

My plan is that the code that establishes a PPP connection using LCP and IPCP will be in a separate code module, and will be jettisoned from memory after the link is established.

 

[lisa2]

Dog Cow,

Great!

I would be happy to test your project on the Lisa when you are ready.

Thanks,

Rick

PS. I have a copy of the source code for MacPPP 2.01 if it would help you, PM me.

 

[Dog Cow]

I would be happy to test your project on the Lisa when you are ready.

PS. I have a copy of the source code for MacPPP 2.01 if it would help you, PM me.

Yes, I will be distributing a beta copy of the PPP code for people to test. I will probably have something in early May. This weekend, I plan to get enough of the PPP Automaton written to establish a PPP connection.

Yes, I have been looking for this source code.

 

[Dog Cow]

Good news guys,

It's not vaporware!

I fully expect to have a beta to distribute by this weekend.

 

[ScutBoy]

Nice!

 

[cheesestraws]

Very nifty!

 

[demik]

Awesome  Nice work !

 

[Dog Cow]

Thanks everyone. I know that's not a terribly exciting screenshot by itself, but the fact of the matter is that my PPP will negotiate a connection with some other PPP's I've tried it with, and also itself. I just ran it on a Power Mac G3 running OS 9.2.2, connected to itself on a Mac 512K and it connects.

Have a few more bugs left to work out, so I'll be doing that and ought to have something to share by Saturday or Sunday.

Once PPP is done I'll stop to think about further architecture/design, and then I'll start on the fun part: IP, ICMP, UDP, and TCP.

 

[cheesestraws]

the fact of the matter is that my PPP will negotiate a connection with some other PPP's I've tried it with, and also itself

The first few link-up events are always exciting.  Must feel good