Use cases for a new AppleTalk stack

[robin-fo]

Hi everyone,

As some of you might be aware, I'm quietly developing new AppleTalk stacks for a year or so now. There is one targeted exclusively to embedded devices with limited resources, therefore only a subset of the possible functionality will be available. Currently, only AEP and NBP responders are implemented, with ATP following next.
The other stack is for bigger systems, supporting multiple network interfaces, routing etc. and currently much more progressed. However, I concluded that this bigger stack needs major rework to let it better integrate itself in other networking software like LwIP or an operating system kernel. In the meantime, I had a look at Netatalk 2 again and I surprisingly realized that porting it to modern macOS (which is one of the goals I'm following with the development of my own stack) might be much easier than I originally thought. This has now left me unsure on how I should continue with my projects and what their target use cases shall be.

I would therefore like to ask you for ideas, dreams and suggestions about where you could see uses for AppleTalk in our current modern+retro computing multiverse.

Below are some ideas I have on my own:

1. A modern desktop (and maybe mobile) App allowing you to inspect your local AppleTalk internet, showing all machines, zones and services as well as offering services like time synchronization, message exchange, basic file sharing and printing.

2. Some AppleTalk IoT devices for home automation (I even consider AppleTalk as being surprisingly well-suited for this) with data exchange done by the to-be-invented "AppleTalk Edge Telemetry Protocol" which might be similar to MQTT. I would however need to rely on others to develop native control applications for the classic MacOS.

3. Dongles allowing you to attach modern devices to vintage networks or vice-versa (e.g. for printers).

4. Build modern ports of network multiplayer games like Super Maze Wars.


What do you think?

 

[cheesestraws]

3. Dongles allowing you to attach modern devices to vintage networks or vice-versa (e.g. for printers).

I have a sketch design here for a USB LocalTalk box based on TashTalk, but there wasn't any software to wire it into. It would be actually really rather simple hardware, and being able to plug modern computers into LocalTalk to get files onto old Macs would be rather cool. Didn't have the energy to think about the software, though.

Other potential use cases:

5. Take mac-minivnc and port it to use ATP instead of TCP. Then port a minimalist VNC client to use ATP on the modern computer. Partial replacement for Timbuktu, now that the client doesn't run any more on newer machines. VNC is client-driven request-response anyway, so ATP is a reasonably decent potential backing protocol for it.

6. Being a codebase that is maintainable and readable enough to act as "living documentation" in a way that netatalk2 cannot.

Thoughts on netatalk2:

The thing about netatalk2 is that it is a big system designed to run big networks. It is designed to go fast and be highly flexible, which is great - but unless you're a massive AppleTalk nerd, you don't need all those features and that going fast and being highly flexible. Things like: it's never felt really happy to me running without a router. So my plan, back when I optimistically thought my run of working executive function would continue, was to try instead to build something less featureful and optimised but more user-focused.

That is not, of course, to denigrate any of the work that's gone into netatalk2, especially recently. It's good stuff being done there. But if the community were to start from scratch, building something that actually fit the needs of hobbyists as well as possible, I don't think the end result would necessarily much look like netatalk2.

 

[pax]

I think all of these use cases have merit.

I don’t know if it qualifies as an additional use case, but let me tell you about a set up that I think would be useful for many enthusiasts, and also perhaps would make it easier to introduce new people to networking on retro macs.

I think it is reasonably common to have a modern mac server, say a mac mini, running various jobs like file server, streaming server, automation, proxy, etc. Most people these days typically have a laptop, and the server is in addition to the laptop and allows them to offload tasks that either requires the computer to run at all times, or do things like video transcoding to serve up media to laptops, iPads, etc. With so much of our data these days being synced to the cloud, is it also a typical solution for hosting a local copy. For instance, I’m guessing many people who use Apple photos can’t have all photos stored locally on all devices, and a mac server is an excellent solution for being the machine where all photos are synced locally in full resolution (the “download and keep originals” option). Same might be true for email, iTunes media, etc. The modern mac server has the advantage of working with all the modern tools available for backup, like Backblaze, Time Machine, SuperDuper!, etc. So everything is safe all the time.

Now, I’d love for this mac server to also be the server for my retro macs. Just like it shares files natively with modern Apple devices, it would share the same files natively with retro mac. No VM or Linux server off to the side running Netatalk with files in AppleDouble format, but the same file server serving modern macs, stored natively on APFS. Ideally, it could also share a printer to retro macs. So it’s one piece of software that you install on your modern mac server and it can serve both your new and your old computers. You can have both your old files, created with retro macs, and your new files created on modern macs, available to all computers at the same time, from the same location, backed up using modern tools and services. Obviously, the mac server can also serve the same files using other protocols, like SMB, if you so choose.

Retro macs should be able to connect using either Ethernet or LocalTalk. I think there’s a version of this future where you can pair this with cheesestraws’ excellent AirTalk devices and you have a single wireless network serving all of your modern and retro networking needs.

 

[Snial]

I was thinking about that today. For me the most obvious use-case for an embedded AppleTalk stack would be to use it as an interface for an SD memory card. Here, it would present itself as an AppleTalk server where files would appear as mounted volumes and the client Mac (e.g. my LCII or PB1400) could then connect, read and write to those volumes. It'd then be easy to exchange data with any other device and wouldn't be as bulky as SCSI to SD (it would have a similar form factor to an actual USB memory stick). The only downside is that it'd be pretty slow, but that's the nature of AppleTalk anyway!

I have one condition for this: that it wouldn't be targeted at an 8-bit PIC, an ARM MCU seems more appropriate to my mind. For example, an RP2040 would easily manage it since it already has about 2MB of storage available for the application stack. Pin count is not important (we'd just need 4 pins for the SD interface and RX/TX for AppleTalk).

 

[robin-fo]

I have a sketch design here for a USB LocalTalk box based on TashTalk, but there wasn't any software to wire it into. It would be actually really rather simple hardware, and being able to plug modern computers into LocalTalk to get files onto old Macs would be rather cool. Didn't have the energy to think about the software, though.

Cool! Do you already have hardware or is it just a concept?

5. Take mac-minivnc and port it to use ATP instead of TCP. Then port a minimalist VNC client to use ATP on the modern computer. Partial replacement for Timbuktu, now that the client doesn't run any more on newer machines. VNC is client-driven request-response anyway, so ATP is a reasonably decent potential backing protocol for it.

What about reverse-engineering Timbuktu? It also uses ATP but its own protocol is certainly difficult to understand and I guess there is no documentation. I never played around with Timbuktu before, and now I have issues thanks to having to use the same serial number for multiple devices. Adapting mac-minivnc is certainly much easier.

6. Being a codebase that is maintainable and readable enough to act as "living documentation" in a way that netatalk2 cannot.

Yeah that's a true point. However, improving and documenting Netatalk itself could be another way to do this.

The thing about netatalk2 is that it is a big system designed to run big networks. It is designed to go fast and be highly flexible, which is great - but unless you're a massive AppleTalk nerd, you don't need all those features and that going fast and being highly flexible. Things like: it's never felt really happy to me running without a router. So my plan, back when I optimistically thought my run of working executive function would continue, was to try instead to build something less featureful and optimised but more user-focused.

What I 'dislike' about Netatalk (at least from the perspective of portability) is that it bundles many additional functions and options into one package (this is somehow similar to LwIP). This is great on a Unix system, but not when you just need a tiny subset of its functionality and want to resolve issues or when you try to get it running on a completely different system.

building something that actually fit the needs of hobbyists as well as possible

This is why I am asking

Now, I’d love for this mac server to also be the server for my retro macs. Just like it shares files natively with modern Apple devices, it would share the same files natively with retro mac. No VM or Linux server off to the side running Netatalk with files in AppleDouble format, but the same file server serving modern macs, stored natively on APFS. Ideally, it could also share a printer to retro macs. So it’s one piece of software that you install on your modern mac server and it can serve both your new and your old computers. You can have both your old files, created with retro macs, and your new files created on modern macs, available to all computers at the same time, from the same location, backed up using modern tools and services. Obviously, the mac server can also serve the same files using other protocols, like SMB, if you so choose.

Sounds like an ideal application for Netatalk on macOS. However, I don't know if there could be any conflicts when files are accessed by multiple file server applications access files at the same time. So +1 for Netatalk2 on modern macOS.

For me the most obvious use-case for an embedded AppleTalk stack would be to use it as an interface for an SD memory card. Here, it would present itself as an AppleTalk server where files would appear as mounted volumes and the client Mac (e.g. my LCII or PB1400) could then connect, read and write to those volumes.

+1 for a tiny embedded AppleTalk stack with minimal AFP support. You would need some sort of external power supply (e.g. from ADB port) though.

I have one condition for this: that it wouldn't be targeted at an 8-bit PIC, an ARM MCU seems more appropriate to my mind. For example, an RP2040 would easily manage it since it already has about 2MB of storage available for the application stack. Pin count is not important (we'd just need 4 pins for the SD interface and RX/TX for AppleTalk).

Yeah 8-bit is outdated in my opinion.

 

[cheesestraws]

RX/TX for AppleTalk

Let's not conflate our layers here. An AppleTalk stack is not the same thing as a LocalTalk/SDLC transceiver. If the 8-bit microcontroller thing is a bit of a dig at Tashtalk, then you're fundamentally misunderstanding what's going on here: the signalling necessary to do transmitter-self-clocked SDLC over RS422 is almost totally distinct from AppleTalk as a protocol. Tashtalk does the former, it is not a protocol stack. Even AirTalk only decodes frames, not packets. The distinction matters.

 

[robin-fo]

I guess he meant something like "RX/TX for an external LocalTalk phy", which would be correct and not really far off a realistic hardware design.

 

[Snial]

Let's not conflate our layers here. An AppleTalk stack is not the same thing as a LocalTalk/SDLC transceiver. If the 8-bit microcontroller thing is a bit of a dig at Tashtalk, then you're fundamentally misunderstanding what's going on here: the signalling necessary to do transmitter-self-clocked SDLC over RS422 is almost totally distinct from AppleTalk as a protocol. Tashtalk does the former, it is not a protocol stack. Even AirTalk only decodes frames, not packets. The distinction matters.

I think what Tashtalk is doing is really great, it's 8-bit PICs I prefer to avoid, and this is mostly because the architecture is so awful, but also because there's a lack of decent open-source compilers / development tools.

I guess he meant something like "RX/TX for an external LocalTalk phy", which would be correct and not really far off a realistic hardware design.

It's more like, I was never quite sure how the personal AppleTalk physical layer worked when you used it to file share two Macs with just a serial cable under System 7. My guess is that it's still SDLC over RS422, but there's no need for any collision detection hardware or transceivers.

Using an ARM Cortex based solution would offer a lot in terms of flexible implementations, but which ones to use would depend upon the size of your AppleTalk stack. Any ARM should be fast enough to be able to bit-bang SDLC at 230400 bits/per second (assuming that's what happens in this case). I'd forgotten that a standard mini DIN serial port doesn't have power, but maybe it can be stolen from the Mac's TX+ / TX- since one of them is always going to be high.

I write MCU firmware for a living so it's not beyond my ability to help with this (I don't do much hardware design, but a serial to MCU to SD dongle should be within my ability).

 

[uyjulian]

Sounds like an ideal application for Netatalk on macOS. However, I don't know if there could be any conflicts when files are accessed by multiple file server applications access files at the same time. So +1 for Netatalk2 on modern macOS.

Actually, (at least on Linux) it is possible to point all of a FTP server, a NFS server, a SMB server, and a AFP server at the same location at the same time.

So, probably in macOS, running the existing file server and Netatalk2 on the same directory shouldn't be a problem.

 

[slipperygrey]

Actually, (at least on Linux) it is possible to point all of a FTP server, a NFS server, a SMB server, and a AFP server at the same location at the same time.

So, probably in macOS, running the existing file server and Netatalk2 on the same directory shouldn't be a problem.

It is definitely possible to do this (I've done it), but there are corner cases that won't work so well. Notably, the other file sharing schemes aren't aware of netatalk2's appledouble metadata (that houses the resource forks) so moving a Mac file around with them will cause breakage. And secondly, Samba has mandatory file locking that can mess with other file sharing schemes. Netatalk3 has improvements for coexisting with Samba that netatalk2 don't.

But again, for very basic operations a setup like this will work alright.

 

[Chopsticks]

i think for the majority or users using appletalk they mainly want to be able to share files from modem computers to retro macs mainly system 6 onwards. due that i think that's why so much effort and work has been done in updating netatalk 2.2.x. my current understanding in that the plan is to oneday merge the code with the main 3.x branch. i have no idea exactly what discussion has been done regarding that only that is mentioned several times around the web.

with that in mind probably the biggest problem to solve and im no coding expert but its removing the appleshare.so kernel driver and moving it into a userspace process. once this is achieved it should enable a much easier ability to write more agnostic code that could be used on a mcu right through to PC/Mac with much more ease.

a side effect of doing so would be having a unified code base across systems/platforms allowing fast code development and also allowing for simpler solutions such as just sharing files on a home network without dealing with the complexity associated with netatalk being designed for large network systems.

 

[Chopsticks]

also i think some sort of config GUI for netatalk 2/3 that can be used to setup shares and config the basics like netbios, users, permissions. etc in a much more 'Human Interface' way would be a massive improvement. perhaps being written in python and being able to autodetect the running OS for config file locations etc would be the way to go.

 

[slipperygrey]

also i think some sort of config GUI for netatalk 2/3 that can be used to setup shares and config the basics like netbios, users, permissions. etc in a much more 'Human Interface' way would be a massive improvement. perhaps being written in python and being able to autodetect the running OS for config file locations etc would be the way to go.

FWIW netatalk does have a config GUI in the webmin module: https://github.com/Netatalk/webmin-module

It's using ancient tech of course, and I haven't even attempted to set it up yet.

 

[Chopsticks]

FWIW netatalk does have a config GUI in the webmin module: https://github.com/Netatalk/webmin-module

It's using ancient tech of course, and I haven't even attempted to set it up yet.

id totally forgotten about webmin, i havent used it since the early/mid 2000's...
i dont really have much of an issue using config files myself but im sure many people dont feel as comfortable doing that.

something else that could be useful as part of the install process would to to make sure the appletalk kernel module isnt blacklisted and is set to load on boot. considering that when building the source it already detects the running system and init services like systemd etc im wondering if this is possible while while 'make install' has elevated sudo permissions?

since most distros seem to blacklist the appletalk module when installing netatalk2 if will install but just fail to run. it takes a bit of google kung-fu to figure out the right fix. im used to loading and unloading modules so i had a good idea pretty quickly but for those not experienced in the underlying linux/unix system this isnt easy to figure out.

so if adding this to the installer isnt possible/easy/preferred for wehatrever reason then some better instructions and info in the readme or something i think would be quite helpful

 

[NJRoadfan]

Which distros are blacklisting the kernel module? Both Debian and Ubuntu seem to work fine in their latest releases. Debian comes with IPDDP set to "Yes" which breaks macipgw, but otherwise works fine. Ubuntu just needs it enabled.

sudo modprobe appletalk

should all you need to start the kernel module.

 

[Chopsticks]

Fedora 37/38 both have the AppleTalk driver blacklisted, my understanding is that it’s commonly done now but maybe that’s just on RHEL based distros then.

I had no issues building or setting up netatalk 2.2.x. When I didn’t work after initial setup I just checked if the kernel module was loaded and it wasn’t so I loaded it but it still wasn’t working so I checked the blacklist file and commented out AppleTalk. All of those things weren’t complex to me I was more pointing out that the average person might get stuck if this happens and I didn’t see any clear info anywhere explaining how to fix that.

I did see a lot of posts that showed the same error I got but the answers had nothing to do with the actual issue

Tbh I’m just happy I can network from system 6 onwards again to my main Linux system. Not really needing any assistance with running it but thank you.

Also on fedora you probably will need to open a port for afp to work fwiw

 

[slipperygrey]

id totally forgotten about webmin, i havent used it since the early/mid 2000's...
i dont really have much of an issue using config files myself but im sure many people dont feel as comfortable doing that.

something else that could be useful as part of the install process would to to make sure the appletalk kernel module isnt blacklisted and is set to load on boot. considering that when building the source it already detects the running system and init services like systemd etc im wondering if this is possible while while 'make install' has elevated sudo permissions?

It's not a bad idea to have autoconf check for the Linux kernel module when configuring netatalk with DDP support. It'll have to be done in a cross platform friendly way though since we're also supporting appletalk on NetBSD and Solaris. Feel free to raise a feature request ticket over at the Github project.

 

[Chopsticks]

It's not a bad idea to have autoconf check for the Linux kernel module when configuring netatalk with DDP support. It'll have to be done in a cross platform friendly way though since we're also supporting appletalk on NetBSD and Solaris. Feel free to raise a feature request ticket over at the Github project.

at the very least some verbose feedback during compile to let the user know.

im not really sure how to raise a ticket in github, but ill look into doing that