Netatalk 4.0 - Future-proofing Apple File Sharing

[slipperygrey]

ya i can usually find time to test things on the weekends. Usually the weekends are packed with things to test and do: last week I wrapped up Vicom Internet Gateway.

edit: Any changes to the codebase? I would expect it to be somewhat dated, but still usable. And simplifyable -- heard something somewhere about it having many ways to activate it.

You should expect macipgw to behave just like you're used to. I simply merged all relevant patches from the netatalk-classic and Jason King forks. They made the code more robust and 64-bit safe.

What do you mean by having many ways to activate it? Many ways to build the code, or many ways to run the macipgw app?

The biggest change that *I* made was to integrate macipgw with Netatalk's Meson build system, so that you can build it in one go with the rest of Netatalk. Previously, you needed to bring your own copy of Netatalk: either a compatible shared libatalk library to link with, or the entire source code.

 

[Mk.558]

Many ways to run it. I personally have never used macipgw. However, I'd be happy to look at it and test it out with ftp, AFP over TCP and some basic WWW stuff. If it works with System 6 and MacTCP 2.0.6 then I'd say it's good because that's generally the lowest spec.

edit: Saw the mention about using macipgw.conf. CLI flags to configure it IMO is messy and too nerdy. I suppose you could argue that I may have been not liking QEMU and the other CLI emulators because if you get a single CLI flag wrong or something is invalid it just quits with no error code and you have no idea what is wrong when you use the .bat script. I guess it's a function of how many flags you use: if it's got a large number of flags, then a .conf file is easier, but if it's fewer, then CLI only is fine. The idea of the .conf file is make it easier to work with webmin (i'm just spitballing here): you just have to edit a conf file and start the service with launchd or init.d. Nice and simple.

 

[slipperygrey]

Many ways to run it. I personally have never used macipgw. However, I'd be happy to look at it and test it out with ftp, AFP over TCP and some basic WWW stuff. If it works with System 6 and MacTCP 2.0.6 then I'd say it's good because that's generally the lowest spec.

edit: Saw the mention about using macipgw.conf. CLI flags to configure it IMO is messy and too nerdy. I suppose you could argue that I may have been not liking QEMU and the other CLI emulators because if you get a single CLI flag wrong or something is invalid it just quits with no error code and you have no idea what is wrong when you use the .bat script. I guess it's a function of how many flags you use: if it's got a large number of flags, then a .conf file is easier, but if it's fewer, then CLI only is fine. The idea of the .conf file is make it easier to work with webmin (i'm just spitballing here): you just have to edit a conf file and start the service with launchd or init.d. Nice and simple.

macipgw presently has 6 parameters (one of them a debug flag) which is right about the threshold where a config file makes sense in my mind. And in general, separating daemon configuration from its init script is arguably preferable.

The caveat with regards to macipgw is that is also requires faily complex network configuration, i.e. bridging and NAT. So it's not as simple as passing a bunch of parameters to macipgw and press start. You need to do prep work.

Jason King's wiki has a helpful write-up on this topic. However, the specific method of setting up the bridge is likely not going to work anymore since 2015 is a very long time ago when it comes to Linux networking...

 

[NJRoadfan]

I wasn't able to get Jason's example working on my network. Even when I got the kernel forwarding enabled, packets wouldn't correctly route to the subnet from my main network. I suspect that my home router had a faulty RIP implementation. What does work is what @mactjaap did for the macipgw VM and Ivan did for A2SERVER is to setup a NATed subnet.

 

[NJRoadfan]

In other news, all the UAMs in Netatalk now use libgcrypt, simplifying configuration (one library does it all), and satisfying the internet lawyers being 100% GPL compatible*. It only took like 15-20 years for it to happen.

*Fun fact: Netatalk bundled with Debian was always compiled with the RandNum and DHCAST128 UAMs disabled because linking OpenSSL libraries was viewed as creating a non-GPL compatible package.

 

[slipperygrey]

I wasn't able to get Jason's example working on my network. Even when I got the kernel forwarding enabled, packets wouldn't correctly route to the subnet from my main network. I suspect that my home router had a faulty RIP implementation. What does work is what @mactjaap did for the macipgw VM and Ivan did for A2SERVER is to setup a NATed subnet.

Do you have resources you could share for people to follow along and reproduce the NATed subnet for macipgw? Maybe that shell script you wrote is the best reference right now?

 

[NJRoadfan]

Instructions below are Debian specific (should work on Ubuntu) and may need tweaking for your distribution.

The janky solution used in A2SERVER involves a systemd service file.

a2server/files/macipgw.service.txt at currentdev · NJRoadfan/a2server

AppleTalk server for Apple II computers. Contribute to NJRoadfan/a2server development by creating an account on GitHub.

github.com

Copy this to: /usr/lib/systemd/system/macipgw.service

Next is the script to setup the NAT and start macipgw. This script assumes you have macipgw installed in /usr/local/sbin/.

a2server/files/macipgw-start.sh.txt at currentdev · NJRoadfan/a2server

AppleTalk server for Apple II computers. Contribute to NJRoadfan/a2server development by creating an account on GitHub.

github.com

Copy this to: /usr/local/sbin/macipgw-start.sh

Be sure to mark it executable. Something like sudo chmod ugo+x /usr/local/sbin/macipgw-start.sh should do the trick.

Finally enable the new service to start on boot with: sudo systemctl enable macipgw

 

[slipperygrey]

A quick heads-up that the macipgw PR has been merged into the main branch now. The addition of a config file is tracked in a separate ticket. If you find any bugs please let us know!

 

[slipperygrey]

4.0.0 is out now!

Release Netatalk 4.0.0 · Netatalk/netatalk

Netatalk 4.0.0 is available! The Netatalk team is proud to announce the first release in the Netatalk 4.0 release series. This release constitutes major changes over Netatalk 3.2, and is recommende...

github.com

If you are a brave early adopter, why don't you take it for a spin today?

 

[mactjaap]

I’m working on a new release of the MacIPRpi. Will test right away!

 

[slipperygrey]

@mactjaap Please let us know if you get stuck!

These are good resources for building netatalk from source:

netatalk/INSTALL.md at netatalk-4-0-0 · Netatalk/netatalk

Netatalk is a Free and Open Source AFP fileserver. A *NIX or BSD system running Netatalk is capable of serving many Macintosh clients simultaneously as an AppleShare file server. - Netatalk/netatalk

github.com

Appendix A. Compile Netatalk from Source

 

[robin-fo]

Every now and then, I play around with Netatalk's source code, especially with my attempt to integrate userspace AppleTalk communication which worked with version 2, but requiring some hacks. These hacks were all related to Netatalks frequent use of fork() to generate child processes. While working around problems related to this, I got the impression that forks are somehow regarded as obsolete on macOS. Netatalk even needs some sort of hack to properly run on the latest versions of macOS (OBJC_DISABLE_INITIALIZE_FORK_SAFETY).

Have there ever been considerations to replace forks with threads or other approaches?

There also appears to be another legacy issue with macOS: Apple dropped/is phasing out PostScript support, so printing using papd might cause issues:

Maybe we can add come ps2pdf conversion in between?

 

[slipperygrey]

@robin-fo You are talking from the hypothetical scenario where we have a userland AppleTalk stack on macOS that someone might be developing?

 

[slipperygrey]

Heads-up that netatalk 4.0.0 is available now as a Debian Sid package. If you add the unstable repo to your package manager you can install it. Although I haven’t tested if all dependencies can be resolved on stable Bookworm or not. At the very least it works great on the latest Testing snapshot (the future Debian Trixie).

Since the netatalk suite is massively sprawling, I created this atomic package structure:

- netatalk — the AFP file server
- netatalk-tools — the various sysadmin tools and scripts
- netatalk-doc — the html manual
- libatalk — the shared library (used by all binaries)
- libatalk-dev — development headers
- atalkd — the AppleTalk controller daemon + sundry AppleTalk network tools
- papd — printer daemon and LocalTalk printing tools
- timelord — time server daemon
- a2boot — Apple II netboot daemon
- macipgw —MacIP gateway daemon

The dependency and recommends should be set up properly so that apt will pull the packages that you need and want.

One neat thing about this structure is that you can install only the AFP server or only the AppleTalk networking suite if you only want, say, the papd print server and nothing else. They all depend on the libatalk shared library but that’s it.

If you run into a dependency bug please let me know!

 

[NJRoadfan]

There also appears to be another legacy issue with macOS: Apple dropped/is phasing out PostScript support, so printing using papd might cause issues:
Maybe we can add come ps2pdf conversion in between?

This is coming once CUPS 3.0 starts to materialize into a working product.

What version of macOS is this? It likely never worked anyway.........Apple long since effectively forked CUPS 2.3 into their own build for macOS while the community maintains it on other platforms still. They likely never supported sending Postscript directly to a print queue.

Starting with CUPS 3.0, we're going to have to handle Postscript to PDF conversion "in-house" by calling libcupsfilters before sending the job to CUPS. On Linux (and basically every non-macOS platform with CUPS 2.x) pre-filtering Postscript to the printer's native format via cupsfilters was how we were handling this. We just have to do it manually in papd in the future. I don't know if this approach will work on macOS though. You'll likely have to build libcupsfilters (not likely since it has a ton of dependencies), or write a custom conversion routine. If Apple has native OS libraries that can convert Postscript to PDF inside of a C program, it can likely be done with a #ifdef code block.

More info here: https://github.com/Netatalk/netatalk/issues/663

 

[robin-fo]

@robin-fo You are talking from the hypothetical scenario where we have a userland AppleTalk stack on macOS that someone might be developing?

Well, yes

What version of macOS is this?

Sonoma/Sequoia with CUPS 2.3.4

The reason I believe this is due to a recent change is that Preview no longer opens PS/EPS files (which has always been possible with earlier versions)

 

[NJRoadfan]

Apple removed the Postscript converter that was built into the OS libraries, which is really annoying. It was nice having Preview automatically convert PS files.

 

[saybur]

Wanted to give a short trip report of my experience installing Netatalk 4. TL;DR: release is working great for me, no problems so far.

I installed 4.0.1, compiled from source on Debian stable. Instructions here worked perfectly. After installation I perused the excellent 2.x->4.0 upgrade guide and associated man pages to migrate my existing config over. That part was fairly simple, the v3 style config is IMHO nicer to use and for the most part the old syntax mapped cleanly to the new one for me.

After that, a few simple systemctl commands later I was up and running. My setup has some quirks (kernel-based VM with 2x NICs, some read-only shares, no EA support on the filesystems) but even with that I didn't run into any major problems. A few possible places to clarify documentation, all assuming I'm not operating under incorrect assumptions and/or doing something wrong:

• I was getting errors when setting both "afp interfaces" and "afp listen," resolved by just using the latter option.
• I needed to 'systemctl start atalkd' to get AppleTalk working, the wording in the migration guide implies (to me anyway) that you only need the 'netatalk' service to manage all daemons.
• It isn't relevant for me (or tested) but I saw "Removed type/creator extension mapping support" in the upgrade guide and was surprised to see 'extmap.conf,' is that line referring to something else?

Netatalk is really important software for many of us, and while I don't know from personal experience what maintaining a large and legacy codebase is like, I can't imagine every minute working on it is pure coding joy. For what it's worth I (and I'm sure many others) greatly appreciate the work you folks are doing to keep Netatalk alive and well in 2024. Thank you!

 

[NJRoadfan]

It involved a lot of cursing. The previous maintainer of code from roughly 2009 to 2013 was terrible at actually leaving comments on commits. Things were changed and moved around without any explanation of "why" and most of those commits were HUGE. It also didn't help that there were obvious bugs in parts of the code that only exposed themselves when I started grafting in the AppleTalk protocol support.

At one point I found an "optimization" commit that was reverted that likely would have worked if one of the bugs I later found was actually fixed.

 

[slipperygrey]

• I was getting errors when setting both "afp interfaces" and "afp listen," resolved by just using the latter option.

Conceptually the two do appear mutually exclusive. I think ideally, they should be combined into one option, with the code detecting whether you specify an interface name or IP addresses.

But for starters, I agree the afp.conf man page should be updated.

• I needed to 'systemctl start atalkd' to get AppleTalk working, the wording in the migration guide implies (to me anyway) that you only need the 'netatalk' service to manage all daemons.

Good call-out. I think it's also implied in places that afp.conf replaces all other config files, which is not true for atalkd.conf and papd.conf.

• It isn't relevant for me (or tested) but I saw "Removed type/creator extension mapping support" in the upgrade guide and was surprised to see 'extmap.conf,' is that line referring to something else?

I will remove that bullet point. The extmap.conf feature wasn't in the initial 3.0 release but added later. They forgot to update the upgrade guide at that time.