Netatalk 4.0 - Future-proofing Apple File Sharing

[slipperygrey]

For all you Webmin fans, I took a few hours today to update the Webmin module to support AppleTalk and the added options (so far) in 4.0.0dev. It turned out really nicely.

 

[NJRoadfan]

It is wild that such a bug was not discovered for 13 years. By orphaned, you're saying that Netatalk won't read it again, or is it still read and processed on every access?

Orphaned as in, the data is still in the .AppleDouble folder, but won't be accessed anymore. Note, the extended attributes I am referring to are the ones created by macOS, such as the infamous "com.apple.quarantine" and other metadata. Off hand, I don't know of any applications that store important data in EAs, but this is still considered data loss. The AFP commands to manipulate them didn't show up until OS X 10.4 with AFP 3.2 (FPSetExtAttr, etc.).

For folks only using netatalk for sharing with classic Macs, its a non-issue as OS9 and older don't use these types of EAs. All the classic file attributes (resource fork, FinderInfo, comments, color, etc.) are transferred properly. Its odd this was missed since netatalk 2.x supported OS X EAs already. I'm guessing they were pretty rare out in the wild back in 2009-10.

 

[slipperygrey]

@NJRoadfan I filed https://github.com/Netatalk/netatalk/issues/1448

Do you have a lead on how to fix it?

 

[shirsch]

...

Most of this was via emulated machines, so real hardware testing is always appreciated. Get out your rare clients like an Apple IIe Workstation card (@shirsch), or AppleShare for MS-DOS and break this thing! Modern TCP/IP client testing is appreciated as well as I'm concerned about any possible regressions.

Unfortunately many of the required packages for Ubuntu build are too new for my environment. Due to persistent breakage I do not upgrade unless forced to by cutoff of security patches. Still on Ubuntu 18.04 here. If/when I upgrade my system I'll be glad to to do some testing.

 

[slipperygrey]

@shirsch Do you have Docker Engine on your Ubuntu 18.04 system by any chance? Or would be open to installing it?

By running netatalk in a Docker container you would bypass any troublesome software dependency caused by your aging OS.

 

[shirsch]

I know nothing about Docker, but can certainly setup a VMware guest OS for the purpose. I really should start thinking about a new distribution, but I cannot and will not use anything that relies on SNAPs. Philosophy aside, SNAP installs don't work from an NFS mounted home directory.

 

[slipperygrey]

Aren't snaps used only for Ubuntu's customized GUI applications, with the the core system packages remaining Debian debs? I guess I don't know your precise usecase. Anyhow, the "correct" upgrade path for you is Debian Bookworm. Guaranteed no shenanigans.

A virtual machine would definitely do the trick as far as running netatalk goes. This is what I do all the time for cross-platform development.

 

[shirsch]

I looked at Bookworm, but am seeing that KDE desktop support is "iffy". I'll take it for a test drive in a VM. Anyway, enough off-topic chatter

 

[slipperygrey]

Netatalk 4.0.0 beta is available as a Docker image now. It comes with all the best parts of the 3.x release series, and the 2.x release series that you all know and love!

https://hub.docker.com/r/netatalk/netatalk

If you have a host computer with an AppleTalk network stack (Linux or NetBSD) you only need to follow the instructions to get started. All options are named in the same way as the previous Docker releases.

 

[shirsch]

Ok, I spun up a Bookworm VM and was able to build Netatalk 4. Tomorrow I'll fire it up and take it for a spin. However, it's NAT'ed out to my network from the VMware 10. so may take some work to get clients talking to it.

 

[NJRoadfan]

Going to have to figure out a bridged networking solution for AppleTalk packets. In other news, Netatalk does not work with the Apple (and later Farallon) MS-DOS clients. A minor bug in the client causes it to not detect RandNum UAM support on the server, but even if you get logged in, you aren't getting much further. The MS-DOS client requires that the AFP server provide "short names" for file names (basically the server converts Macintosh file names into the DOS 8.3 format), something Netatalk does not support.

On the bright side, printing from the DOS PAP client works with Netatalk's papd, including the Epson LQ Postscript emulator that is included. You'll need Farallon PhoneNET PC 3.0 for this though as earlier versions expect a real Postscript printer and not a spooler on the other end.

 

[shirsch]

That's very strange. I added "short name" support to netatalk in the late 90s in order to get it cooperating with the //e Workstation card. Has that been removed?

 

[NJRoadfan]

The "short name" is the one explained starting on page 13-13 of Inside AppleTalk. It is requested using bit 7 in the file or directory bitmap with a GetFileDirParams call. Apple only specified its usage for MS-DOS clients. Pretty sure ProDOS clients do not use it as the Workstation Card and GS/OS work fine... and support longer filenames anyway.

 

[shirsch]

Ok, so something different from the capability I added. I can assure you that ProDOS would not interoperate with netatalk before I coded in what I thought was called "short name" support.

 

[NJRoadfan]

Only thing I see in the source that comes to mind is the ProDOS Info Block that contains the File Type/Aux Type information.

 

[shirsch]

I'll have to dig through my archives to refresh Ye Olde brain cells.

From an FAQ in the late 90s (predating my work):

        * I can't get PhoneNet PC, MacLan Connect, or Apple ][ AppleTalk to
          connect to the netatalk fileserver.

          Netatalk doesn't implement the entire AFP protocol. One thing it
          lacks is short-filename support, which these systems require. One
          would think that with Windows 95 having support for longer
          filenames, it would be OK, but Mirimar still uses short file-name
          calls. Use Samba for your PC's.

And here is a diff from my sandbox directory that I believe shows all the mods for ProDOS 8 operation. You may be correct that the info block was key. Been over 25 years.

 

[slipperygrey]

@Mk.558 Would you be able to test a macipgw integrated with netatalk?

I have started a PR here and got as far as building it and starting the binary to confirm that it doesn't segfault. But this is as far as I could get before bed this evening.

If you could build this and make sure nothing is broken, it would be very helpful. Please let me know if you need pointers on how to use Meson to build the code. There are instructions in INSTALL.md. You just need to enable the whole AppleTalk stack with the `-Dwith-appletalk=true` flag.

 

[robin-fo]

Every now and then, I play around with Netatalk's source code, especially with my attempt to integrate userspace AppleTalk communication which worked with version 2, but requiring some hacks. These hacks were all related to Netatalks frequent use of fork() to generate child processes. While working around problems related to this, I got the impression that forks are somehow regarded as obsolete on macOS. Netatalk even needs some sort of hack to properly run on the latest versions of macOS (OBJC_DISABLE_INITIALIZE_FORK_SAFETY).

Have there ever been considerations to replace forks with threads or other approaches?

 

[Mk.558]

ya i can usually find time to test things on the weekends. Usually the weekends are packed with things to test and do: last week I wrapped up Vicom Internet Gateway.

edit: Any changes to the codebase? I would expect it to be somewhat dated, but still usable. And simplifyable -- heard something somewhere about it having many ways to activate it.

 

[slipperygrey]

Every now and then, I play around with Netatalk's source code, especially with my attempt to integrate userspace AppleTalk communication which worked with version 2, but requiring some hacks. These hacks were all related to Netatalks frequent use of fork() to generate child processes. While working around problems related to this, I got the impression that forks are somehow regarded as obsolete on macOS. Netatalk even needs some sort of hack to properly run on the latest versions of macOS (OBJC_DISABLE_INITIALIZE_FORK_SAFETY).

Have there ever been considerations to replace forks with threads or other approaches?

It is also my understanding that forking as a design decision is outdated, and that threads are superior in this era of multi-core CPUs. This is something that has indeed been on my mind ever since Apple started enforcing their anti-fork policy in Sonoma. (Bypassing the policy is not a hack per se. But Apple may revoke the policy override in a future macOS release.)

In the name of future-proofing Netatalk, I agree that moving away from old school forking is a good idea. Since you have seen the code, you are probably aware that forking is deeply embedded into the afpd code, so it is bound to be a major coding project. We are accepting volunteer contributions.