Netatalk 4.0 - Future-proofing Apple File Sharing

[slipperygrey]

Over 13 hours uptime so far with the latest fix. It seems to be working OK. Yes, there are currently 64 zones visible on the network. 2 of those are my local zones; the rest are coming from Apple Internet Router.

This might be the ideal stress test scenario for these code changes!

To explain what I'm up to: I want to proactively shore up security and memory safety in 4.2, leveraging SonarQube static analysis.

So far, I was able to resolve all security bugs – the outstanding ones had to do with PAM authentication validation, and TOCTOU file access race conditions. And I've made about 50% progress on addressing blocker level reliability bugs, such as buffer overflows, null pointer dereferences, memory leaks, and so on.

There's always a risk of subtly breaking things, false positive asserts and so on.

 

[jmacz]

Sorry if this question has been answered before... long thread and simple search didn't turn up anything nor did I spot anything obvious in the official Netatalk documentation.

If I access Netatalk files externally and rename/move them, everything works as expected.

If I am in a shell in the Linux host that's running Netatalk (I'm using the basic containerized version), then renaming/moving files causes those files to lose their resource fork, and/or get the wrong resource fork (for example when one file is moved to take on the name of a file I previously renamed).

Conceptually I am assuming that Netatalk must be using some other meta data store for holding the resource fork for files, and that it needs to know when a rename/move is occurring so that it can update references/etc. This is straight forward with proper external access to files as you know the operations that are being executed. However when directly manipulating the filesystem via the shell on the host system, you'd need some type of notification of the change. Normally you'd take advantage of inotify for example but there are lots of limitations to that so not sure if that's a viable option. I saw a note about Tracker in the docs (and it mentions Tracker uses inotify) so I'm trying to understand if this is supposed to work and it's a bug, or whether this is not supported and I should not be manipulating files directly from within the host.

Note that my use case was moving around a bunch of locally saved old 68K software... I could do it by accessing via my vintage Macs and/or accessing via my modern Macs through AFP, but I was doing it from the command line because it was faster than UI/drag-n-drop.

 

[treellama]

I don't know if it still works this way in Netatalk 4, but in older versions there is the "ad" command for manipulating files.

 

[slipperygrey]

Yep, "ad" is your friend. See the man page for usage notes: https://netatalk.io/stable/htmldocs/ad.1

Note that netatalk has to be running to use ad, because it relies on access to the CNID daemon to carry out the metadata update.

 

[jmacz]

Yep, "ad" is your friend. See the man page for usage notes: https://netatalk.io/stable/htmldocs/ad.1

Note that netatalk has to be running to use ad, because it relies on access to the CNID daemon to carry out the metadata update.

Ok great, looks like 'ad' is available inside the container. That'll work. Thanks!

 

[slipperygrey]

Quick poll: of you netatalk users here, is anyone relying on the "include = /path/to/extra_afp.conf" directive in afp.conf?

For context, this is a directive that allows you to dynamically load and nest a secondary afp.conf file. However this relied on local hacks to the iniparser library. I am on the fence whether to discard this feature, or reimplement it.

 

[mactjaap]

Quick poll: of you netatalk users here, is anyone relying on the "include = /path/to/extra_afp.conf" directive in afp.conf?

For context, this is a directive that allows you to dynamically load and nest a secondary afp.conf file. However this relied on local hacks to the iniparser library. I am on the fence whether to discard this feature, or reimplement it.

I do not use it.

 

[pl212]

Here's an oddball question: is there a way to run multiple DDP-based AppleShare servers on the physical interface? I think I understand how to do this on IP networks (what we used to call multihoming, nowadays done via this or simiar), but I'm not quite sure how to do so with AppleTalk.

(This is all a vanity project to get multiple AppleShare servers to show up in the Chooser, nothing more! )

 

[NJRoadfan]

Netatalk 2.x supports this. Support was removed from Netatalk 3.0 to simplify the codebase.

 

[slipperygrey]

• The documentation isn't bad, but having to bounce between the html documentation, the INSTALL.md readme and the compile instructions is a little busy and makes things more confusing.

I've been tinkering with the organization of the documentation lately, since I was able to lift it all into Markdown in the last release cycle which makes it much easier to work with.

Right now my thinking is that the entry point is the documentation page on the website:

Netatalk - documentation

Netatalk Wiki

netatalk.io

If you look at the Installation section, you have the Quick Start guide first (same as INSTALL.md in the code repo), then the Compile from Source guide (previously the Compile appendix in the html manual.)

These two together should be enough to get you sorted with building from source.

There's still the Installation chapter in the manual, but this is more of a high level overview, plus in-depth explanations of the purpose of each dependency. If you already know what you want (i.e. build from source with all features enabled) you don't have to read it.

@Mk.558 What do you think about this structure? Please let me know if you have any improvement suggestions.

 

[Mk.558]

I think it looks really good. Having one stop-shop for getting it going is probably the way to go for sure.

The layout is nice too. The only thing I'd suggest is increasing the font side of the code text, so it's easier to read. People with high PPI displays on small laptops will probably not be able to read any of that.



The red box is probably missing something. Perhaps start with a line that installs the minimum required dependencies as an example, with another example that shows a couple extra things? Should be enough to allow someone to understand how to do it. Granted, if they're using Linux, they should already know, but we have to remember that most of the young buckaroos these days aren't like them old diehards in the 1990s who only built their own kernels from source and maintained their system from scratch.

The green line I'm thinking should be changed to something like "Further down this page is a list of optional packages for additional features of Netatalk."

Think that line that says ...

sudo ninja -C build uninstall                           # install Netatalk

... might have an error. I do like the extra # tags that describe what the command does. Overall though, I think it's looking good.

 

[slipperygrey]

@Mk.558 Thanks for the solid feedback, as always!

I restructured the Quick Start page and put the list of requirements in the middle, as part of a more natural flow of operations. To your point, it might be worth experimenting with splitting the list so that only mandatory requirements come in the middle, with the optional at the end. Let me experiment with this another day.

The code text now has 120% font size. It does look more readable this way. Hopefully no corner case layout will break.

And I fixed the "uninstall" typo!

 

[Mk.558]

Neat. I'd love to see the updated version.

FWIW, I installed Netatalk 4.0.4 (can't remember if I did 4.0.5 or not...it's been a few months) without ninja. It worked fine. -shrugs-

 

[thecloud]

"Install required software dependencies" is going to be different for each platform, based on its package manager.

I updated to Netatalk 4.2.1dev today and it went smoothly. All seems stable with 48 zones currently on the network.

 

[slipperygrey]

"Install required software dependencies" is going to be different for each platform, based on its package manager.

In my experience it's surprisingly consistent. I test nonstop on 12 different platforms.

 

[slipperygrey]

In addition to the containerized webmin discussed in the netatalk 2.4 thread, I also got the mysql CNID backend up and running in containers a few days ago. The original intention was for testing purposes: to test the mysql backend in continuous integration and stop me from breaking it (again.)

But it could potentially be used for production deployments as well. As a recap, netatalk uses a berkeleydb CNID backend by default presently. However the mysql backend allows for better control and scalability, while also being a living technology unlike bdb which has been abandoned for many years at this point.

However, mysql/mariadb is a bit more cumbersome since you have to install and configure database server software. It isn't particularly hard, but not plug'n'play either.

Anyways, by combining the docker compose yaml examples that I have compiled, you can now create a container constellation like below: netatalk running with a mariadb backend, webmin as the web UI for netatalk and adminer as the web UI for mariadb.

[+] Running 7/7

 ✔ Network netatalk_default                                            Created                                      0.1s
 ✔ Volume "netatalk_cnid_db_data"                                      Created                                      0.0s
 ✔ Container netatalk-adminer-1                                        Created                                      0.1s
 ✔ Container netatalk-mariadb-1                                        Created                                      0.1s
 ✔ Container netatalk-webmin-1                                         Created                                      0.0s
 ✔ Container netatalk-netatalk-1                                       Recreated                                   10.2s