cheesestraws
Well-known member
NetBSD AppleTalk certainly works fine on amd64, but I've never tried it on ARM - if you do get around to trying it out let me/us know!
Netatalk 4.0 - Future-proofing Apple File Sharing
- Thread starter slipperygrey
- Start date
slipperygrey
Well-known member
Great sleuthing work! Hopefully the commented out code was because no one had bothered to test it on aarch64…Just to confirm, I pulled down cdn.netbsd.org/pub/NetBSD/NetBSD-10.0/source/sets/syssrc.tgz and had a look at /usr/src/sys/arch/evbarm/conf/GENERIC.common. Sure enough, the option for NETATALK is commented out. If I compare with /usr/src/sys/arch/amd64/conf/GENERIC, the NETATALK option is *not* commented out there. I suppose I can try building my own kernel to see if it works.
Edit: built the new GENERIC64 kernel with NETATALK support. It was simple and took about 5 minutes on Apple Silicon.atalkd seems to start up and work now. Not sure why it was commented out from that port.
May I ask you to add a section to the NetBSD wiki page with a summary of your findings?
Installing Netatalk on NetBSD
Netatalk is a Free and Open Source AFP fileserver. A *NIX or BSD system running Netatalk is capable of serving many Macintosh clients simultaneously as an AppleShare file server. - Netatalk/netatalk
github.com
It would be a handy resource for the next person who runs into the same problem!
Good idea. I updated the AppleTalk section of the NetBSD wiki page to cover this.
Today I built and installed Netatalk 4 from the main branch. There's no going back to earlier versions now; full steam ahead!
This is working on the arm64 build of NetBSD (aarch64), after rebuilding the kernel to include netatalk support. Once atalkd runs, ifconfig -a shows that the virtual network interface includes the atalk address family:
I set up a minimal test server configuration in afp.conf:
I can then see the server registered on both TCP (port 548) and on AppleTalk (NBP):
TCP port 548 is forwarded from the macOS host to the NetBSD VM, and other Macs with AFP-over-TCP clients are able to connect and use the share. That is very cool. However, I'm ultimately hoping to figure out a way to get AppleTalk and LToUDP clients connecting to netatalk running on the VM.
1. For AppleTalk-over-Ethernet, I would normally have QEMU set up a tap interface on the host and bridge it to the physical Ethernet interface so the VM dumps data onto the wire at the data link layer rather than at the IP layer. But one small problem: macOS hosts don't have tap devices! There is a macOS kernel extension that provides tun/tap devices which comes with Tunnelblick. I used something similar for Intel machines but don't have experience with this on Apple Silicon yet.
2. For LToUDP, I think this can probably be done with QEMU just forwarding UDP port 1954 to the VM. But I imagine this also requires software on the VM which can convert AppleTalk to UDP and back. Is that a job for TashRouter? Will see if I can set that up on NetBSD.
Today I built and installed Netatalk 4 from the main branch. There's no going back to earlier versions now; full steam ahead!
Code:
arm64# /usr/local/sbin/atalkd -V
atalkd 4.0.9dev - AppleTalk Network Manager DaemonThis is working on the arm64 build of NetBSD (aarch64), after rebuilding the kernel to include netatalk support. Once atalkd runs, ifconfig -a shows that the virtual network interface includes the atalk address family:
Code:
arm64# ifconfig -a
vioif0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ec_capabilities=0x1<VLAN_MTU>
ec_enabled=0
address: 52:54:00:12:34:56
status: active
atalk 65280.234 phase 2 range 1 - 65534 broadcast 65280.255
inet6 fe80::5054:ff:fe12:3456%vioif0/64 flags 0 scopeid 0x1
inet6 fec0::ddce:8bb2:7350:5d16/64 flags 0x40<AUTOCONF>
inet 10.0.2.15/24 broadcast 10.0.2.255 flags 0
lo0: flags=0x8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33624
status: active
atalk 0.0 phase 2
inet6 ::1/128 flags 0x20<NODAD>
inet6 fe80::1%lo0/64 flags 0 scopeid 0x2
inet 127.0.0.1/8 flags 0I set up a minimal test server configuration in afp.conf:
Code:
;
; Netatalk 4.x configuration file
;
[Global]
; Global server settings
appletalk = yes
appledouble = v2
ea = ad
afp listen = 10.0.2.15
hostname = Puppet Head
uam list = uams_guest.so
[Homes]
basedir regex = /home
[Shared Data]
path = /var/netatalk/AFPShareI can then see the server registered on both TCP (port 548) and on AppleTalk (NBP):
Code:
arm64# netstat -an | grep LIS
tcp 0 0 10.0.2.15.548 *.* LISTEN
tcp 0 0 127.0.0.1.4700 *.* LISTEN
tcp 0 0 10.0.2.15.22 10.0.2.2.49306 ESTABLISHED
tcp 0 0 *.22 *.* LISTEN
tcp6 0 0 *.22 *.* LISTEN
arm64# nbplkup
Puppet Head:AFPServer 65280.234:128
arm64:netatalk 65280.234:4
arm64:Workstation 65280.234:4TCP port 548 is forwarded from the macOS host to the NetBSD VM, and other Macs with AFP-over-TCP clients are able to connect and use the share. That is very cool. However, I'm ultimately hoping to figure out a way to get AppleTalk and LToUDP clients connecting to netatalk running on the VM.
1. For AppleTalk-over-Ethernet, I would normally have QEMU set up a tap interface on the host and bridge it to the physical Ethernet interface so the VM dumps data onto the wire at the data link layer rather than at the IP layer. But one small problem: macOS hosts don't have tap devices! There is a macOS kernel extension that provides tun/tap devices which comes with Tunnelblick. I used something similar for Intel machines but don't have experience with this on Apple Silicon yet.
2. For LToUDP, I think this can probably be done with QEMU just forwarding UDP port 1954 to the VM. But I imagine this also requires software on the VM which can convert AppleTalk to UDP and back. Is that a job for TashRouter? Will see if I can set that up on NetBSD.
robin-fo
Well-known member
It has! Qemu supports vmnet on macOS.
Excellent, thank you for the pointer! I did not know that had been added to QEMU. This invocation now automagically sets up both the virtual interface and the bridge interface on the host, where en4 is the physical Ethernet interface on the Mac that's hosting the VM:
Code:
qemu-system-aarch64 \
-accel hvf \
-m 2048 \
-cpu cortex-a57 -M virt,highmem=off \
-drive file=edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on \
-drive file=netbsd.qcow2,format=qcow2,media=disk \
-nographic \
-netdev vmnet-bridged,id=vmnet0,ifname=en4 \
-device virtio-net,netdev=vmnet0,mac=52:54:00:f0:0d:01 \
-serial tcp::4444,server,telnet,wait \
#-drive file=NetBSD-10.0-evbarm-aarch64.iso,format=raw,media=cdrom \Using tcpdump, I see NBP lookup packets being received by other machines on my network when I invoke 'nbplkup' on the VM:
Code:
13:31:50.732261 AT 65280.187.nis > 0.nis: nbp-lkup 1: "=:=@*" [addr=65280.187.129]Next steps: test this with an actual classic Mac on the network, and get TashRouter running on the VM so my Mini vMac 37 instances (with LToUDP) can see the server.
The good news is that Netatalk 4.0.9dev is working fine on arm64, running in a NetBSD 10.0 VM (with rebuilt kernel) on a macOS Apple Silicon host. From a PowerBook G3 on the same physical network, connected via AppleTalk-over-Ethernet, I can see the server and copy files to the shared volume, including the screen capture below.
The not-so-good news is that I haven't been able to get TashRouter to route packets from LToUDP to another port yet. The internals of TapPort assume Linux (where a tun port can be opened at '/dev/net/tun'), but this is BSD (where '/dev/tun0' can be opened, but the subsequent ioctl call fails.) Will follow up in the TashRouter thread for this issue.
The not-so-good news is that I haven't been able to get TashRouter to route packets from LToUDP to another port yet. The internals of TapPort assume Linux (where a tun port can be opened at '/dev/net/tun'), but this is BSD (where '/dev/tun0' can be opened, but the subsequent ioctl call fails.) Will follow up in the TashRouter thread for this issue.
Attachments
If you happen to run Netatalk on a macOS host, some significant enhancements are on the way. This should nearly replicate the functionality of Apple's now deprecated AFP server in macOS.
The big one is that resource forks are now natively stored in the file system as opposed to using "._" AppleDouble files. This works regardless of the filesystem your drive is formatted in, so you can share from APFS, HFS+, or even exFAT drives (the OS handles the AppleDouble translation here).
Next up is FinderInfo is synced with the native filesystem metadata, which appears as
The big one is that resource forks are now natively stored in the file system as opposed to using "._" AppleDouble files. This works regardless of the filesystem your drive is formatted in, so you can share from APFS, HFS+, or even exFAT drives (the OS handles the AppleDouble translation here).
Next up is FinderInfo is synced with the native filesystem metadata, which appears as
com.apple.FinderInfo. Things like the filetype/creator information will now be synced with the host. In addition, storage of extended attributes for macOS (AFP3.2 and newer) clients should now be working properly.
pax
Well-known member
Wow, that’s definitely a game changer. I assume that opens up the possibility to have a single file server for both classic and modern Macs, including using native tools like Time Machine and SuperDuper to back up the file server?
Right now I'm openly sharing files off my APFS formatted Catalina machine with MacOS 8 and 9 clients, which is the primary use case for this integration. Unlike Apple's server, Netatalk is fully AFP2.x compliant and supports legacy UAMs, not just DHX2. No more hacking in some older version of the AFP server from old versions of OS X to get file sharing working.
There shouldn't be anything stopping you from backing up that data to a Time Machine USB backup drive or Time Machine server.... like say a FreeBSD box running Netatalk. Both FinderInfo and resource forks are stored where Apple expects them to be and running
There shouldn't be anything stopping you from backing up that data to a Time Machine USB backup drive or Time Machine server.... like say a FreeBSD box running Netatalk.
Both FinderInfo and resource forks are stored where Apple expects them to be and running xattr will show them as extended attributes. In addition, Netatalk still stores its private data in its own extended attribute.absurd_engineering
Well-known member
First off, I wanted to thank everybody involved in reintegrating AppleTalk into 4.0. What a relief!
The decision to remove it from 3.x ranks as one of the sillier missteps I've run across in the free software arena.
I'm curious if there's any motion towards an AppleTalk implementation for modern macOS. It's great to be able to access my modern machines via AppleShare over IP, but the fact that we have to use a non-Mac to get AppleTalk support working again is vexing.
The decision to remove it from 3.x ranks as one of the sillier missteps I've run across in the free software arena.
I'm curious if there's any motion towards an AppleTalk implementation for modern macOS. It's great to be able to access my modern machines via AppleShare over IP, but the fact that we have to use a non-Mac to get AppleTalk support working again is vexing.
slipperygrey
Well-known member
There is no AppleTalk userland stack available today, to my best knowledge. If you search these forums you will find that it has been attempted, however. I think it's absolutely feasible.
BTW, I agree that getting rid of the AppleTalk transport layer amounted to throwing out the baby with the bathwater. The previous maintainer team did a good job reducing tech debt (albeit while introducing new bugs along the way) and streamlining the core of the AFP daemon. For instance, pre-3.0 netatalk had the ability to define an arbitrary number of "virtual servers" with unique configurations including the transport layers enabled (TCP, AppleTalk, or both... or none). The code would then fork the AFP daemon for each virtual server. Massive complexity underneath the hood for managing those forks. I'm glad they did away with that structure.
But as our own NJRoadfan has since demonstrated, there was a middle of the road solution for keeping AppleTalk alive while still removing obsoleted cruft.
Tashtari
PIC Whisperer
I've got one on one of my many back burners, working title TashNode. The architecture is sort of like inetd, except that instead of plain text, a chain of processes communicates up and down using a simple TLV (type, length, value) protocol. I was going to explain this further but I realized I can't remember anything about AppleTalk since it's been ten months since I worked on it, heh. The bird's eye view is that the main 'node' registers NVEs (network-visible entities) and assigns sockets for them, spawning processes for each protocol in turn - an ImageWriter might spawn ATP processes that spawn PAP processes that spawn some process that renders the output to a modern printer, for example. I got as far as that, but ran into some behavior I couldn't explain trying to get it to play nice with the LaserWriter driver and stalled there. Maybe one day.
slipperygrey
Well-known member
slipperygrey
Well-known member
Good news: netatalk 4 is now available as a Homebrew formula for macOS (and Linux, for you weirdos who use brew on Linux)
If you have brew installed already, just do:
If you're using an Intel Mac, this should work out of the box.
If you're on Apple Silicon, one additional manual step is required:
If you try it out, please let me know how it went! This my first attempt at creating a brew formula, so there may be other improvement areas.
If you have brew installed already, just do:
Code:
brew update
brew install netatalk
sudo brew services start netatalkIf you're using an Intel Mac, this should work out of the box.
If you're on Apple Silicon, one additional manual step is required:
Code:
sudo cp /opt/homebrew/etc/pam.d/netatalk /usr/local/etc/pam.dIf you try it out, please let me know how it went! This my first attempt at creating a brew formula, so there may be other improvement areas.
mactjaap
Well-known member
No go. Tested on a MacBook Air
(Model-ID: MacBookAir7,2)
Dual-Core Intel Core i5 1,8 GHz
It stops here:
==> Pouring berkeley-db@5--5.3.28_1.monterey.bottle.tar.gz
/usr/local/Cellar/berkeley-db@5/5.3.28_1: 5,272 files, 86.4MB
==> Installing netatalk dependency: libunistring
Error: An exception occurred within a child process:
Errno::EACCES: Permission denied @ dir_s_mkdir - /usr/local/Cellar/libunistring/1.3
(Model-ID: MacBookAir7,2)
Dual-Core Intel Core i5 1,8 GHz
It stops here:
==> Pouring berkeley-db@5--5.3.28_1.monterey.bottle.tar.gz
/usr/local/Cellar/berkeley-db@5/5.3.28_1: 5,272 files, 86.4MB==> Installing netatalk dependency: libunistring
Error: An exception occurred within a child process:
Errno::EACCES: Permission denied @ dir_s_mkdir - /usr/local/Cellar/libunistring/1.3
slipperygrey
Well-known member
mactjaap
Well-known member
Thanks.... I made some progress ...
Code:
==> Downloading https://github.com/Netatalk/netatalk/releases/download/netatalk-4-0-8/netatalk-4.0.8.tar.xz
Already downloaded: /Users/tjabring/Library/Caches/Homebrew/downloads/521ecb3601bdf04fbe97e79c91318f9f6addf7217af57ce00be4ec24e9a8f0c9--netatalk-4.0.8.tar.xz
==> Installing dependencies for netatalk: mysql and openldap
==> Installing netatalk dependency: mysql
==> Patching
==> cmake -S . -B build -DCOMPILATION_COMMENT=Homebrew -DINSTALL_DOCDIR=share/doc/mysql -DINSTALL_INCLUDEDIR=include/mysql -DINSTALL_INFODIR=share/info -DINSTALL_MANDIR=share/man -DINSTALL_MYSQLSHAREDIR
==> cmake --build build
==> cmake --install build
==> ./mysql-test-run.pl status --vardir=/private/tmp/mysql-20250110-45578-o8gzfq/mysql-9.1.0/mysql-test-vardir
==> Downloading https://formulae.brew.sh/api/formula.jws.json
################################################################################################################################################################################################### 100.0%
==> /usr/local/Cellar/mysql/9.1.0_1/bin/mysqld --initialize-insecure --user=tjabring --basedir=/usr/local/Cellar/mysql/9.1.0_1 --datadir=/usr/local/var/mysql --tmpdir=/tmp
🍺 /usr/local/Cellar/mysql/9.1.0_1: 326 files, 292.3MB, built in 127 minutes 54 seconds
==> Installing netatalk dependency: openldap
....
==> Installing netatalk
==> Patching
==> Applying 206fb7771862b9b98452c934dac884aaa397c8ca.patch
==> meson setup build -Dwith-afpstats=false -Dwith-appletalk=false -Dwith-bdb-path=/usr/local/opt/berkeley-db@5 -Dwith-docbook-path=/usr/local/opt/docbook-xsl/docbook-xsl -Dwith-init-dir=/usr/local/Cell
==> meson compile -C build --verboseBut I think an old MacBook Air with Monterey 12.7.6 is not a good idea to test this on.... brew is warning me too.
Code:
Warning: You are using macOS 12.
We (and Apple) do not provide support for this old version.
It is expected behaviour that some formulae will fail to build in this old version.
It is expected behaviour that Homebrew will be buggy and slow.
Do not create any issues about this on Homebrew's GitHub repositories.
Do not create any issues even if you think this message is unrelated.
Any opened issues will be immediately closed without response.
Do not ask for help from Homebrew or its maintainers on social media.
You may ask for help in Homebrew's discussions but are unlikely to receive a response.
Try to figure out the problem yourself and submit a fix as a pull request.
We will review it but may or may not accept it.
Last edited:
mactjaap
Well-known member
Maybe I can borough my daughter her MacBook. This one has an M3.
Just checking. Do I need to edit the settings?
This one? Should it now be true?
Just checking. Do I need to edit the settings?
Code:
def install
inreplace "distrib/initscripts/macos.netatalk.in", "@sbindir@", opt_sbin
inreplace "distrib/initscripts/macos.netatalk.plist.in", "@bindir@", opt_bin
inreplace "distrib/initscripts/macos.netatalk.plist.in", "@sbindir@", opt_sbin
inreplace "distrib/initscripts/systemd.netatalk.service.in", "@sbindir@", opt_sbin
bdb5_rpath = rpath(target: Formula["berkeley-db@5"].opt_lib)
ENV.append "LDFLAGS", "-Wl,-rpath,#{bdb5_rpath}" if OS.linux?
args = [
"-Dwith-afpstats=false",
"-Dwith-appletalk=#{OS.linux?}", # macOS doesn't have an AppleTalk stack
"-Dwith-bdb-path=#{Formula["berkeley-db@5"].opt_prefix}",
"-Dwith-docbook-path=#{Formula["docbook-xsl"].opt_prefix}/docbook-xsl",
"-Dwith-init-dir=#{prefix}",
"-Dwith-init-hooks=false",
"-Dwith-install-hooks=false",
"-Dwith-statedir-path=#{var}",
"-Dwith-pam-config-path=#{etc}/pam.d",
"-Dwith-rpath=false",
"-Dwith-spotlight=false",
]This one? Should it now be true?
Code:
"-Dwith-appletalk=#{OS.linux?}", # macOS doesn't have an AppleTalk stackslipperygrey
Well-known member
If the compilation hasn't completed after 24h on your Monterey Mac, I would suspect that the system is running out of memory and clang gets starved and stalls. I've seen that happen on f.e. Raspberry Pis when you try to compile software with too many concurrent threads. You could try shutting down unnecessary processes and try again. If you can get it to build, netatalk should work very well on an older machine.
I believe Homebrew distributes pre-built binaries for the three latest macOS versions. On your daughter's Mac, brew won't even attempt to compile the software (unless you force it to.) Rather, it will install the binary package instantaneously.
No need to modify the formula. This appletalk option is correct.
I believe Homebrew distributes pre-built binaries for the three latest macOS versions. On your daughter's Mac, brew won't even attempt to compile the software (unless you force it to.) Rather, it will install the binary package instantaneously.
No need to modify the formula. This appletalk option is correct.
