Netatalk 2.4.0 available
- Thread starter slipperygrey
- Start date
slipperygrey
Well-known member
+1 for afpfs-ng. Earlier this year I made a fork to modernize the code base, normalize the various abandoned forks and community patches, and get it compiling and running again.
Alas, I was only able to get the interactive AFP client working reliably. The fuse file system layer isn’t working well at all with fuse v3 and gets very unstable shortly after mounting on Linux… and errors out immediately on FreeBSD.
At that point I had to put the project on ice and pivot back to netatalk.
Here’s the code if you want to give it a whirl. At the very least the “afpcmd” interactive client is handy for emergencies and testing!
github.com
Alas, I was only able to get the interactive AFP client working reliably. The fuse file system layer isn’t working well at all with fuse v3 and gets very unstable shortly after mounting on Linux… and errors out immediately on FreeBSD.
At that point I had to put the project on ice and pivot back to netatalk.
Here’s the code if you want to give it a whirl. At the very least the “afpcmd” interactive client is handy for emergencies and testing!
GitHub - rdmark/afpfs-ng: afpfs-ng/libafpclient is an open source client for the Apple Filing Protocol
afpfs-ng/libafpclient is an open source client for the Apple Filing Protocol - rdmark/afpfs-ng
github.com
slipperygrey
Well-known member
Correction: I couldn't get it to work properly with fuse 2.9.9 either.
ironborn65
Well-known member
Hi @slipperygrey, thanks for maintaining this software.
I've successfully installed Netatalk2 using docker in my Synology.
I found it more convenient to use only the YAML composer without the Substitute parameter to the docker run.
So I added the volume mapping as
volumes:
- /mylocalpath/:/mnt/afpshare
Is there a reason for your choice? Maybe you have in mind to containerize the web admin as well, which I'd support
, in any case, a named volume and mapping are supported in the YAML. The web admin would fit nicely in the YAML.
I've successfully installed Netatalk2 using docker in my Synology.
I found it more convenient to use only the YAML composer without the Substitute parameter to the docker run.
So I added the volume mapping as
volumes:
- /mylocalpath/:/mnt/afpshare
Is there a reason for your choice? Maybe you have in mind to containerize the web admin as well, which I'd support
, in any case, a named volume and mapping are supported in the YAML. The web admin would fit nicely in the YAML.
slipperygrey
Well-known member
@ironborn65 You're welcome! Thank you for using netatalk.
If I understand the question correctly, you're asking why I chose Docker bind mounts over Docker volumes in the docker-compose.yml example? If this is the question, the answer is: because bind mounts are self-managed and independent of a particular user's file system. So as a plug'n'play example, it's the ideal choice.
I have contemplated putting Webmin inside the container in the past. However, the main reason I decided against it, is because it makes the whole setup inherently insecure. It exposes another port and gives you remote root access to the entire netatalk container after authentication. If you have an idea on how to secure it, I'd be all ears though.
If I understand the question correctly, you're asking why I chose Docker bind mounts over Docker volumes in the docker-compose.yml example? If this is the question, the answer is: because bind mounts are self-managed and independent of a particular user's file system. So as a plug'n'play example, it's the ideal choice.
I have contemplated putting Webmin inside the container in the past. However, the main reason I decided against it, is because it makes the whole setup inherently insecure. It exposes another port and gives you remote root access to the entire netatalk container after authentication. If you have an idea on how to secure it, I'd be all ears though.
ironborn65
Well-known member
Hi,
I understand it's a matter of principles, so this is only my two pence. I prefer the docker-compose file because I have a single configuration file to maintain and backup. Since I have to modify the docker-compose YAML file for username, password and volume name, it's more convenient to also add the local path of my shared folder, instead of having two places where to set configurations
About the webmin, since, as I understand, it "just" parses and updates the apf.conf file, it could stay in a container by itself, as an additional service in docker-compose accessing the netatalk apf.conf via a shared folder, this is what docker-compose is for. No need to put it in the same Netatalk container.
I hope this helps
I understand it's a matter of principles, so this is only my two pence. I prefer the docker-compose file because I have a single configuration file to maintain and backup. Since I have to modify the docker-compose YAML file for username, password and volume name, it's more convenient to also add the local path of my shared folder, instead of having two places where to set configurations
About the webmin, since, as I understand, it "just" parses and updates the apf.conf file, it could stay in a container by itself, as an additional service in docker-compose accessing the netatalk apf.conf via a shared folder, this is what docker-compose is for. No need to put it in the same Netatalk container.
I hope this helps
slipperygrey
Well-known member
@ironborn65 So if I understand your suggestion correctly, you would set up a separate Docker container that runs only webmin, which then mounts the file system of the netatalk container to access the netatalk config files?
Two challenges to overcome for this to work, off the top of my head:
Assuming that you would define two services for docker compose (netatalk and webmin), figure out the cleanest (most docker-like?) way to make the live netatalk config files available to the webmin container. By defining a shared volume just for that, which both services mount? This is uncharted territory for me.
How to enable the other webmin functionality, namely the ability to operate the netatalk services (start/stop/restart, i.e. via the init system inside the netatalk container) and report on the status of said services (whether they're running or not, i.e. through the pid).
Two challenges to overcome for this to work, off the top of my head:
Assuming that you would define two services for docker compose (netatalk and webmin), figure out the cleanest (most docker-like?) way to make the live netatalk config files available to the webmin container. By defining a shared volume just for that, which both services mount? This is uncharted territory for me.
How to enable the other webmin functionality, namely the ability to operate the netatalk services (start/stop/restart, i.e. via the init system inside the netatalk container) and report on the status of said services (whether they're running or not, i.e. through the pid).
ironborn65
Well-known member
@slipperygrey , for the first point, sharing a file via a volume, it can be something like this (the YAML example is not computable or complete):
services:
netatalk:
container_name: netatalk2
image: netatalk/netatalk2:latest
network_mode: "host"
cap_add:
- NET_ADMIN
volumes:
- shared-apfconf:/etc
- /var/services/homes/pierino23/Drive/personal/retro:/mnt/afpshare
- /var/run/dbus:/var/run/dbus
environment:
- "SERVER_NAME=Netatalk Server"
- "SHARE_NAME=Shared-Retro"
- "AFP_USER=xxxx"
- "AFP_PASS=xxxxxx"
- "AFPD_OPTIONS=-icon -mimicmodel RackMac"
- "ATALKD_INTERFACE=eth0"
webmin:
container_name: webmin
image: webmin/webmin:latest
network_mode: "host"
volumes:
- shared-apfconf:/something
volumes:
shared-apfconf:
Both services mount the same folder.
The /something path must be set properly in the Webmin container, I don't know if it can be configured.
So, Webmin reads from the file /something/apf.conf that is stored in /ect of the netatalk container.
For the ability to operate the netatalk services via the init system, I need to understand which RPC technology is used to start/stop and retrieve the status: inter-process based, IP/network bases protocol, another file, or else?
services:
netatalk:
container_name: netatalk2
image: netatalk/netatalk2:latest
network_mode: "host"
cap_add:
- NET_ADMIN
volumes:
- shared-apfconf:/etc
- /var/services/homes/pierino23/Drive/personal/retro:/mnt/afpshare
- /var/run/dbus:/var/run/dbus
environment:
- "SERVER_NAME=Netatalk Server"
- "SHARE_NAME=Shared-Retro"
- "AFP_USER=xxxx"
- "AFP_PASS=xxxxxx"
- "AFPD_OPTIONS=-icon -mimicmodel RackMac"
- "ATALKD_INTERFACE=eth0"
webmin:
container_name: webmin
image: webmin/webmin:latest
network_mode: "host"
volumes:
- shared-apfconf:/something
volumes:
shared-apfconf:
Both services mount the same folder.
The /something path must be set properly in the Webmin container, I don't know if it can be configured.
So, Webmin reads from the file /something/apf.conf that is stored in /ect of the netatalk container.
For the ability to operate the netatalk services via the init system, I need to understand which RPC technology is used to start/stop and retrieve the status: inter-process based, IP/network bases protocol, another file, or else?
slipperygrey
Well-known member
The webmin module uses the standard webmin config scheme, so it can be set to look for netatalk binaries and config files at arbitrary locations. This should not be a problem.
One correction to my previous post: we don’t actually use the init system but rather start afpd/netatalk (in non-forking mode) as the final step in the entry point script. So if we shut down netatalk, the container will terminate.
With this revelation, we have another challenge at hand: how to get netatalk to gracefully re-read its config files after editing them with webmin. I think the proper (only?) way is to send a SIGHUP signal to afpd. Is it possible to pass signals between docker containers?
One correction to my previous post: we don’t actually use the init system but rather start afpd/netatalk (in non-forking mode) as the final step in the entry point script. So if we shut down netatalk, the container will terminate.
With this revelation, we have another challenge at hand: how to get netatalk to gracefully re-read its config files after editing them with webmin. I think the proper (only?) way is to send a SIGHUP signal to afpd. Is it possible to pass signals between docker containers?
ironborn65
Well-known member
it's possible. You must install Docker daemon in the webmin container by mounting the Docker socket
docker run -v /var/run/docker.sock:/var/run/docker.sock -it webmin-container-name
then you can run docker commands from webmin like:
docker kill --signal=SIGHUP netatalk-container-name (or id)
Be aware that exposing docker sockets is considered insecure in a production environment or in an uncontrolled network.
As an alternative, you can add a task to the netatalk container that monitor the apf.conf: when there is a change you send the SIGHUP signal to afpd. No cross-container communications required. This assumes that the apf-conf file is not continuously modified by webmin, but replaced with the new version only when committing the changes.
BTW: you can keep a container running if you launch another task, for example using the sleep infinity command or tail. Any dump task works.
docker run -v /var/run/docker.sock:/var/run/docker.sock -it webmin-container-name
then you can run docker commands from webmin like:
docker kill --signal=SIGHUP netatalk-container-name (or id)
Be aware that exposing docker sockets is considered insecure in a production environment or in an uncontrolled network.
As an alternative, you can add a task to the netatalk container that monitor the apf.conf: when there is a change you send the SIGHUP signal to afpd. No cross-container communications required. This assumes that the apf-conf file is not continuously modified by webmin, but replaced with the new version only when committing the changes.
BTW: you can keep a container running if you launch another task, for example using the sleep infinity command or tail. Any dump task works.
slipperygrey
Well-known member
These are good insights. I can see how this would be theoretically feasible. Do you have a particular mechanism in mind for monitoring changes to the config files (for option two)?
The webmin module only writes to the config files when you hit the save button, so this won't be a problem.
The webmin module only writes to the config files when you hit the save button, so this won't be a problem.
ironborn65
Well-known member
foreword: I'm not a linux/unix expert.
en.wikipedia.org
Differently from Docker, I never used Inotify in my job, this is just a Google thing
I hope this helps, others here might have different ideas
inotify - Wikipedia
Differently from Docker, I never used Inotify in my job, this is just a Google thing
I hope this helps, others here might have different ideas
slipperygrey
Well-known member
@ironborn65 Late response. Apologies.
So this is all good stuff. Will definitely take some development work but I can see the potential benefit for Docker users. May I ask you to create an issue ticket over at the GitHub project so that it can be tracked?
So this is all good stuff. Will definitely take some development work but I can see the potential benefit for Docker users. May I ask you to create an issue ticket over at the GitHub project so that it can be tracked?
ironborn65
Well-known member
will do
slipperygrey
Well-known member
Thank you!
At a related note: Netatalk 2.4.8 is available. I have not been announcing the last few releases here on the forums, because they were strictly addressing issues in the build system and packaging.
However, this last one is a security release, specifically for the bundled WolfSSL library. If you're building with the embedded SSL option, this is a mandatory upgrade for you.
At a related note: Netatalk 2.4.8 is available. I have not been announcing the last few releases here on the forums, because they were strictly addressing issues in the build system and packaging.
However, this last one is a security release, specifically for the bundled WolfSSL library. If you're building with the embedded SSL option, this is a mandatory upgrade for you.
@twelvetone12 There IS a command line AFP client that supports AppleTalk: https://github.com/demonfoo/afp-perl
Written in Perl of all things.
Written in Perl of all things.
twelvetone12
Well-known member
Ahh Perl, so many memories, I will give this a try, thanks!
mactjaap
Well-known member
Thanks for this tip! I managed to get it going and it works nice and indeed .... over AppleTalk!
Code:
$/usr/local/bin/afpclient.pl --atalk-first "afp://MacIPRpi/MacIPRPi DISK"
afpclient MacIPRpi:MacIPRPi DISK/> ls
-rw-r--r-- 1 0 0 6148 Jan 9 01:24 .DS_Store
drwxr-xr-x 2 0 nogroup 0 Jan 8 23:08 Network Trash Folder
drwxr-xr-x 5 nobody nogroup 0 Jan 8 23:08 SHARED_DISK
drwxr-xr-x 2 0 nogroup 0 Jan 8 23:08 Temporary Items
afpclient MacIPRpi:MacIPRPi DISK/> cd SHARED_DISK/
afpclient MacIPRpi:MacIPRPi DISK/SHARED_DISK> ls
drwxr-xr-x 3 ftp nogroup 0 Jan 5 12:45 FTP
-rw-r--r-- 1 0 0 2031592 Jan 8 23:09 IMG_4761.PNG
afpclient MacIPRpi:MacIPRPi DISK/SHARED_DISK> get IMG_4761.PNG
100% |*************************| IMG_4761.PNG 191.12 MB/sec
Transferred 2031592 bytes in 0h0m0.01sslipperygrey
Well-known member
Being stuck at home over the Easter weekend fighting off a cold, I had the inspiration/time to work on this.
Over at the feature request ticket that you raised, I have explained the solution at a high level. In the end, I settled on the config file polling option. Hopefully it won't have too much of a negative effect on performance. You can customize the polling frequency with the "AFP_CONFIG_POLLING" env variable. So if you set it to '1', the script will poll for changes to afp.conf every second, and send a SIGHUP to netatalk if a change is detected so that it re-reads its config file. Sleek!
The one major UX flaw right now, obviously, is that the start/stop/restart service buttons won't have any effect, since the daemons run in a container separate from webmin. I'm toying with the idea of containerizing a slightly modified version of the webmin module that simply hides those buttons. Ideally without creating a copy of index.cgi that has to be maintained.
Bleeding edge container images are published on the github registry: https://github.com/Netatalk/netatalk/pkgs/container/netatalk-webmin
Below is the POC docker compose yaml using the bridge network driver. For appletalk support you need to use the host network driver as usual, but for a TCP only setup this works nicely. I'm quite happy with how this turned out. Webmin has an infamously lax security posture, so confining it in a container makes a lot of sense.
Please note that this is for bleeding edge netatalk 4.x. I don't plan to backport it to 2.x. Thanks for your understanding!
YAML:
services:
netatalk:
image: netatalk:latest
networks:
- afp_network
ports:
- "548:548"
volumes:
- afpshare:/mnt/afpshare
- afpbackup:/mnt/afpbackup
- afpconf:/etc/netatalk
- /var/run/dbus:/var/run/dbus
environment:
- AFP_USER=atalk1
- AFP_USER2=atalk2
- AFP_PASS=
- AFP_PASS2=
- AFP_GROUP=afpusers
- AFP_CONFIG_POLLING=5
- MANUAL_CONFIG=1
webmin:
image: netatalk_webmin_module:latest
networks:
- afp_network
ports:
- "10000:10000"
volumes:
- afpconf:/etc/netatalk
environment:
- WEBMIN_USER=admin
- WEBMIN_PASS=
depends_on:
- netatalk
volumes:
afpshare:
afpbackup:
afpconf:
networks:
afp_network:
driver: bridgeslipperygrey
Well-known member
A solution to this issue has been merged into the main branch now. I introduced a dynamic module option that hides away the "service actions" that expect the netatalk daemons to be locally available. This option gets enabled by default when building the webmin module via the Dockerfile.
So now you get a streamlined GUI in the containerized webmin that should make sense in that context! Feedback much appreciated.
