ModTashtalk: lt0 driver for Linux

[tashtari]

Offtopic, but say, how'd you become the PIC Whisperer Tash?

@cheesestraws christened me as such during an IRC conversation about ADB in September 2021 (after I had made TashTalk but before any of my other PIC projects). =)

 

[twelvetone12]

I have a fair bit of experience with submitting changes to mainline recently due to another project I've been blogging about, and would be happy to help you out with that process if you get to the point that you would like to give it a shot. Just let me know if/when you're interested!

I would love to try to get thin into mainline, but I will surely need some help
For serdev, I think the complex part is really what happens in the upper layers, so when time will come I think it should not be too complex to port it.
In any case I would like to get this properly working, I still need a couple hardware pieces and I will be travelling so I can come back to this in a couple weeks

 

[3lectr1cPPC]

@cheesestraws christened me as such during an IRC conversation about ADB in September 2021 (after I had made TashTalk but before any of my other PIC projects). =)

Definitely deserved

 

[robin-fo]

Silly question... ..but would this also work in Darwin/Mac OS X?

 

[NJRoadfan]

Not likely. OS X uses a different driver architecture.

 

[twelvetone12]

I worked more on this, and success! (almost). It is starting to work. The journey to get there was... tough to say the least. And it does not help that tools are old or non existent.

I managed to communicate with my mac - turns out I was doing the CRC wrong (d'oh). Once the CRC was ok my mac started responding back to pings. The difficult part here was that aecho from netatalk depends, well, on netatalk so I had to write a stripped down version by myself (in the file nbp.c, I know, originally it was meant to do nbp, I will fix it next time . I could see the pings in tcpdump but they would never show in recvfrom(). I could never get any data actually - I saw it (properly decoded) with tcpdump but it never made it to userspace.

After lots of debugging and sweat turns out there is some magic happening with the DDP header which I don't completely understand. ddp.c in the Kernel says:

/*
* Receive a LocalTalk frame. We make some demands on the caller here.
* Caller must provide enough headroom on the packet to pull the short
* header and append a long one.
*/

What does that mean? Well the clue comes later:

/*
* The push leaves us with a ddephdr not an shdr, and
* handily the port bytes in the right place preset.
*/
ddp = skb_push(skb, sizeof(*ddp) - 4);

It replaces the shotr DDP header from localtalk with the extended one. Fair enough. But it does not work and printing bytes shows that the data offsets are wrong. Well cops.c comes to the rescue when pushing data:

skb_reset_mac_header(skb); /* Point to entire packet. */
skb_pull(skb, 3);
skb_reset_transport_header(skb); /* Point to data (Skip header). */

The magic like is the skb_pull, which makes it possible to neatly overlay the header with the incoming data. Sincerely now I'm super tired and I can't figure out why it works, but it works. Adding that like makes the pings come back!

24 bytes from 0.56: aep_seq=17. time=16. ms

Why didn't I copy that like of code originally? well, it breaks tcpdump! I always (incorrectly) assumed the data I saw with tcpdump was exactly the data sent out by the ddp layer, but it is not. So now tcpdump is broken but DDP actually works. This means netatalk is the next step. Ah yes and also addressing, now tashtalk is hardcoded as device 2.

 

[tashtari]

turns out I was doing the CRC wrong

Oh, the LocalTalk/SDLC CRC ought to be illegal under the Geneva convention. I spent more time than I care to admit getting it right both on the PIC and Python sides...

Glad to hear you're succeeding! I'm really eager to see this realized.

 

[cheesestraws]

turns out I was doing the CRC wrong (d'oh).

Oh, the LocalTalk/SDLC CRC ought to be illegal under the Geneva convention. I spent more time than I care to admit getting it right both on the PIC and Python sides...

Yup! And I ended up borrowing Tashtari's implementation...

 

[NJRoadfan]

Things might be a little weird once the packet leaves the driver and goes to the Linux AppleTalk kernel module. Mostly because the module only supports EtherTalk Phase 2, which has a different header. If I recall, the COPS drivers made LocalTalk PC cards look Ethernet-like to the kernel resulting in these workarounds.

What isn't working with netatalk now? The COPS driver has some notes for configuring atalkd to work with it.

The COPS LocalTalk Linux driver (cops.c) — The Linux Kernel documentation

 

[twelvetone12]

And if you look at the git diff you will see what I very shamefully did wrong
In any case communication works, which is good. I managed to get netatalk to write stuff to lt0 but I could not see anything on the mac (it was an afp share). I will write a small NBP responder just for testing (I did a prototype in python), I'm starting to think it is probably easier to write an AFP server from scratch than to figure out how it works with netatalk
Another step will be getting MacIP to work, it is still there in the kernel (!) but the tools to use it date from 1999 and nothing works anymore. A USB to localtalk adapter would be cool

 

[twelvetone12]

Things might be a little weird once the packet leaves the driver and goes to the Linux AppleTalk kernel module. Mostly because the module only supports EtherTalk Phase 2, which has a different header. If I recall, the COPS drivers made LocalTalk PC cards look Ethernet-like to the kernel resulting in these workarounds.

What isn't working with netatalk now? The COPS driver has some notes for configuring atalkd to work with it.

The COPS LocalTalk Linux driver (cops.c) — The Linux Kernel documentation

yup that is exactly what it does. I'm just not really sure how it does it, because it shrinks the incoming packet, than it extends it to overlay the last 4 bytes of the extended header to the last 4 of the short header, I'm just not really sure I understand the mechanics of what is happening.

Those configurations don't seem to work anymore with a modern netatalk, unfortunately. I can see lt0 traffic if I set it to phase 2, but if I set it to phase 1 afpd refuses to start and I see nothing going out of lt0. But this is a debugging project for another day

 

[NJRoadfan]

For MacTCP, the macipgw project is fully functional and works well. Chris Kobayashi even integrated it into the netatalk-classic project..... which he seems to have deleted the repo to. The big catch is that you need to compile the AppleTalk kernel module WITHOUT the IPDDP support enabled, otherwise the packets get filtered out before macipgw can process them.

One of these days I have to get around and play with this. Going into atalkd's code is certainly "there be dragons" territory! At least most of the commit history is there so it might be straight forward to find the breaking change. It can't be any worse than papd's text processing code right?

 

[twelvetone12]

papd's text processing code right?

I have yet to venture there but I'm already scared

 

[robin-fo]

I'm starting to think it is probably easier to write an AFP server from scratch than to figure out how it works with netatalk

This is somehow what I constantly think when I read through the Netatalk code. CAP is even worse, but mostly due to its ancient K&R C code style.

 

[robin-fo]

Btw in the recent days I‘m contemplating about how a userspace AppleTalk stack should be designed. Especially about how the individual user applications communicate with the core parts of the stack (running in a separate process with root permissions). My current approach is to use UNIX domain sockets and wrap data and commands in a bespoke protocol.

The interesting thing now would be to create a common, system independent AppleTalk socket API and modify Netatalk accordingly. Hopefully, this would allow us to achieve a much more clean code and better portability in Netatalk. As soon as this is done, any kind of LocalTalk integration would be doable using very limited effort.

 

[twelvetone12]

That could be interesting. It took me around 20 minutes to implement a basic NBP responder in python and LToUDP, and around 6 hours to figure out why my DDP packets were silently dropped in the kernel in lt0
But also the kernel code contains lots of interesting stuff, like all the routing stuff. I think we could gather all the tools that we have and then see what is the optimal solution for the future.

 

[robin-fo]

But also the kernel code contains lots of interesting stuff, like all the routing stuff

PM me if you want to have a look inside my AppleTalk stack. All the basic routing stuff should already be implemented there, although sometimes not thoroughly tested. Routing between LocalTalk (LToUDP) and Ethernet works nice though. I use this almost daily

 

[slipperygrey]

Those configurations don't seem to work anymore with a modern netatalk, unfortunately. I can see lt0 traffic if I set it to phase 2, but if I set it to phase 1 afpd refuses to start and I see nothing going out of lt0. But this is a debugging project for another day

Modern netatalk, as in both contemporary netatalk3 and netatalk2?

 

[slipperygrey]

For MacTCP, the macipgw project is fully functional and works well. Chris Kobayashi even integrated it into the netatalk-classic project..... which he seems to have deleted the repo to. The big catch is that you need to compile the AppleTalk kernel module WITHOUT the IPDDP support enabled, otherwise the packets get filtered out before macipgw can process them.

FWIW my fork of netatalk-classic is still alive... https://github.com/rdmark/netatalk-classic

I didn't sync it with the final year of his progress though. He had a half-working sqlite cnid backend by mid last year IIRC. I wonder why he decided to erase his work.

 

[twelvetone12]

I only tested it with netatalk 2, really in passing through, so I will need to put more energy into that. I did use your fork netatalk-2.x which compiles very nicely
How does macipgw work with localtalk networks? It interfaces with TashTalk in some way? I did not have a chance yet to try it.