TashRouter: An AppleTalk Router

[thecloud]

The PowerBook still isn't seeing all zones, but a complete shutdown of both TashRouter and atalkd and then restart should clear that up. Its completely normal for routers to develop stale zone tables when you are starting up and shutting down routers one at a time.

Indeed, that was the case. Now that I could see the router actually working, I ended up assigning one zone name to the LtoudpPort instance and a different single zone name to the TapPort instance, so all the Mini vMac instances appear in the "Danger Zone" and the rest of the network is "EtherTalk Network". Of course I could have used the same zone name everywhere, but it's nice to see zones appear in the Chooser!

[atalkd.conf]
vioif0 -seed -phase 2 -net 2 -addr 2.22 -zone "EtherTalk Network"
tap0 -seed -phase 2 -net 3 -addr 3.33 -zone "EtherTalk Network"
#note: net range and zone for tap0 interface must exactly match tap0 configuration in TashRouter!

[TashRouter]
router = Router('router', ports=(
  LtoudpPort(seed_network=1, seed_zone_name=b'Danger Zone'),
  TapPort(tap_name='tap0', hw_addr=b'\xDE\xAD\xBE\xEF\xCA\xFE', seed_network_min=3, seed_network_max=3, seed_zone_names=[b'EtherTalk Network']),
))
#note: hw_addr must NOT match your actual tap0 interface address; use a fake MAC address like DE:AD:BE:EF:CA:FE here

Both the real and virtual Macs see two zones in this configuration, and can mount shared volumes from servers in either zone. This will make Mini vMac a lot more usable; previously, it was difficult to move files between its filesystem and a modern system.

More pictures!

 

[thecloud]

Here is a cleaner diff for the BSD support. With this change applied, TashRouter is working fine on NetBSD, and should behave as it did before on Linux.

diff --git a/tashrouter/port/ethertalk/tap.py b/tashrouter/port/ethertalk/tap.py
index eb0fba6..86223b2 100755
--- a/tashrouter/port/ethertalk/tap.py
+++ b/tashrouter/port/ethertalk/tap.py
@@ -40,8 +40,13 @@ class TapPort(EtherTalkPort):
   __repr__ = short_str
 
   def start(self, router):
-    self._fp = os.open('/dev/net/tun', os.O_RDWR)
-    ioctl(self._fp, self.TUNSETIFF, struct.pack('16sH22x', self._tap_name.encode('ascii') or b'', self.IFF_TAP | self.IFF_NO_PI))
+    try:
+      # open and configure Linux tap device
+      self._fp = os.open('/dev/net/tun', os.O_RDWR)
+      ioctl(self._fp, self.TUNSETIFF, struct.pack('16sH22x', self._tap_name.encode('ascii') or b'', self.IFF_TAP | self.IFF_NO_PI))
+    except:
+      # open BSD tap device if no Linux tap device
+      self._fp = os.open('/dev/' + self._tap_name, os.O_RDWR)
     super().start(router)
     self._reader_thread = Thread(target=self._reader_run)
     self._reader_thread.start()

 

[superpete]

I've written a rough pcap implementation for ethertalk.

I've tested it under Debian and Windows.

Under Linux you'll need libpcap installed. Under Windows you'll need NPCAP installed. You'll also need pypcap or pcap-ct installed in python.

diff --git a/tashrouter/port/ethertalk/pcap.py b/tashrouter/port/ethertalk/pcap.py
new file mode 100644
index 0000000..d25cdd8
--- /dev/null
+++ b/tashrouter/port/ethertalk/pcap.py
@@ -0,0 +1,91 @@
+'''Port driver for EtherTalk using libpcap.'''
+
+from queue import Queue
+import logging
+import time
+from threading import Thread, Event
+
+try:
+    import pcap
+except ImportError:
+    print("[!] Missing modules pcap, try running 'pip install --upgrade pcap-ct'")
+
+from . import EtherTalkPort
+
+class PCapPort(EtherTalkPort):
+  '''Port driver for EtherTalk using libpcap.'''
+  
+
+  def __init__(self, interface_name, hw_addr, **kwargs):
+    super().__init__(hw_addr, **kwargs)
+    self._reader_thread = None
+    self._reader_started_event = Event()
+    self._reader_stop_requested = False
+    self._reader_stopped_event = Event()
+    self._interface_name = interface_name
+    self._writer_thread = None
+    self._writer_started_event = Event()
+    self._writer_stop_flag = object()
+    self._writer_stopped_event = Event()
+    self._writer_queue = Queue()
+
+  def short_str(self):
+    return self._interface_name
+
+  __str__ = short_str
+  __repr__ = short_str
+
+  def start(self, router):
+  
+    self._sniffer = pcap.pcap(name=self._interface_name, promisc=True, immediate=True, timeout_ms=250)
+  
+    # Todo, add a packet filter to only capture EtherTalk packets?
+  
+    super().start(router)
+    self._reader_thread = Thread(target=self._reader_run)
+    self._reader_thread.start()
+    self._writer_thread = Thread(target=self._writer_run)
+    self._writer_thread.start()
+    self._reader_started_event.wait()
+    self._writer_started_event.wait()
+
+  def stop(self):
+    self._reader_stop_requested = True
+    self._writer_queue.put(self._writer_stop_flag)
+    self._reader_stopped_event.wait()
+    self._writer_stopped_event.wait()
+  
+    self._sniffer.close()
+  
+    super().stop()
+
+  def send_frame(self, frame_data):
+    self._writer_queue.put(frame_data)
+
+  def _reader_run(self):
+    self._reader_started_event.set()
+
+    while not self._reader_stop_requested:
+      # Todo: make sure we're not dropping packets here
+      self._sniffer.dispatch(1, self._pcap_callback)
+    self._reader_stopped_event.set()
+  
+
+  def _writer_run(self):
+    self._writer_started_event.set()
+    while True:
+      frame_data = self._writer_queue.get()
+      if frame_data is self._writer_stop_flag: break
+      try:
+        self._sniffer.sendpacket(frame_data)
+      except OSError:
+        logging.warning("Couldn't send packet")
+        logging.debug(self._sniffer.geterr())
+        pass
+    self._writer_stopped_event.set()
+
+  def _pcap_callback(self, ts, pkt):
+    self.inbound_frame(pkt)
+
+  def get_interface_names():
+    pcap.findalldevs()
\ No newline at end of file

You'll need a line in your router to enable it, eg under Windows I have:

router = Router('router', ports=(
  LtoudpPort(seed_network=1, seed_zone_name=b'LToUDP Network'),
  PCapPort(interface_name='\\Device\\NPF_{B7D4E073-2185-4912-BBE8-3948C6636D02}', hw_addr=b'\xDE\xAD\xBE\xEF\xCA\xFE', seed_network_min=3, seed_network_max=5, seed_zone_names=[b'EtherTalk Network']),
))

Under Windows, you can get the interface name from the attached EthList.exe. Under Linux just use your interface name, eg eth0 or ens123.

If anyone tests this please let me know how you go.

 

[Tashtari]

Cool stuff! It was on my list to make a pcap driver but never got around to it.

 

[cheesestraws]

yay libpcap!

 

[fergycool]

I'm seeing fairly regular hangs with TashRouter. This is quite recent and I think this is probably happening due to the number of zones on #globaltalk getting a lot of interest over #MARCHintosh. ..or at least the stress of all those zones is causing this to hang undoubtedly due to an error in my config

Over the past week or so Tashrouter just stops routing and my Localtalk part of the network just disappears and so I need to restart it TashRouter. I'm assuming this is due to the number of zones as previously it has been running fine for six months or so. I've updated the RPi and rebooted it with no change.

Mar 21 11:51:17 wolfalice python3[14932]: Exception in thread Thread-10 (_reader_run):
Mar 21 11:51:17 wolfalice python3[14932]: Traceback (most recent call last):
Mar 21 11:51:17 wolfalice python3[14932]: File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
Mar 21 11:51:17 wolfalice python3[14932]: self.run()
Mar 21 11:51:17 wolfalice python3[14932]: File "/usr/lib/python3.11/threading.py", line 975, in run
Mar 21 11:51:17 wolfalice python3[14932]: self._target(*self._args, **self._kwargs)
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/port/ethertalk/tap.py", line 69, in _reader_run
Mar 21 11:51:17 wolfalice python3[14932]: self.inbound_frame(os.read(self._fp, 65535))
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/port/ethertalk/__init__.py", line 263, in inbound_frame
Mar 21 11:51:17 wolfalice python3[14932]: self._router.inbound(datagram, self)
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/router/router.py", line 107, in inbound
Mar 21 11:51:17 wolfalice python3[14932]: self.route(datagram, originating=False)
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/router/router.py", line 152, in route
Mar 21 11:51:17 wolfalice python3[14932]: entry.port.broadcast(datagram)
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/port/localtalk/__init__.py", line 164, in broadcast
Mar 21 11:51:17 wolfalice python3[14932]: self.send_frame(bytes((0xFF, self.node, self.LLAP_APPLETALK_SHORT_HEADER)) + datagram.as_short_header_bytes())
Mar 21 11:51:17 wolfalice python3[14932]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/datagram.py", line 118, in as_short_header_bytes
Mar 21 11:51:17 wolfalice python3[14932]: raise ValueError('invalid hop count %d, short-header datagrams may not have non-zero hop count' % self.hop_count)
Mar 21 11:51:17 wolfalice python3[14932]: ValueError: invalid hop count 3, short-header datagrams may not have non-zero hop count

I also noticed that if I started the app ChatNET (on a Quadra connected via LocalTalk) then TashRouter immediately hangs with the same error message.

The Python script that starts TashRouter has:

router = Router('router', ports=(
LtoudpPort(seed_network=17767, seed_zone_name=b'scotgate'),
TashTalkPort(serial_port='/dev/ttyAMA0', seed_network=17768, seed_zone_name=b'scotgate', verify_checksums=False),
TapPort(tap_name='tap0', hw_addr=b'\x76\xb1\xb6\x25\xce\x46', seed_network_min=17769, seed_network_max=17769, seed_zone_names=[b'scotgate']),
))

My network has Apple Internet Rrouter running via QEMU then TashRouter/TashHat to connect a few Localtalk connected Macs and one printer. The same RPi that runs TashRouter also has NEtatalk 2.4 installed and running. The RPi 3 model B 1GB is running Raspbian Bookworm.

 

[Tashtari]

invalid hop count 3, short-header datagrams may not have non-zero hop count

Interesting. This doesn't point to a load issue, I don't think. This could still be an issue with TashRouter, but it might also be an issue with someone on the network creating an illegal packet and TashRouter gagging on it. TashRouter really should just discard the packet and keep on trucking. I'd have to look into this more to say for sure, but I'm extremely busy of late... if you want to paper over the issue, I'd suggest going into port/localtalk/__init__.py and putting a try-except around the self.send_frame call at line 164 that just swallows ValueErrors, comme ça:

try:
    self.send_frame(...)
except ValueError as e:
    print('oh no: %s' % str(e))

 

[fergycool]

Interesting. This doesn't point to a load issue, I don't think. This could still be an issue with TashRouter, but it might also be an issue with someone on the network creating an illegal packet and TashRouter gagging on it. TashRouter really should just discard the packet and keep on trucking. I'd have to look into this more to say for sure, but I'm extremely busy of late... if you want to paper over the issue, I'd suggest going into port/localtalk/__init__.py and putting a try-except around the self.send_frame call at line 164 that just swallows ValueErrors, comme ça:

try:
    self.send_frame(...)
except ValueError as e:
    print('oh no: %s' % str(e))

Thanks for that extremely quick reply! I think the ink had barely dried on my post before you give me a solution I will try as you suggest now and report back! Cheers

 

[cheesestraws]

Over the past week or so Tashrouter just stops routing and my Localtalk part of the network just disappears and so I need to restart it TashRouter. I'm assuming this is due to the number of zones as previously it has been running fine for six months or so. I've updated the RPi and rebooted it with no change.

Not number of zones.

Looking at the stacktrace:

Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/port/ethertalk/tap.py", line 69, in _reader_run
Mar 21 11:51:17 wolfalice python3[14932]: self.inbound_frame(os.read(self._fp, 65535))
Mar 21 11:51:17 wolfalice python3[14932]: File "/home/chris/src/tashrouter/tashrouter/port/ethertalk/__init__.py", line 263, in inbound_frame
Mar 21 11:51:17 wolfalice python3[14932]: self._router.inbound(datagram, self)
...
Mar 21 11:51:17 wolfalice python3[14932]:   File "/home/chris/src/tashrouter/tashrouter/port/localtalk/__init__.py", line 164, in broadcast
Mar 21 11:51:17 wolfalice python3[14932]:     self.send_frame(bytes((0xFF, self.node, self.LLAP_APPLETALK_SHORT_HEADER)) + datagram.as_short_header_bytes())
Mar 21 11:51:17 wolfalice python3[14932]:                                                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 21 11:51:17 wolfalice python3[14932]:   File "/home/chris/src/tashrouter/tashrouter/datagram.py", line 118, in as_short_header_bytes
Mar 21 11:51:17 wolfalice python3[14932]:     raise ValueError('invalid hop count %d, short-header datagrams may not have non-zero hop count' % self.hop_count)
Mar 21 11:51:17 wolfalice python3[14932]: ValueError: invalid hop count 3, short-header datagrams may not have non-zero hop count

Someone's sending short-header packets over EtherTalk at all? For a packet routed from EtherTalk to LocalTalk I'd expect an LLAP layer 2 header followed by a long DDP header.

@Tashtari this feels like something that should be a 'log-shrug-and-carry-on' rather than a thrown exception

 

[fergycool]

Thanks for that extremely quick reply! I think the ink had barely dried on my post before you give me a solution I will try as you suggest now and report back! Cheers

Yup. That does the trick. Previously just launching and clicking LOGIN on ChatNet would cause the issue. Now the network remains up. I assume that the sporadic network issue will do too since it was the same error. Again, many thanks!

 

[fergycool]

Not number of zones.

Someone's sending short-header packets over EtherTalk at all? For a packet routed from EtherTalk to LocalTalk I'd expect an LLAP layer 2 header followed by a long DDP header.

@Tashtari this feels like something that should be a 'log-shrug-and-carry-on' rather than a thrown exception

Thanks. Where do you think this is coming from? I did notice that launching NetChat also immediately causes this error (although not with the workaround @Tashtari suggested. At a wild guess could somebody else be running this app and that's causing it. ..or could the cause be closer to home (i.e. my network!). Otherwise somebody else must have seen this as surely there are others running TashRouter on #GlobalTalk?

 

[cheesestraws]

Sorry if terse, brain not behaving well today.

I'm not particularly inclined to debug globaltalk issues but I think this actually might be a TashRouter bug.

What I'm wondering is if this is a directed-broadcast issue. That is to say, some node in a distant network sends a packet to yournet.255, which is the "all nodes in yournet" broadcast address. That is routed through the routers on its path to you per the network ID, with the hop count increasing. It then arrives at TashRouter through EtherTalk (l.7 in the stacktrace above) and is passed through to the LocalTalk port. The LocalTalk port then assumes that it's a short-header packet, and barfs at the fact it has a hop count > 0. But it's not a short-header packet, it's a long-header packet that is directed to a broadcast address.

So TashRouter is actually treating the packet incorrectly. I think the deeper problem here is that it seems to need to work out whether a packet is a long- or short-header DDP packet by means of its source and destination, and that won't work in all legit circumstances; the header length of the packet needs to be a property of the packet, not inferred from where it has come from and where it is going.

Edit: in OmniTalk, for example, the header length of the packet is stored here https://github.com/cheesestraws/omnitalk/blob/main/omnitalk/main/mem/buffers.h#L36 and set on incoming LLAP frames here: https://github.com/cheesestraws/omnitalk/blob/main/omnitalk/main/lap/llap/llap.c#L37

 

[NJRoadfan]

This might have been a side effect of the fixes made for RTMP or the ImageWriter LocalTalk card. Watching actual traffic from AppleTalk routers, the only packets that must always use Short DDP packets are the RTMP broadcasts from the router itself. Everything else uses Long DDP packets on a network that has a router. This is due to older devices like the IIe Workstation Card and the original LaserWriter/LaserWriter Plus only being able to "see" RTMP broadcasts in short DDP packets.

 

[thecloud]

Coming back to this thread late, but I can confirm that I also saw the same TashRouter crash that @fergycool reported. At the time, I saw the self.hop_count check was failing and figured it could safely be ignored rather than bringing down the router, so I just commented it out and everything has been running smoothly since then. The weird thing is that TashRouter was running fine for a month prior to this, without encountering the hop count issue, then the crashes started happening suddenly around the 21st. That suggests some new source of those packets was introduced which hadn't been running earlier.