Ah, I should have said something about that: this system is running MacTCP, mostly due to how little RAM it has (5MB). I'll find some 4MB 30 pin modules and try OT. I'd be curious if my current Daynaport issues go away with it too.
I should re-benchmark with Classic networking on my SE/30. I should have noted that I was running Classic networking on my SE when I was testing it earlier - OT requires an 030. (I was actually a bit surprised, booting the same 7.5.3 image which is normally Open Transport automatically switched to Classic networking when I booted it on my SE and magically reverted back to Open Transport when I went back to my SE/30).
That is really weird. This network is two computers: the IIsi and the Linux VM via a bridged adapter. There is basically no background traffic at all. At least on this hardware (my original V1 prototype), I was having unstable transmissions (lots of stalls / error flags set) until I redid things to be much more aggressive with re-transmitting. I have no idea why there's a difference. Maybe I have something else going on here, like sketchy hardware somewhere on the networking side.
I will say that things are vastly more stable on the transmission front after making those firmware changes, FWIW. The ENC28J60 is a buggy chip. If you want some nerdy entertainment, it's worth reading the errata sheet; a significant number of pretty important parts (full duplex, the pattern filter, etc) have severe enough flaws to make them basically unusable.
I doubt this is it, but could you take a look at the date stamp on the ENC28J60, to the right of the Microchip label? Mine starts with 1904 for reference.
I'm running on my regular network so there is definitely a lot of background activity. That being said, between the ENC settings, the pattern match filter and the filtering I do in the firmware, only wanted packets reach the Mac driver. My setup has no period appropriate hardware other than my client (ie. SE/30, SE or 145b) - The Synology (FTP Server) is a modern Linux-based NAS and Netatalk is a Linux-based AFP server. The Netatalk server is running on a VMware VM and connected to the LAN via a virtual adapter. So if there is an issue due to networking differences, moving to a vintage server solution will further increase those differences.
Interesting thought on the ENC chip. Mine is seems newer although I'm not fully sure how to interpret the date code - 20237EY. Maybe mine is a future-model - 2023 July where all the bugs were worked out??
Instead of using the master branch for a non-FATfs setup, would you mind checking the Version 2.0 hardware branch? What you're seeing on the master branch is interesting - When I put in the Mac address filtering (which is in the Version 2.0 hardware) I was starting to play around with trying to pre-cache some of the incoming packets. I'm not sure what happened exactly but at some point my system started to exhibit similar behavior where the Mac driver wouldn't see the inbound packets. The fix was to add a second delay after the MAC address gets transmitted which is what the current Multipacket branch does. I honestly don't know what exactly caused this to be required as I hadn't had any issues previously but I'm thinking that might be the issue you're seeing. The longer packets you're noting below might give the driver enough time to recovery without the extra post-MAC address delay.
I also did some research on the current 'master' branch Daynaport issues, which are... also weird. Fetch reliably crashes with a type 10 error. MacTCP ping will get one packet out right after a reboot, then will timeout on all subsequent packets. Each packet is being seen by Wireshark, and I've got the board's debugging interface reporting that replies are being sent to the host correctly - it's like the driver on the Mac is just rejecting the packet. Even more strangely, if I increase the packet payload size within MacTCP Ping it starts working fine! Packets larger than about 300-400 bytes are totally reliable. AppleTalk works fine. The Echo Test for Daynaport is completely fine too, even with small packets. Very strange.