• Hello, Guest! Welcome back, and be sure to check out this post for more info about the recent service interruption and migration.

SCSI to Ethernet Adapter on New Hardware

demik

Well-known member
Yup, that's the most current hardware revision. Be aware that at least the ENC28J60 appears to be widely out of stock currently, so you may have to be creative sourcing the proper chips.

Thanks !
Alright, better to get the NC28J60 first I guess.... Maybe the 2.1 board will be out while waiting for it :)
 

superjer2000

Well-known member
Thanks ! Alright, better to get the NC28J60 first I guess.... Maybe the 2.1 board will be out while waiting for it :)

Before the recent posts on the prior thread were lost, Saybur promised the version 2.1 board would incorporate an 68040 class accelerator and, if the ATXmega you select has enough RAM, some limited ability to travel through time. So not a bad idea to wait!
 

saybur

Well-known member
Before the recent posts on the prior thread were lost, Saybur promised the version 2.1 board would incorporate an 68040 class accelerator and, if the ATXmega you select has enough RAM, some limited ability to travel through time. So not a bad idea to wait!

Don't forget a solution for world peace, plus bonus RGB lighting. A man's gotta have some goals in mind! :)

More seriously, the only significant change I'd make for v2.1 would probably just be a better USB plug, that little micro USB one is a real trick to hand solder. That's not a big deal at the moment given the lack of any firmware support for the USB (beyond the DFU firmware Microchip ships on blank chips). I also wouldn't want to release a new board without some personal testing, and I have plenty of version 1 and 2 boards on hand right now as well to disincentive me from ordering anything new.
 

superjer2000

Well-known member
Don't forget a solution for world peace, plus bonus RGB lighting. A man's gotta have some goals in mind! :) More seriously, the only significant change I'd make for v2.1 would probably just be a better USB plug, that little micro USB one is a real trick to hand solder. That's not a big deal at the moment given the lack of any firmware support for the USB (beyond the DFU firmware Microchip ships on blank chips). I also wouldn't want to release a new board without some personal testing, and I have plenty of version 1 and 2 boards on hand right now as well to disincentive me from ordering anything new.

I think you had mentioned before that you weren't too keen on it but the killer v 2.1 feature for me, assuming somebody can come up with a 2.5" powerbook size scuznet, is a Scuznet that includes the IDC50 and DB25 options, and a 2 prong header for a secondary power input. I wouldn't be concerned in the slightest if the board had to be enlarged but it would be great to have a single purpose solution that could be mounted internally. As noted previously, I have a number of short RJ45 panel mount adapters so putting a ScuzNet into an SE or SE/30 would be super simple.

Maybe if I repeat the above paragraphs enough times it will subliminally convince you...
 

saybur

Well-known member
Definitely can't fault you for giving it a go :D At least the last time I tried, I really didn't have luck doing the rough-layout for a board that incorporated those features. That's not saying it's impossible or anything: I wouldn't be shocked for a second if someone could make it work. It's just tricky, particularly if (like me) you have some features that are must-haves. Plus side here is that the open-source design and swappable hardware definitions should at least make it an option to spin a new board and not have to worry about making your own firmware.

On the software side, I think the hard drive related freezes I've been battling here are related to the Xmega DMA subsystem usage during disk writes. There is a busy-wait loop that checks for the DMA channel to report done after it writes a block to the memory card: for some reason, and infrequently, it will just... never complete, and the busy-wait loop will go infinite. It sometimes takes an hour for this crash to happen, which has slowed down finding a fix.
 

Chopsticks

Well-known member
hi guys, I haven't had much free time the last week to get on the forums here and check out the current status of the scuznet, I should have some free time this weekend though to do some testing though.
as for the IDC50 pin version of the hardware, I did make some a little while ago with the intention of making an internal 3.5inch HDD mountable version. I ended up making a mistake in kicad so the idc connector was mounted on the opposite side of the board. it also had a pin header for a IDC cable to mount a ethernet jack external from the main board, say on the back of the se/30 pds opening.
I do plan to make the fixes and get some fresh boards made but time has unfortunately been crazy limited recently. once ive done that though and made sure it works I'll post the Gerber files ect
 

aladds

Well-known member
powerbook size scuznet
Working on it! An initial version is currently in the post from JLCPCB - I know this version will have issues, but the way I've done it is to have a 2.5" board as one piece, and a second board with the ENC28J60 on - in my case I've made it the same size as a PowerBook 100 modem card. It'll connect with an IDC cable. The eventual plan is to be able to swap this board with e.g. a WiFi module

Issues I am seeing that might arise from this:
  • My use of the autorouter might mean that some board traces haven't been routed entirely sensibly, so we will see how well it works
  • The IDC cable is carrying the SPI comms - this can be up to 10Mhz, so I might find that I can't use it "as fast as possible" as indicated in the code. Or I need to redesign that bit and use a more appropriate cable/connector
  • Whilst I have been able to order some ENC28J60 chips (and one order is actually on its way) I don't know how many I'll get. I did deliberately use the "standard" ENC28J60 pinout as lots of cheap boards from AliExpress use, so I could always get one of these for testing.

Unrelated - I have begun construction of the bits I already have from the first set of (normal) boards and the soldering isn't as bad as I thought (full disclosure: I am an EE, so my experience/opinion may not be representative; I've done a lot of soldering!). As I'm not going to need all of these boards, and I do have enough parts to make at least 4 of them on the way, I might list them on FS/FT at cost as long as @saybur is ok with that! (Note that I'm in the UK so consider what that means regarding shipping.)
 
Last edited:

saybur

Well-known member
We've been stickied, thanks mods!

Those are both exciting pieces of news about hardware, I'm looking forward to seeing what y'all have come up with.

I might list them on FS/FT at cost as long as @saybur is ok with that!

No issues here: they're yours to do with as you please. I'm a spirited amateur with PCB design, so if you see areas where you'd suggest board changes, definitely feel free to post. I'm always interested in learning more with this stuff.

I've identified the pathology behind the bug I've been working on: the DMA channel receiving data off the memory card SPI will periodically miss a byte. Reads are 1:1 with writes, so once the write channel stops the read channel will just spin forever, waiting for that last byte that never arrives. A fix has been created and seems to be working well. I'll push the update when I can verify it captures all the error cases.
 

Chopsticks

Well-known member
On the software side, I think the hard drive related freezes I've been battling here are related to the Xmega DMA subsystem usage during disk writes. There is a busy-wait loop that checks for the DMA channel to report done after it writes a block to the memory card: for some reason, and infrequently, it will just... never complete, and the busy-wait loop will go infinite. It sometimes takes an hour for this crash to happen, which has slowed down finding a fix.
I can confirm this change has fixed my ftp freezing the system, in fact with since my last round of testing I have seen a slight increase in ftp transfer speeds, I now get around 91KB/s, and it also ramps up much faster too.

as for appleshare, transferring my OS X server 1.4.11 I still get very poor transfer speeds, around 8KB/s, though it does eventually max out at 16KB/s
 

Chopsticks

Well-known member
after running the appleshare transfer I left the room and came back a bit later and it seems to have frozen my se/30 are 5mb of transfer

interestingly though if I connect to an appleshare on my Main SE/30 then my testing se/30 (hopefully that makes sense) with the daynaport it starts off pretty close to 16KB/s and is able to get about 36kKB/s max transfer but again the freezing/scuznet lockup happens (mouse still moves).
Im quite surprised about the big change in transfer going from OS X server to using a se/30 with sys7.5.5 on it tbh.

as you mentioned above about the bug and memory card SPI missing a byte hopefully when you get the time to push the update to GitHub it might fix the lockups?
 

saybur

Well-known member
I think these weird speed differences are being caused by buffer spaces on the scuznet receiving side being shorter than what software is expecting. During FTP experiments, I keep seeing the same pattern of TCP retransmits for the last packet or two, and the sizes correlate to the buffer space in the ENC28J60 being exhausted. It wouldn't shock me for a moment if OS X sent more data per AFP transaction: that would normally improve performance, but with shorter buffers it causes a retransmit to occur, slowing things down. I'm exploring options for trying to squeeze out some more buffer space (to get 8K while reading) which might help this problem out.

For the freezes, I've got a test running right now; if I can verify it works I'll be pushing the update shortly. So far, I've been able to swap a 300MB file back and forth over FTP without problems, which is a positive sign.
 

superjer2000

Well-known member
I think these weird speed differences are being caused by buffer spaces on the scuznet receiving side being shorter than what software is expecting. During FTP experiments, I keep seeing the same pattern of TCP retransmits for the last packet or two, and the sizes correlate to the buffer space in the ENC28J60 being exhausted. It wouldn't shock me for a moment if OS X sent more data per AFP transaction: that would normally improve performance, but with shorter buffers it causes a retransmit to occur, slowing things down. I'm exploring options for trying to squeeze out some more buffer space (to get 8K while reading) which might help this problem out.

For the freezes, I've got a test running right now; if I can verify it works I'll be pushing the update shortly. So far, I've been able to swap a 300MB file back and forth over FTP without problems, which is a positive sign.
Something seems weird there. I am not seeing any real speed differences on the forked Scuznet Daynaport firmware I posted earlier between AppleTalk and FTP and I haven’t messed with the buffer space at all. I am also not seeing any retransmits. I have tested on an SE30 and pb145b. It might be an issue with the Nuvolink emulation but there shouldn’t be any differences on the Daynaport firmware.

it would be good to have the updated Daynaport firmware tested by at least one other person as right now it’s a pretty rough in that it only supports Daynaport and is hard coded to be on, but if others have success it can be ported to the main Scuznet code.

 

saybur

Well-known member
The Daynaport has some pretty aggressive polling, so it may be able to empty the buffers faster than the Nuvo reconnect-then-send approach. I'll watch for that when I test your firmware here, which I'm planning on doing after this bug gets fixed.

This issue I'm working on is on the hard drive side and will affect any firmware using the DMA-driven FatFs implementation. I'm not terribly surprised I didn't find it earlier, it's a pretty rare beastie, but since it causes a hard-lock it is also annoyingly severe and I'd like to get rid of it quickly.

Something else that's kind of interesting. I was getting ~200KB/s writes to the vsftpd server from my IIsi using Fetch with the Nuvolink firmware. That was in 640x480, 256 colors. When I switched the IIsi to black and white, write performance jumped to ~270KB/s. I know the IIsi had a cut-rate video subsystem, but seriously, that's really something. I think I'll benchmark in black and white mode from now on.
 

saybur

Well-known member
OK, finally got the fix verified and it's posted on Github. This incorporates some other changes, including expanding the ENC28J60 receive buffer by 1536 bytes at the expense of the second transmit buffer: the TX impact appears to be quite minor, and RX performance improves significantly.

I'm now confident this whole issue was the result of a race condition with the DMA units. In some cases an interrupt could fire after the write unit started, but before the read unit started, which was causing the first few USART bytes to be missed. The protection code I added will still be useful but should now hopefully never be needed, since I've added the atomic protection that should have been there in the first place. D'oh.

Also, the speeds in my previous post were from Fetch's bytes/s estimate, which is not lining up with the actual transfer rate I'm seeing. Sigh. There's always something. Figure the real speed is about half or worse of what I said, which is much more in line with previous figures. I'll use a stopwatch next time.
 

superjer2000

Well-known member
The Daynaport has some pretty aggressive polling, so it may be able to empty the buffers faster than the Nuvo reconnect-then-send approach. I'll watch for that when I test your firmware here, which I'm planning on doing after this bug gets fixed.
That is still a bit odd though. Once the Nuvolink connects successfully reconnects to the Mac, I think it keeps sending packets until it gets asked to disconnect so you’d think it would be able to keep up. I am also pretty sure I was getting around 50kbs on Appleshare with Nuvolink before on my SE30 when I was playing around with it (when I had added a few delays to make it stable on my SE30 other than the bus errors at shutdown).
 

superjer2000

Well-known member
OK, finally got the fix verified and it's posted on Github. This incorporates some other changes, including expanding the ENC28J60 receive buffer by 1536 bytes at the expense of the second transmit buffer: the TX impact appears to be quite minor, and RX performance improves significantly.

I'm now confident this whole issue was the result of a race condition with the DMA units. In some cases an interrupt could fire after the write unit started, but before the read unit started, which was causing the first few USART bytes to be missed. The protection code I added will still be useful but should now hopefully never be needed, since I've added the atomic protection that should have been there in the first place. D'oh.

Also, the speeds in my previous post were from Fetch's bytes/s estimate, which is not lining up with the actual transfer rate I'm seeing. Sigh. There's always something. Figure the real speed is about half or worse of what I said, which is much more in line with previous figures. I'll use a stopwatch next time.
Interesting...
I assumed Fetch estimates of kB per second would be accurate but a point in time estimate. i.e. it shows the current rate of transfer but not the average speed to the current point in time. I will try the stop watch test on the 6mb test file I use for AppleShare.

For the dma issue, it sounds like this would impact all versions of Scuznet.ini firmware? I haven’t had any lockups at all and I’ve spent quite a bit of time (hours?) transferring files, bbsing, installing software etc. Any thoughts as to whether there is a way to force the issue to occur? I would think I would have been affected by it at least once - unless this is just an exfat issue?
 

saybur

Well-known member
That is still a bit odd though. Once the Nuvolink connects successfully reconnects to the Mac, I think it keeps sending packets until it gets asked to disconnect so you’d think it would be able to keep up.

The TCP bursts are big and short, so maybe throughput is less important than latency? Not sure. Hopefully after some benchmarks and watching how the traffic behaves on the different firmwares I can upgrade from wild guesswork to merely poorly-informed guesswork! :D

For the dma issue, it sounds like this would impact all versions of Scuznet.ini firmware? I haven’t had any lockups at all and I’ve spent quite a bit of time (hours?) transferring files, bbsing, installing software etc. Any thoughts as to whether there is a way to force the issue to occur? I would think I would have been affected by it at least once - unless this is just an exfat issue?

The lack of atomic guards would have been affecting everyone in theory, but there were things that would make the issue more common:
  • There are more interrupts firing on the Nuvolink firmware (reconnection is implemented through those).
  • More interrupts were added around July 3rd/4th with the change to net.c reading packet headers in the interrupt context.
That's likely why you weren't bitten by this bug.
 

saybur

Well-known member
I've flashed your firmware to my V1 board and ran some tests. For context, I'm running a fresh "this Macintosh" version of 7.5.3 on a IIsi, Daynaport 7.7.2, writing to the scuznet flash card with a volume image in 'forcefast' mode. The "server" is a Debian VM running vsftpd on my desktop. It also has 7.5.3 running inside of Basilisk II networked via sheep_net as the AppleTalk host system. The test file is a Mac Binary copy of Escape Velocity, weighing in at 5.4 megabytes (base-10).

Right off the bat, performance looks good. For FTP downloads, Fetch is reporting ~22KB/s, which is quite similar or slightly better than what the Nuvolink was doing with this size of buffer. Watching Wireshark, I'm seeing a nearly identical pattern of TCP retransmits as with my firmware on Nuvolink. That seems to point to vsftpd being the underlying issue; like what @Chopsticks reported before, the choice of FTP server appears to have quite an impact on performance.

For FTP writes, there is some really bizarre traffic on Wireshark. I don't think that has anything to do with your changes. The code you branched from did a poor job of handling the ENC28J60 transmission bugs, and I'd bet a shiny nickel the issues here are my fault. Speed is very inconsistent, with Fetch reporting ~12.8KB/s, but actual performance is worse. The test file did successfully transfer in the end and the SHA256 checksum matched the origin file, so no data loss or anything like that.

Over on the AppleTalk side, I got 77KB/s downloading. Uploading was a dreadful 9KB/s, but again that's likely my shoddy transmission code at work.

To give comparisons, here's the same values after I re-flashed the board to my current 'master' branch. Only the firmware was changed, everything else was left exactly the same.
  • FTP down / up: could not be tested, Fetch kept crashing with a type 10 error when listing the server contents. I'll have to investigate, likely my focus on Nuvolink has broken something here.
  • AFP down: 43.2KB/s
  • AFP up: 37.2KB/s
Seems pretty clear your packet-stuffing discovery with the Daynaport helps out quite a bit. Another interesting tidbit: the Daynaport driver really did not like it when the Linux VM started sending it ARP requests during the AppleTalk transfers: it wouldn't respond to ARP, and the transfer stalled out for a few seconds each time those packets showed up. I re-did the transfer after they stopped and things were fine, getting the results shown above.

I also have a mirrored copy of the same Mac system for the IIsi, just with the Nuvolink driver instead of Daynaport. Here are results for the Nuvolink, which should be pretty apple-to-apples:
  • FTP down: 33.3KB/s
  • FTP up: 192.9KB/s (according to Fetch, stopwatch says 98.2KB/s FWIW)
  • AFP down: 131.7KB/s
  • AFP up: 98.1KB/s
 

superjer2000

Well-known member
Thanks @saybur - Those results are interesting - network setup must play a pretty significant role - Your software setup seems to mirror what I had suggested before maybe with the exception of not updating your Open Transport to 1.3 - 68k custom install. (I am assuming you're running Open Transport here and I am also assuming you didn't do the "Easy Install" of 7.5.3 for any Macintosh but instead did a custom install specifically for the IIsi).

In my environment (Synology FTP server) I am not seeing any issues with transmitting packets. For comparison on my SE/30 with the Daynaport
Multipacket Firmware branch:

All from SE/30 (doomII.sit test file 6,281kb)
My AFP server is Debian server running Netatalk 2.2.6)

AppleShare UP: 65kb/s
AppleShare DOWN: 75kb/s
Fetch UP and DOWN: 80kb/s (Timed with a stopwatch - Fetch's point in time transfer estimates seemed reasonable if I was to average out the estimates over the transfer period.)

This seems to contradict the theory about issues with TX code, unless there is something it's interacting with on your network that isn't present on mine - very odd! Note that I still haven't changed the buffer setup either.

I did pull out my SE FDHD as well to see if the Daynaport Multipacket firmware worked there. It did out the gate but I did end up increasing one of the delays slightly in link.c which helped to speed up AppleShare transfers - Without the increased delay, there was a bit of a pause at various points in the transfer - slower cpu driver induced retransmissions I'm guessing but I didn't have WireShark open. Increasing the delay slightly eliminated those delays and I got 37kbs down on AppleShare on a 68000 Macintosh SE. I couldn't test Fetch on my SE though. I only had Fetch 4 installed which kept crashing with an out of memory error on my 4MB SE. I was then going to try to use Fetch 3 but it was Stuffed... Unfortunately it seemed to be a Stuffit 5.x archive and I couldn't run Stuffit 5.x on my SE...

Maybe I'll drag my IIsi out and see if there is anything odd happening there. I'll PM my email address as well - if you send me the image file you had on your SD card you were using when testing the updated Daynaport firmware, I can throw it on my Scuznet and try it on my IIsi and report back.
 
Last edited:

saybur

Well-known member
Your software setup seems to mirror what I had suggested before maybe with the exception of not updating your Open Transport to 1.3 - 68k custom install. (I am assuming you're running Open Transport here and I am also assuming you didn't do the "Easy Install" of 7.5.3 for any Macintosh but instead did a custom install specifically for the IIsi)

Ah, I should have said something about that: this system is running MacTCP, mostly due to how little RAM it has (5MB). I'll find some 4MB 30 pin modules and try OT. I'd be curious if my current Daynaport issues go away with it too.

This seems to contradict the theory about issues with TX code, unless there is something it's interacting with on your network that isn't present on mine - very odd! Note that I still haven't changed the buffer setup either.

That is really weird. This network is two computers: the IIsi and the Linux VM via a bridged adapter. There is basically no background traffic at all. At least on this hardware (my original V1 prototype), I was having unstable transmissions (lots of stalls / error flags set) until I redid things to be much more aggressive with re-transmitting. I have no idea why there's a difference. Maybe I have something else going on here, like sketchy hardware somewhere on the networking side.

I will say that things are vastly more stable on the transmission front after making those firmware changes, FWIW. The ENC28J60 is a buggy chip. If you want some nerdy entertainment, it's worth reading the errata sheet; a significant number of pretty important parts (full duplex, the pattern filter, etc) have severe enough flaws to make them basically unusable.

I doubt this is it, but could you take a look at the date stamp on the ENC28J60, to the right of the Microchip label? Mine starts with 1904 for reference.

I got 37kbs on a 68000 Macintosh SE

Wow, very nice! That is significantly better than the last time I tried the Nuvolink on my SE.

Maybe I'll drag my IIsi out and see if there is anything odd happening there. I'll PM my email address as well - if you send me the image file you had on your SD card you were using when testing the updated Daynaport firmware, I can throw it on my Scuznet and try it on my IIsi and report back.

Before you have to do that, let me pull the modern system out of the network and start trying transfers with a period-appropriate server here. I have a feeling that your thought about networking differences might be on the money as the trouble source. To replace it, I've got a good selection of PPC hosts, just in case something about this Debian bridged networking setup is the source of the problems.

I also did some research on the current 'master' branch Daynaport issues, which are... also weird. Fetch reliably crashes with a type 10 error. MacTCP ping will get one packet out right after a reboot, then will timeout on all subsequent packets. Each packet is being seen by Wireshark, and I've got the board's debugging interface reporting that replies are being sent to the host correctly - it's like the driver on the Mac is just rejecting the packet. Even more strangely, if I increase the packet payload size within MacTCP Ping it starts working fine! Packets larger than about 300-400 bytes are totally reliable. AppleTalk works fine. The Echo Test for Daynaport is completely fine too, even with small packets. Very strange.
 
Top