SCSI to Ethernet Adapter on New Hardware

[saybur]

I've been spending some time recently making a SCSI to Ethernet adapter using all-new hardware.  I managed to snag a Nuvotech NuvoLink SC off eBay, and after doing some analysis of how it talks with the driver over SCSI, I've decoded the vendor-specific parts of the communications protocol and implemented a compatible version on a custom PCB.  Here's some pictures of the board and test setup:

https://imgur.com/a/ZJFfrSG

The board is based on a ENC28J60 Ethernet chip and XMEGA AU series microcontroller, with a SD card thrown in the mix for good measure to emulate a hard drive.  Networking has been tested under System 6 and EtherTalk seems to work fine with an unmodified Nuvotech driver.  That said, this is definitely just a prototype, so there are plenty of things that still need to be done:

1. Test with MacTCP or OpenTransport on newer versions of the OS. There is conflicting information I've found online about how compatible this particular driver is.
   
2. See about compatibility with other SCSI to Ethernet adapters.  Focus bought Nuvotech and appears to have re-released everything under their brand, so getting their driver working would likely be pretty easy.  I'd ideally aim to get compatibility with a driver known to be stable with Open Transport in particular (if one exists).  Other, wiser members might have some ideas on that.
   
3. Finish the HDD emulator, which is only partially working.
   
4. Spin a board revision that has USB for firmware updates instead of the vendor PDI interface, and fix some of the other annoying problems I caused myself with the hardware design.
   
5. Fix the significant bugs that I'm sure still are lurking in the code.
   

It's been a bit of a slog to get to this point, and I mainly wanted to share this with the community and see what others thought of the project.  Plus, I was pretty happy to have pictures of this thing working: I've been banging my head on my keyboard with it for a while.  )

My short-term plan for this board is to release the protocol documentation, design, and code under GPLv3 and let others poke around with this.  I'll be working on getting the firmware into a slightly nicer state over the next few days and then throw all the files up on Github.  I do not have any interest in making this to a kit / product / etc: if anyone would like to take a stab at that, you'll be free to do so under the license.  At minimum, the protocol documentation will hopefully be of some value to the community.

If anyone has questions or comments, please fire away.  Thanks for reading!

 

[blusnowkitty]

I've been looking for an excuse to buy a SMD hot air gun. I feel like this is that excuse.

 

[cheesestraws]

This is seriously cool.  I'd love to play with one of these.  Time to learn surface mount soldering...

 

[uyjulian]

Would this be any faster than serial?

 

[saybur]

Would this be any faster than serial?

It should be significantly faster.  AppleTalk caps out at 230.4kbits/sec, so maximum throughput on that should be something like ~28 kilobytes per second.

You do have me wondering though, so I'll check and see how this performs vs. the real Nuvolink SC and a NuBus Ethernet card and report back.

 

[dougg3]

Wow, that's awesome! Really nice work. This seems like for most use cases, it's going to be a much better option than LocalTalk-to-Ethernet adapters.

 

[LaPorta]

This would be very, very useful for networking with my other machines and moving things back and forth.

 

[saybur]

Per request, here are some rough speed tests.  All are on a Mac IIci running 7.5.5, copying files from the network to the local HDD (SCSI2SD v4 clone) via the devices below.  The remote computer is a PC running Basilisk under Linux.  The files being copied are a directory of several items ranging from ~800kB to 5MB.  I timed the transfers with a stopwatch, so take the results with a grain of salt.

Farallon EtherWave NuBus Card: 38.6 MB in 1m53s, or ~2.73 Mbit/sec

Nuvolink SC, real hardware: 38.6 MB in 4m23s, or ~1.17 Mbit/sec

"scuznet" emulated device: 5.1 MB in 35s, or ~1.17 Mbit/sec

Note that a single file was used for the emulated device, because this testing revealed there is a major firmware bug: larger batches of files are causing the device to lock up.  I'll have to dig into that one and figure out what exactly is going on.  That's what I get for only testing with smaller text files initially, lol.  Firmware bugs, whee!

 

[archer174]

Wow, this is very cool! Will the hard drive emulation work in parallel with the ethernet as a separate SCSI device?

 

[cheesestraws]

Firmware bugs, whee!

Network firmware bugs are the best bugs ;-)

 

[olePigeon]

This is seriously cool.  I know it's super early in development, but you know what would be nifty?  Design a SCSI2SD with a 2nd SCSI port that bridges perfectly to this card.  Then you secure the two together with some standoffs.  You'd then have a combo storage + SCSI device that'd sit nicely in the drive bay.

 

[uyjulian]

I'm curious if it's possible to make a SCSI port multiplier, or if it's daisy chain only.

There's also the Scuzzygraph, which provides a display through SCSI. A device similar to that would be nice to easily add more displays.

---

I'm curious if some processing is offloaded (similar to the Apple Ethernet NB card).

 

[saybur]

Wow, this is very cool! Will the hard drive emulation work in parallel with the ethernet as a separate SCSI device?

The device responds as through it were two different targets, one a HDD and the other the Ethernet device.  To the computer they appear to be separate.

This is seriously cool.  I know it's super early in development, but you know what would be nifty?  Design a SCSI2SD with a 2nd SCSI port that bridges perfectly to this card.  Then you secure the two together with some standoffs.  You'd then have a combo storage + SCSI device that'd sit nicely in the drive bay.

This PCB version is designed to be put into one of these cases when finished:  https://www.hammfg.com/part/1455L802

I was hoping that one of these should (hopefully?) remove the need to equip a computer with a SCSI2SD at all.  Final testing will require the HDD emulation being finished, but the SPI clock for the SD card should allow speeds in the range of ~1.5 MB/s.  That's not great, but I can't imagine pushing a Mac Plus/SE/IIsi/etc much beyond that on the SCSI bus anyway.  That said, the schematic will be open, so anyone wanting to experiment with alternate approaches will be welcome to do so!

I'm curious if it's possible to make a SCSI port multiplier, or if it's daisy chain only.

There's also the Scuzzygraph, which provides a display through SCSI. A device similar to that would be nice to easily add more displays.

---

I'm curious if some processing is offloaded (similar to the Apple Ethernet NB card).

I'm guessing it would be possible, but it could cause signal integrity issues when using faster transfer rates.  As it was, the SCSI-2 spec has limits on cable length due to propagation issues, which having a split up topology couldn't help with.  The 25 pin SCSI port also wasn't well regarded by SCSI fans at the time due to having bad grounding, so anything that adds to that might be troublesome.

The command protocol that Nuvotech used doesn't do hardly any offloading to the device.  The main commands the driver executes are a vendor-specific CMD 0x05 command that dumps a packet to the device for sending, and reads data back by having the device reconnect and write a slightly modified version of what it found on the wire.  Almost all processing is done on the computer CPU.  I do wonder if other devices do things differently, given how little standardization appears to have happened with these things.

 

[saybur]

Well, I think I've identified the problem.  During packet reception, after a DATA IN phase when the packet is sent, the device moves to MESSAGE OUT and usually gets NO OPERATION.  It then moves either back to DATA IN to send another packet or moves to MESSAGE IN, sends DISCONNECT, and releases the bus.  However, there are times that instead of getting NO OPERATION it gets an extended message code (0x01).  The emulated device treats that as an error, sending back MESSAGE REJECT: apparently, that is not OK, and causes subsequent operations to fail.  After implementing basic extended message handling, the exact message bytes sent each time I've observed this are [0x01, 0xFF, 0x00, 0x2B].  Unfortunately, that is a vendor-specific message, and my cheap logic analyzers are having problems catching the real device performing this relatively rare step.  I'm guessing it's probably just "resend last packet," so I'll do some more debugging and see if grumpy-pants over here will accept that response

 

[olePigeon]

@saybur  I probably didn't articulate myself very well.  I didn't intend for the device to be dependent on the SCSI2SD, just that it'd be cool if you could piggyback them together in addition to having one externally.  That way you could put both of them in a single drive bay, then snake out an ethernet cable somewhere (maybe through the Lock slot.)

 

[cheesestraws]

During packet reception, after a DATA IN phase when the packet is sent, the device moves to MESSAGE OUT and usually gets NO OPERATION.  It then moves either back to DATA IN to send another packet or moves to MESSAGE IN, sends DISCONNECT, and releases the bus.

What are you using to see what's going on on the SCSI bus, just out of interest?

 

[Compgeke]

@saybur  I probably didn't articulate myself very well.  I didn't intend for the device to be dependent on the SCSI2SD, just that it'd be cool if you could piggyback them together in addition to having one externally.  That way you could put both of them in a single drive bay, then snake out an ethernet cable somewhere (maybe through the Lock slot.)

But he's already building in disk emulation support - negating the need for a SCSI2SD. 

 

[olePigeon]

Oh, I missed that part.  Nevermind, then.

 

[saybur]

What are you using to see what's going on on the SCSI bus, just out of interest?

It's a pair of cheap 8 bit USB logic analyzers, using sigrok to capture the data to VCD files.  You can see them in the pictures attached directly to the 50 pin internal SCSI port on the open-top Mac LC on my desk.  I've got some separate programs I threw together to merge the captures, sync them based on a shared line, and decode the SCSI information.  This method only gets 15 lines of the 18 SCSI lines, but /ATN, /RST, and /DBP can be (mostly) inferred from the other signals.  The bigger problem I've had is getting the captures reliably synced up: the clocks on these units must be really bad, since they drift quite a bit over a capture period.