TashTwenty: Single-Chip DCD (Hard Disk 20) Interface

[Tashtari]

Somebody stop me.

Introducing... TashTwenty!

Elevator Pitch

It's a DCD (the interface used by the Hard Disk 20 to plug in through the disk port) interface, contained entirely within a PIC16F1825 (14 pins, ~$1.70) microcontroller. It bit-bangs the low-level IWM protocol using the read/write and phase lines on the "DB"-19 disk port, no programmable logic devices of any kind required.

(Despite the name, there is nothing limiting it to 20 megabytes.)

Project Status

Nascent. Low-level protocol implemented and apparently working with a rudimentary implementation of the higher-level command/response protocol.

Caveats

Much more testing is necessary before the implementation can be considered reliable, especially of the interrupt/holdoff flow. Also, crucially, it is not yet determined what the chip will interface to and how.

What's Next

Answering the question of what to interface to, by necessity. The PIC has just enough RAM to hold a single transfer buffer in either direction, and its own flash (that which is not already used by the firmware) amounts to about 4 kilobytes, so anything self-contained for testing is pretty much out. Fortunately, there are four pins free on the microcontroller, so the possibilities are numerous. The obvious one is interfacing to a flash memory card, for to make something like the SCSI2SD... could be interesting. There's also the possibility of a UART or SPI or I2C interface to a Raspberry Pi host that reads and writes a locally hosted image file. Combining this with my other projects could pack a lot of neat functionality into a single RPi, but it might not be sufficiently self-contained to be appealing as a product. In either case, I have to deal with the fact that the mac is 5V logic and both the RPi and SD cards are 3.3V...

The Eureka Moment

 

[cheesestraws]

The PIC whisperer strikes again ;-)

 

[olePigeon]

Somebody stop me.

I say the flood gates are already open. Might as well go for it.

I like the idea of the SCSI2SD. If it were implemented somehow and added a 20 pin connector, it could then be put inside an HD20 enclosure for people with dead HDDs.

 

[Tashtari]

it could then be put inside an HD20 enclosure for people with dead HDDs

Ooh, I didn't think of that, retrofitting an HD20 enclosure is a neat idea. I don't have one, though, so I might have to just watch and envy someone who does...

 

[Tashtari]

It BOOTS! It legit boots from an SD card. Aw yisss.

 

[cheesestraws]

!!!!!

 

[Tashtari]

Aaand the code, nearly forgot:

GitHub - lampmerchant/tashtwenty: An interface for the DCD protocol used by Apple's Hard Disk 20.

An interface for the DCD protocol used by Apple's Hard Disk 20. - lampmerchant/tashtwenty

github.com

 

[demik]

That's awesome ! Good job !

 

[superjer2000]

I love your project write ups. This is super cool!

 

[Tashtari]

I can provide a little more background, if anyone's interested!

As you may know, the HD20 was the result of Apple doing a bit of cheating. The Macintosh exemplified Steve Jobs's idea of the computer as an appliance, with everything built in and only the two serial ports for expansion, and even those had the pre-ordained roles of "printer" and "modem". However, with a maximum baud rate of 230.4 kbps, they were a bit on the slow side for disk access, though I believe some companies offered hard drives that connected this way. In making the HD20, Apple decided they'd use their insider knowledge of the IWM to create a communication protocol for storage devices that signalled at the effective rate of 428.38 kbps. Hardly blistering, but at least faster than the serial ports could provide, and they called it DCD for "Directly Connected Disk". It's described in a leaked document here, preliminary but nearly complete. They might have intended it for wider use, but it ended up that the HD20 was the only device that ever made use of it.

DCD uses the "phase" lines, which ordinarily serve to select registers on a floppy disk drive, to indicate states of the protocol, and each combination translates into the state of some signals called HOST, HOFF, and !HSHK, as well as the purpose of the RD (read) line, the device's lone means of communication back to the mac. In addition, the fourth phase line, PH3, was repurposed to become a signal which, when pulsed, indicated that the mac wanted to talk to the next device in a chain. The DCD protocol was flexible enough not only to support multiple DCDs from the one external floppy drive port, all using the same shared set of signals, but also to support an external floppy drive on the end of the chain. Clever!

It was the behavior of those signals, not in bitbanging the serial protocol itself, where the biggest risk lay in implementing this protocol on a microcontroller. If I was using a programmable logic device, it'd be straightforward - make up a circuit like the one in the document to implement the interaction between PH3 and the enable line, translate the remaining PH* lines to their logical equivalents, and have the microcontroller react to those - but in the absence of such a device, the microcontroller would have to act on its own, and act fast. Particularly crucial was the process of identifying that the device on the floppy port was a DCD at all. Three of the eight states (eight combinations of the three remaining PH* lines) were dedicated to having the RD line be either high or low in a combination that would be impossible on a floppy drive, thus signalling to the mac that this was not a floppy drive but a DCD.

I ended up doing this with some jump tables, designed to put as few instructions between a transition on one of the important signals and the reaction to that signal. With an 8 MHz instruction clock on the PIC, the reaction time ended up being about one microsecond on average. I was afraid this was going to be too slow, but luckily, it wasn't - at least not on a Classic, itself running at 8 MHz (faster machines TBD!), and the device was recognized and talked to, resulting in the first photo above with my silly icon. I wanted my BMOW moment.

I must give credit to BMOW for inspiring this project as being possible at all, as well as for pointing out that the first two bytes after the sync byte were for the length of a command and its expected response. Happily, though, the leaked document from May 1985 turned out to be a nearly accurate representation of the final protocol, without which fact the project would almost certainly have failed, as I lack his skills with MacsBug and such.

Anyway, future work on this project will likely include making less wasteful use of the MMC card. Currently I'm doing single-sector reads and writes, which are inefficient, and in the case of writes, probably wearing the NAND out a lot faster than necessary. Additionally, I may make an alternate version of the firmware wherein the SPI interface is used to connect to a Raspberry Pi or similar, to talk to a server of sorts there that will make alterations to an image kept elsewhere. Would be silly but neat to say that I've booted a mac off of a ZFS zvol, accessed over the network. Also neat would be to make a pi hat for this, one that can stack with the TashTalk hat and the hopefully-eventually-coming TashKM hat, to make a great big sandwich of Tash-projects, hee.

One pipe dream might be to figure out some more commands in the protocol. You can get a lot of mileage, evidently, out of the read, write, and device-status commands, enough for a working device, but if you choose to initialize a disk, the mac sends out commands with IDs 0x19 and 0x1F, and what those mean is mysterious to me - they are not mentioned in the document. If anyone wants to go digging into the depths of the HD20 INIT, I'd love to hear what you find...

 

[Tashtari]

Implemented multiblock writes, updated Github.

I also had a go at making a PCB for this. I'm a pretty serious noob when it comes to making PCBs so I'm bracing for failure but hoping for success. It mates with one of SparkFun's level-shifting MicroSD boards, because no way am I going to try and integrate that design myself. Don't know how much interest there is, but if I succeed here, I might be able to put together kits for the cost of parts and such (probably on the order of $15-20 when all is said and done).

Here's what it looks like. Those pads are for soldering a "DB"-19 on, but as far as Eagle CAD is concerned, they're test pads...

 

[Tashtari]

Aaaand just as I submitted the order, I realized that I forgot to put in a pullup resistor for MISO. Fortunately, it hadn't been panelized yet so I could cancel and retry. Presenting: Rev 1!

 

[ajacocks]

This looks really neat. I’ll be watching to see how it comes out.

 

[Tashtari]

It seems I was working in "easy mode" when I was testing this on my Classic. I recently upgraded my 512K to a 512Ke, which has HD20 support in the ROM and generates an interrupt every time the mouse is moved, and started using that for testing. Now I'm discovering lots of interesting things about DCD's holdoff protocol, which is not accurately documented in the May 1985 document.

For starters, there is no negotiation. When the mac says it's going to suspend, it transitions from the communication state (1) to the suspend state (0), that's it, and the DCD just has to put up with it. Dealing with this required a modification to the receive code and a complete rewrite of the received-data-decoding code.

Unfortunately, the complications don't end there. At the moment the firmware survives some interruptions, but not all. The ones it doesn't survive cause the mac to send an 0x3F command, the meaning of which I do not know, though it carries a data payload with it. The firmware gives a dumb reply of the correct length, to which the mac replies with a NAK (an 0x7F command), so obviously something's not quite right there.

This is going to require some work, and quite possibly some better tooling. I'm currently using a USB oscilloscope and a second PIC with a hacked-up version of the firmware as a primitive protocol analyzer. Testing I'm doing from within the Finder. Hmm...

 

[Tashtari]

I think I've figured out the problem with suspends while the mac is the one transmitting. The IWM speaks in terms of signal inversions (inversion == 1, no inversion == 0), but when it's transmitting a sync byte (0xAA) to resume after suspending, it only likes to do that with the signal starting out low, so if it happens to be high after transmitting a bunch of data, it has to go low first. My code was interpreting the high-to-low transition as the beginning of the sync byte and the sync byte as a data byte, corrupting all the data that came after it in the decoding process. Problem ostensibly solved.

The suspends while the disk is transmitting are proving more elusive. Strangely, after a suspend and subsequent resume, the mac seems to be cutting the disk off before it's finished delivering its payload, then behaving like it gave no response at all and retrying the command after a long delay. Don't know what to make of this yet.

 

[Tashtari]

Progress is being made! I found a bug in the firmware where I wasn't decrementing the groups-to-send counter after going into a suspend while transmitting, so the firmware would send the right data but then include an extra group of junk data after it, which confused the mac. Correcting that seems to have solved the case of suspends where the disk is the one transmitting.

Unfortunately, I'm not out of the woods yet. I can still cause a disk error by moving the mouse too much while the mac is transmitting, and I'm not sure why. More wild mouse movements followed by more squinting at scope traces is required.

 

[Tashtari]

Aaaand I think I've got it. I wasn't able to capture this on the scope, but I think the problem was, once again, with that return-to-zero pulse and my code's handling of it. A quite small variation in timing could send the code down the wrong path, causing data corruption in its wake, but that is now resolved. Just some of the problems you experience when you send a microcontroller to do what could be done much more simply by a programmable logic device - but, on the other hand, where's the fun in that?

Boards were sent to the fab last Tuesday. I'm excited.

 

[Tashtari]

FOR POSTERITY

Details Missing or Inaccurate in the May 1985 DCD (Directly Connected Disks) Document

• The sync byte, in either direction, is always 0xAA, 0x96 is not used.
• When Mac is transmitting a command, the sync byte is followed by two more raw IWM bytes before the 7-to-8 groups begin. DCD transmits only a sync byte and does not transmit these extra bytes.
  • The first is 0x80 plus the number of 7-to-8 groups in the command being transmitted by Mac.
  • The second is 0x80 plus the number of 7-to-8 groups that Mac expects to receive in response.
• The holdoff protocol is completely different than specified.
  • In either direction, a holdoff is initiated by Mac transitioning from state 1 to state 0.
  • If Mac is transmitting, it will finish the 7-to-8 group that it has begun transmitting. This data is valid.
  • If DCD is transmitting, it must finish the 7-to-8 group that it has begun transmitting. This data will be treated by Mac as valid.
  • A holdoff is ended by Mac transitioning from state 0 to state 1. There is no negotiation.
  • If Mac is transmitting and WR is high at the end of the last byte transmitted before holdoff, it will transition WR from high to low right before transitioning back to state 1.
  • Mac will resume transmission with an 0xAA byte, followed by the bytes in the next group after the group where the holdoff began.
  • DCD must resume transmission with an 0xAA byte, followed by the bytes in the next group after the group where the holdoff began.
• The Controller Status (command 0x03) block is slightly different than specified.
  • The total size of the Controller Status block is 343 bytes (49 groups), not 532 bytes.
    • 336 bytes of data, 6 byte header, checksum byte == 343 bytes == 49 7-to-8 groups
  • The Icon field contains a 32x32 icon as a 1-bit bitmap, followed by its 32x32 mask, also as a 1-bit bitmap, for a total of 256 bytes.
    • The format of the bitmaps is identical to that of ICON resources.
  • The Filler field is replaced by a 16-byte Pascal string (first byte is length) that determines what appears in the "Where:" field of the Get Info dialog box.

Details Out of Scope for DCD Documentation But Useful to Know

• The IWM transmits and receives MSB first and the MSB is always set; the chip uses this for timing.
• The IWM transmits at its "fast" speed, 47/96 MHz, or approximately 489.58 Kbps, data cell width 2.043 us.
• The IWM's output (on WR pin) is in NRZI format (inversion == 1, no inversion == 0).
• The IWM's input (on RD pin) detects only falling edges.

Other details about the protocol at the signal level and the read, write, and controller status commands are accurately reported by the May 1985 document; other commands are unknown to me, but hopefully this information will be useful to anyone in the future who wishes to implement DCD.

 

[Tashtari]

Correction:

• The Controller Status (command 0x03) block is slightly different than specified.
  • The total size of the Controller Status block is 343 bytes (49 groups) 336 bytes, not 532 bytes.

Addendum:

• The checksum byte is chosen such that all data bytes in all 7-to-8 groups (not including the sync byte or the command/response length IWM bytes) sum to 0 modulo 256.

 

[Tashtari]

This just in from the department of "I can't believe that worked":



Stay tuned...