Fast! CompactFlash for Macintosh PDS/NuBus

volvo242gt

Well-known member
@David Cook tbh compactflash is still current (even if specifically eg in term of consumer cameras its a limited market atm) and you can still buy a new 64mb media if your particular industrial machine still needed it. like this for example https://synchrotech.com/products/compact-flash-industrial-grade?variant=42012800680024

so I guess the only real limit is that of course non-consumer medias costs a lot more tho

(even then this one has no prices but I imagine that since it seem to be consumer-facing media its likely somewhat cheap for the smallest 1gb one tho https://us.transcend-info.com/product/memory-card/compactflash-133 )
I'd just recommend a 2GB or 4GB card, and partition it, if using on a 68K Mac. Can be had cheap on the used market, since most DSLRs that use CF are now equipped with 32GB or larger media, like this 128GB card - https://www.glazerscamera.com/collections/cf-cards/products/extreme-pro-160mb-s-cf-128gb

@zigzagjoe Maybe CFFM5000 would be a good name for it, since it's not unlike the CFFA3000, just for Macintosh machines and more powerful.
 

Trash80toHP_Mini

NIGHT STALKER
Lucked out for a change! Looks good to me so far, dunno about connector availability. How does memory mapping line up?
Screenshot 2025-02-11 at 22-08-16 PowerBook 520_520c_540_540c - dn_520_520c_540_540c(1).pdf.png
Luckily the Blackbirds have a pokey bridged PDS slot, later systems are 25MHz if memory serves.

I'll drop this tangent now . . . couldn't help myself. :oops:
 

zigzagjoe

Well-known member
Lucked out for a change! Looks good to me so far, dunno about connector availability. How does memory mapping line up?
View attachment 83282
Luckily the Blackbirds have a pokey bridged PDS slot, later systems are 25MHz if memory serves.

I'll drop this tangent now . . . couldn't help myself. :oops:
Looks identical to the 030 PDS, so with an electrical adapter it ought to just work as far as the rest of the prototype design is concerned.
 

Trash80toHP_Mini

NIGHT STALKER
@zigzagjoe if someone here or over there comes up with a source for the PDS connector, I've got a small box full of Blackbird bits and pieces headed your way. Bringing those internals to life and put to good use in a test rig chassis will be most entertaining, if and when. ;)

Hope you don't mind this tangent, I'm fascinated by your progress as is. Can't help but imagine the bits on your prototype pic melded with Bolle's NIC equipped SE/30 riser.
 
Last edited:

zigzagjoe

Well-known member
@zigzagjoe if someone here or over there comes up with a source for the PDS connector, I've got a small box full of Blackbird bits and pieces headed your way. Bringing those internals to life and put to good use in a test rig chassis will be most entertaining, if and when. ;)

Hope you don't mind this tangent, I'm fascinated by your progress as is. Can't help but imagine the bits on your prototype pic melded with Bolle's NIC equipped SE/30 riser.

I'd be game to give it a shot (of course, need the connector).

Here's today's work: an integrated early-boot partitioning utility. If a card is installed but does not have a valid APM, this will pop up before the happy mac (or after Welcome To Mac, if the boot order was overridden by the user). It allows some simple partitioning options (as HD Setup, etc, will not be able to work with my CF). For more advanced partitioning, an image from a Zuluscsi can be DD'd to the CF card.

This means the card is driver/utilityless, install the card with a CF and it'll automatically select itself as a bootable device. If the card isn't partitioned, it'll prompt you as below. Then, boot the system normally: Mac OS will ask you to initialize the new volumes after which you can drag a system folder to them and go. This process writes a valid APM to the CF card, so you can take an image of the card and use it elsewhere (ie SCSI emulator, though you need to use HD Setup to write a valid driver to it).

You can also for any reason hold 'C' and 'F' to return to this dialog at boot time to repartition or just to view card status. Additionally, a CF icon will appear in the bottom right at boot to reflect if the CF card was detected correctly and driver loaded. Or, if the card is not functional, it'll be crossed out.

The "fun" part of this is due to needing to run at early boot time, it can't use much of the usual toolbox routines. So the entire UI for this is handcrafted quickdraw: rectangles, lines, text, etc, and my own event handling. It's not as much fun as it sounds, and took a veeeery long time. It behaves nearly identical to dialogs drawn by Mac OS including cancelling clicks (holding mouse and leaving button).

1739497018350.jpeg1739497024817.jpeg

Other fun development notes

In the Control and Status calls infoCC and StsCode respectively, you need to write 22 bytes from your DrvSts2 structure into the csParam buffer, not the sizeof(DrvSts2) (which is 30). Otherwise this will overflow storage and cause crashes. DrvSts2 is a superset of DrvSts, so it's fine.

The Prime routine will be called in 32bit mode, even if the system is nominally in 24 bit mode. At least on IIsi ROM. I haven't dared to test on the stock SE/30 ROM yet, and the idea terrifies me.

QuickDraw and the keyboard/mouse low memory globals are available at BootRec time when occurring at boot.

System Picker does NOT correctly set PRAM for booting from NuBus/PDS cards. Don't use it, use startup disk instead.
 

gcp

Well-known member
This is badass -- I especially appreciate your attention to detail with the look _and_ interaction patterns in your utility. Awesome stuff!
 

zigzagjoe

Well-known member
This is badass -- I especially appreciate your attention to detail with the look _and_ interaction patterns in your utility. Awesome stuff!
May as well do it right if I've got to implement a UI! I want this to be user friendly, even easier than a SCSI emulator, and looking right is I think a big part of that :) I did rework the Icon to be better looking. Shift may be held before it opens in order to increase the max size to 4000MB (requires system 7.5+).

The icon will also show in the bottom left of the screen at boot to indicate status, much like an extension would.

1740086150258.jpeg

A brief test in IIsi had it hitting 7.2 MB/s reads and writes when combined with one of my booster cards. More work will be needed as timing appears marginal, most CF cards weren't working but it's likely IIfx could match or exceed that.

More fun facts of the day about disk drivers.

The control routine for drive info expects details in the format described in Inside Macintosh 5 p471, with more details of note in the PB 150 Devnote. I set it as follows: *((uint32_t*)&pb->csParam) = 0b011000000001; // Unspecified drive type, internal, SCSI, fixed, primary drive.
Realistically, it doesn't seem to matter much as I was returning nothing at all for a while and it did not change behavior at all.

You can use slot addresses greater than $B in IIsi and SE/30, slot manager will happily scan these and honor any DeclROMs found. Of course, you have no IRQ, but the CF doesn't need one; so I will default to slot $D.

Setting the checksum and start of driver to 0 in the APM header describing the device driver partition seems to be sufficient: it won't cause the system to immediately crash when scanning a SCSI volume that is missing a driver (ie. what would happen if you were to DD a CF card image off one prepared by my utility). Using HD setup you can update the driver and make it bootable. If you like you could write that image back and it'd work too as my code does not require nor use the driver on disk. I could technically put one on there, but that'd be 10KB of mostly-useless data in a 32KB flash - pass.

Mac OS doesn't seem to correctly deal with a case where a prime routine (reads/writes data) returns less data than expected with an OK status. Despite having fields to indicate how much data was read/written. It may be better to return an error status instead to force Mac OS to notice something's gone wrong, but due to lack of documentation it's not clear if you return an error should you still update the device position/offset fields.
 
Last edited:

jeremywork

Well-known member
This is an exciting project! Excellent work!

A brief test in IIsi had it hitting 7.2 MB/s reads and writes when combined with one of my booster cards. More work will be needed as timing appears marginal, most CF cards weren't working but it's likely IIfx could match or exceed that.
As a reference point, I've hit a peak of 9.6MB/s using a 32GB SanDisk Extreme (120MB/s) > Stratos CF AztecMonster > Sixty Eight Thousand SCSI Bolt > IIfx (50MHz clock). Seems likely the SCSI Bolt is actually the bottleneck here since it's a 10MB/s fast/narrow bus.

I'll be excited to see what you can hit on the IIfx!
IMG_0376.png
System Picker does NOT correctly set PRAM for booting from NuBus/PDS cards. Don't use it, use startup disk instead.
In my experience Startup Disk will correctly set the disk/interface, while System Picker will correctly set the partition/system folder (if you have multiple bootable partitions and/or system folders on the same physical disk.)

Using a combination of both has worked well for me, but maybe I've missed a more elegant solution.
 

joevt

Well-known member
In my experience Startup Disk will correctly set the disk/interface, while System Picker will correctly set the partition/system folder (if you have multiple bootable partitions and/or system folders on the same physical disk.)
Each partition in an Apple Partition Map has a "Startup" status bit. I suppose it's like the active partition flag in MBR formatted disk.

If my Old World Mac isn't booting from the currently selected partition on an SSD, I can connect it to a modern Mac and use iPartition.app to set which partition has the Startup bit (or I can use dd and xxd to read and set the flags).

A partition may have multiple System Folders. bless on a modern Mac can set which folder of a partition gets booted. This info is stored in the drFndrInfo of a HFS or HFS+ partition.

dumpvols.sh is a script you can use to view those status bits and the drFndrInfo of a disk.
https://gist.github.com/joevt/a99e3af71343d8242e0078ab4af39b6c

Attached contains PRAM notes. I suppose default boot is stored in DefBoot (if the Mac uses XPRAM). The supermario source code on github has some classic macOS PRAM info. Which Macs added XPRAM?

Other XPRAM notes:
https://okmij.org/ftp/xPRAM.html
 

Attachments

  • XPRAM notes.txt
    7.9 KB · Views: 3

zigzagjoe

Well-known member
This is an exciting project! Excellent work!


As a reference point, I've hit a peak of 9.6MB/s using a 32GB SanDisk Extreme (120MB/s) > Stratos CF AztecMonster > Sixty Eight Thousand SCSI Bolt > IIfx (50MHz clock). Seems likely the SCSI Bolt is actually the bottleneck here since it's a 10MB/s fast/narrow bus.

I'll be excited to see what you can hit on the IIfx!
View attachment 83652

In my experience Startup Disk will correctly set the disk/interface, while System Picker will correctly set the partition/system folder (if you have multiple bootable partitions and/or system folders on the same physical disk.)

Using a combination of both has worked well for me, but maybe I've missed a more elegant solution.

Nice! I imagine the SCSI bolt is using DMA. I will have to borrow an IIfx to test with as in theory the tighter timings at 20mhz ought to be fine (from a CF timing perspective) but some cards like it, some don't. I'm hoping the IIsi will work as an intermediate test case too.

I'm incorporating bus drivers on a revised prototype to make it easier on the cards. Technically, it might be possible to use the "fast" address space with a PIO-6 card too but we'll see what happens. I admit, PDS isn't my main focus here so it's a bit of an extra if I manage to get something working there.

Your experience matches what I observed. I just ended up special-casing system picker so my ROM will pick up its intent and make the correct changes to the startup device instead.

If you felt like hacking on it you'd need to do something along the lines of taking the selected volume, locate the driver, and find its AuxDCE entry that will describe what slot and resource number to detect the special case. Or take the opposite approach and go via slot manager. Then for slot devices correctly update the PRAM.

Each partition in an Apple Partition Map has a "Startup" status bit. I suppose it's like the active partition flag in MBR formatted disk.

If my Old World Mac isn't booting from the currently selected partition on an SSD, I can connect it to a modern Mac and use iPartition.app to set which partition has the Startup bit (or I can use dd and xxd to read and set the flags).

A partition may have multiple System Folders. bless on a modern Mac can set which folder of a partition gets booted. This info is stored in the drFndrInfo of a HFS or HFS+ partition.

dumpvols.sh is a script you can use to view those status bits and the drFndrInfo of a disk.
https://gist.github.com/joevt/a99e3af71343d8242e0078ab4af39b6c

Attached contains PRAM notes. I suppose default boot is stored in DefBoot (if the Mac uses XPRAM). The supermario source code on github has some classic macOS PRAM info. Which Macs added XPRAM?

Other XPRAM notes:
https://okmij.org/ftp/xPRAM.html
Mac Plus added XPRAM. Believe me, I've done enough digging on PRAM to now to know what's going on here :)

The issue in particular is that System Picker is not special casing volumes on Nubus/PDS cards - these must have startup the startupDevice PRAM location (4 bytes at $78) in the form $00000SRR, where S is slot # and RR is the resource number. This is required for a nubus card presenting storage to have its BootRec executed at early boot time so it can install its driver and be booted from. I detailed a bit on this earlier in the thread.

However, system picker just puts the driver refnum into startupDevice in conjunction with changes to the HFS volume flags. That's fine for SCSI, and floppy (?), but it renders Nubus cards unbootable as @jeremywork noted. I ended up special casing this as I do like using System Picker; my driver will stash its refnum in private slot PRAM and primaryinit will correct the startupDevice PRAM if required.


--------------------------------------------------------------------------------------------------------------------------------------------
Another dose of notes on disk drivers.

Any volumes added in DrvrOpen belonging to this driver should be dequeued and their entries disposed of if DrvrClose is called (Mode32, A/UX only known to do this...)

System picker sets startup info to the driver refnum (incorrectly). But you can detect and check for that in PrimaryInit, if you choose...

Prime notes: This is the routine used to read/write data from drivers.
IO will always be in 512 byte multiples unless you support the character-based routines.
Prime should return ioErr if anything went wrong, but still advance the drive position up to the point the error occurred and update ioActual.
Support of 4GB volumes on sys 7.5 requires treating the long (signed long int32) parameters of IOParamBlock as unsigned longs. Nice and easy.

Expectation of the rdVerify mode is to check buffer contents against data read using the parameters and return dataVerErr if it doesn't match
Not sure if it's actually used (outside of floppy formatting) or is even considered valid with a HDD/HD20 type device.
If a partial read/write is performed (not all the requested data) then an error must be returned, it will not be handled correctly otherwise

Valid Prime return codes:
nsDrvErr (no such drive) is considered a fatal error and should never be retried
offlineErr is a temporary failure e.g no medium present
ioErr should be used for actual issues accessing the device - if it's fatal or not is up to Mac OS
dataVerErr only should be used with rdVerify
noErr (duh)

Control call Eject isn't called on Mac OS 7.1 just before shutdown. Might just be the A/UX version? 7.5 certainly does this.

If you have no need for an interrupt, there is nothing precluding you from using slot $C, $D address spaces. I'm going to default to $D as this will mean it should never have a slot conflict with another card.
 

zigzagjoe

Well-known member
Latest update on this is I've hit functional completeness on the driver, after nearly a month of work, and now comes the fun part: extensive software testing. I did tweak the utility a bit more...even though it's a utility that most people will see once and never again (ideally), I added some more functionality to list existing volumes (if any) before wiping 'em. Oh, and it's got a name now: NuCF.

1740784663411.jpeg

I've got some revised prototypes coming for field testing purposes, as well as a nubus bridge board to begin developing in that direction too. While the form factor is a revised version of my current prototype, electrically it is expected to be similar enough to the final design to allow for hardware compatibility testing.

For the "final" PDS version I am thinking something like the below form factor: mount the card socket on the "rear", with the card overlapping (but not contacting) the eurodin on top. I said I was done with these blasted passthrough card designs and yet they're sucking me back in! I really hope I can manage this in 4 layers, though, not 6, as I'd like to make these as affordable as at all possible.

There's not really otherwise a great way to mount the CF socket without making the card taller, and this would allow a single design to serve SE/30, IIsi, IIfx. I tried on the protruding end of the card too, but I think it'd end up too long for IIsi and realistically no position allows for swapping the card easily. Given this is intended to be a fixed-disk replacement, I don't expect much swapping to be needed anyways. But, I intend to put termination resistors if possible as to not preclude the possibility of use with a CF-card extension cable. I don't intend to support hot swapping though.... That way madness lies.


1740784319961.jpeg1740784293528.jpeg1740784306587.jpeg

---------------------
Last bit of notes on the driver stuff, I think.

I've posted a template driver here https://github.com/ZigZagJoe/Mac-StorageCard-DeclROM/
This should capture most of the annoying bits about device drivers that aren't well documented (IMO). Of course, after doing this all the hard way I found the EDisk ERS (pg 472) within Apple Blue Book Vol2 is the closest thing to authoritative disk driver reference in one spot. Ugh. The older inside macintoshes also have info (spread across the books...) that is not in the newer inside mactintosh, so watch out for that too.

dNeedGoodByeMask on the driver is called if the driver needs to know prior to system shutdown!
definitely needed on system 6 as otherwise no notice before shutdown. can be a problem with drive caches!
it is also called at heap reinitialization (multifinder load, on system 6) so don't do anything irreversible, but definitely make your data safe for shutdown.... possibly there's a way to detect if a shutdown is in progress, haven't tried.

Without minorBaseOS and minorLength, you don't get a valid devBase! Oops.
 

LaPorta

Well-known member
Small suggestion: why not mount the CF holder on an angle so you can access and slide out the CF card while the unit is installed in the chassis?
 

zigzagjoe

Well-known member
Small suggestion: why not mount the CF holder on an angle so you can access and slide out the CF card while the unit is installed in the chassis?

That angle does allow a straight pull in the IIsi and IIfx, but for SE/30 I couldn't come up with an angle that is any more accessible while being mechanically practical. CF cards seat into that socket firmly and it's not going anywhere without a good grip on it.
 

zigzagjoe

Well-known member
I am wondering if this would fit (and work) in the top slot of a riser like @Bolle ’s Ethernet + Carrera slot inside my SE/30
That will be the million dollar question, yeah. With the design I'm thinking it will mechanically fit, and even will fit in conjunction with one of my grayscale cards or LCD cards on top. But if it will work at all will be a wildcard as that's asking a lot of a system originally designed for one add-in card, having a total of 4 at that point. Proof will be in the pudding, I suppose...
 

eharmon

Well-known member
That angle does allow a straight pull in the IIsi and IIfx, but for SE/30 I couldn't come up with an angle that is any more accessible while being mechanically practical. CF cards seat into that socket firmly and it's not going anywhere without a good grip on it.
Probably worth seeing if the signal is strong enough for a CF extension cable to work. One of those could route it to the back panel.
 
Top