68K Globals & Async Low Level File Manager Functions

[jmacz]

I was working on some code that utilizes the low level file manager functions (PBxxxx) and wanted to try out the async IO capabilities, specifically using ioCompletion routines as opposed to completion polling.

The approach is straight forward: I fill out a ParamBlockRec, provide the address of a completion routine, called a low level function like PBRead, and then once the IO request completes, my completion routine is called with the a pointer to the param block record provided via the a0 register.

However, there does not seem to be any facility with these low level file manager functions to pass my own custom data (for example a pointer) which is the usual design pattern for callback routines. As it is, the only info I get in the completion routine is a pointer to the param block record. In this type of scenario, the ugly workaround is to use global variables and access the globals from the callback (completion routine). But looking at the documentation (Inside Macintosh), that won't work. Why? Because apparently the A5 register can be invalid when this completion routine is called and thus you don't have access to your application globals.

The documentation then suggests you utilize SetA5 / SetCurrentA5 so that you can access your application globals. But I'm scratching my head regarding how they expect that to work? The A5 pattern is usually to call SetCurrentA5 to get your application's A5 value (while the A5 is valid) and then during the interrupt or completion routine, you pass that saved A5 value to SetA5 (and also restore the old A5 using SetA5 at the end of your completion routine). But how do you pass the application's A5 into the completion routine (in order to use SetA5) when again the only value provided to your completion routine is the param block record?

I'm sure I can figure out a way to hack this to make it work, for example setting aside some memory that can be accessed via an offset, or stuffing a value into an unused field in the param block record, or something... but I'm stubborn and I'm trying to understand what was the "blessed" or proper way to access globals in your IO completion routine (ie. how do you actually implement the Inside Macintosh suggestion of setting the A5 so you can access globals from the completion routine)?

Has anyone worked on async low level file manager routines using a completion handler? If so, do you remember the best practice for accessing globals and/or accessing your own custom data structures from within the completion handler? Without that, there's not a lot of utility with these IO completion handlers because if you only have access to the param block record, what can you possibly do from your callback?

 

[Snial]

The approach is straight forward: I fill out a ParamBlockRec, provide the address of a completion routine, called a low level function like PBRead, and then once the IO request completes, my completion routine is called with the a pointer to the param block record provided via the a0 register.

However, there does not seem to be any facility with these low level file manager functions to pass my own custom data (for example a pointer) which is the usual design pattern for callback routines.

Shouldn't it be possible, and safe to provide an extended ParamBlockRec, with an appended parameter that contains a pointer to your custom data? If your ParamBlock rec is one of your own variables, either allocated with NewPtr or NewHandle, then the System software can't assume what's after your ParamBlock - i.e. it can't start using data that follows it, because it could be trampling on other legitimate data for your application or driver. This means you should be free to extend it.

So, you treat it like subclassing in C++ where subclass data is appended (but this also works in C of course):

typedef struct {
    // Whatever Apple defines the fields as.
}tParamBlock;

typedef struct {
    tParamBlock iParamBlock;
    tCustomData *iCustom;
    tMyExtraData iExtraData;
}tParamBlockExt; // Basically, it's a subclass!

void MyCompletionRoutine(tParamBlock *aBlock)
{
    tParamBlockExt *pb=(tParamBlockExt*)aBlock;
    pb->iExtraData=... ; // Some miscallaneous, but non-custom fields
    pb->iCustom->iUsefulField=...; // my custom data.
}

void AsyncIoSetup(tCustomData *aCustom)
{
    tParamBlockExt *aPbExt;
    aPbExt->iParamBlock->iCompletionRoutine=&MyCompletionRoutine;
    aPbExt->iCustom=aCustom;
    aPbext->iExtraData=0xdeadbeef; // just for fun.
    PBxxx((tParamBlock*)aPbExt); // PBxxx can just treat it as normal.
}

The iExtraData stuff is just there to illustrate another way of adding custom data, which you might not want or need.

 

[jmacz]

Yup, that should work and was one of the hacks I was thinking of but still curious if there’s an official non hacky way this was supposed to be done since Inside Mac references setting up the A5. Just curious if I am missing something obvious.

 

[cheesestraws]

I've always done it and seen it done by extending the ParamBlockRec (except that one time I saw it done with self-modifying code, and, no, please no). Never got the globals thing to work - but also haven't tried very hard. I'm fairly sure that if you need access to your application globals, passing the required A5 into the routine is your problem not the system's, and I think the blessed way to do that (it's somewhere in IM:Files, I'm relatively sure) is to extend the ParamBlock.

But if you're going to pass in an A5 value, you might as well just pass in the stuff you actually need and then your program will be better because you're not using globals and global state is bad for the soul.

 

[Snial]

Yup, that should work and was one of the hacks I was thinking of but still curious if there’s an official non hacky way this was supposed to be done since Inside Mac references setting up the A5. Just curious if I am missing something obvious.

I've always done it and seen it done by extending the ParamBlockRec (except that one time I saw it done with self-modifying code, and, no, please no). Never got the globals thing to work - but also haven't tried very hard. I'm fairly sure that if you need access to your application globals, passing the required A5 into the routine is your problem not the system's, and I think the blessed way to do that (it's somewhere in IM:Files, I'm relatively sure) is to extend the ParamBlock.

But if you're going to pass in an A5 value, you might as well just pass in the stuff you actually need and then your program will be better because you're not using globals and global state is bad for the soul.

Cool, then I think we're all aligned! Apologies if my Pascal and Symbian OS heritage conflicts with your naming conventions. I'm a CamelCase guy as all Mac programmers should be and tend to use 'i' as a prefix for instance variables (a Symbian OS convention). Also, I hate the common 'C' convention of defining struct templates and then having to type struct myTemplate myStructInstance; all the time when a proper type just makes things a whole lot cleaner IMO.

Talking about alignment, of course if callback e.g. in a driver has different alignment rules to the application code setting up the callback (e.g. if the driver turns into a fat 68K/PPC library, but the application is still 68K), then it's possible for the structure alignment to clash. In that case, one would have to provide a portable mechanism for ensuring identical alignment, e.g. target architecture routines for assigning and accessing extended fields; and possibly corresponding application routines for accessing custom fields from the driver. But based on this thread and @jmacz 's list of vintage Macs, we're talking 68K only here!

 

[nyef]

My own read on this scenario is that CurrentA5 is still valid in the interrupt context (see IM II-386, the Assembly-language note for SetUpA5). That said, I don't know how this would interact with MultiFinder or other multi-process environment.

 

[jmacz]

I've always done it and seen it done by extending the ParamBlockRec (except that one time I saw it done with self-modifying code, and, no, please no). Never got the globals thing to work - but also haven't tried very hard. I'm fairly sure that if you need access to your application globals, passing the required A5 into the routine is your problem not the system's, and I think the blessed way to do that (it's somewhere in IM:Files, I'm relatively sure) is to extend the ParamBlock.

Ok, well this helps clarify. I just thought it was odd that IM clearly describes the problem (invalid A5) and then suggests SetA5/SetCurrentA5 but then doesn't take the helpful step of providing guidance on how to pass the saved A5 into the completion routine. Most things I've worked on (including some other areas of Mac Toolbox) provide an optional parameter for this type of thing. Of course, once it's clear what the "proper" method to pass this value is, you don't need globals. Alright, param block extension it is. Thanks guys!

But if you're going to pass in an A5 value, you might as well just pass in the stuff you actually need and then your program will be better because you're not using globals and global state is bad for the soul.

Yup, totally!

Also, I hate the common 'C' convention of defining struct templates and then having to type struct myTemplate myStructInstance; all the time when a proper type just makes things a whole lot cleaner IMO.

Yeah, I am happy typedefs allow us to shorten that syntax. Which means the only place I need to use struct myTemplate myStructInstance is within the same struct definition, for example with a linked list next pointer or something.

My own read on this scenario is that CurrentA5 is still valid in the interrupt context (see IM II-386, the Assembly-language note for SetUpA5). That said, I don't know how this would interact with MultiFinder or other multi-process environment.

It's possible that might be pre multifinder - I thought I saw another Apple doc that says you need to make that call only while your app is the active one.

 

[David Cook]

I ran into this exact situation for Tiny Transfer. Macintosh Technical Note #180 (MultiFinder Miscellanea) covers this. The solution provided below has been tested in all Systems from 1 to 9.x, on 68K and PowerPC, in Finder and MultiFinder.

Apple recommends placing A5 before the ParamBlock structure. Also, somewhere else I read (late Inside Macintosh? Apple source code? CodeWarrior documentation?) that they recommend a standalone callback that sets A5 and then calls your 'normal' callback routine. This decouples this hack from the remainder of your code. Lastly, by setting A5 and not just including just the variables you need, this again decouples the hack from the remainder of your code.



Here is my code for CodeWarrior 11 for a serial port callback.

#if defined(powerc) || defined (__powerc) #pragma options align=mac68k #endif typedef struct { unsigned long myA5; ParamBlockRec paramBlock; } SerialAsyncParamBlock; #if defined(powerc) || defined(__powerc) #pragma options align=reset #endif ...... SerialAsyncParamBlock outputWriteAsyncParamBlock; outputWriteAsyncParamBlock.myA5 = SetCurrentA5(); outputWriteAsyncParamBlock.paramBlock.ioParam.ioCompletion = SerialWriteAsyncWriteCallbackAsm; ...... asm pascal void SerialWriteAsyncWriteCallbackAsm(void) { // Technical note #180 says we can't rely on SetCurrentA5 during completion routines. // But it says A0 will be the paramBlock. // We put the application A5 immediately before the paramBlock. move.l A5,-(sp) // Save old A5 movea.l -4(A0),A5 // Set my application globals jsr SerialWriteAsyncWriteCallbackGlobalsValid movea.l (sp)+, A5 // Restore old A5 preturn // Pascal return } void SerialWriteAsyncWriteCallbackGlobalsValid(void) { // Normal C routine here }

- David

 

[cheesestraws]

Most things I've worked on (including some other areas of Mac Toolbox) provide an optional parameter for this type of thing.

Yup, pretty sure this was a design error and was known to be such. Very annoying.

Apple recommends placing A5 before the ParamBlock structure.

Reading the text you've posted, that looks like it's for convenience from assembly? Is there another reason why before, rather than after?

Lastly, by setting A5 and not just including just the variables you need, this again decouples the hack from the remainder of your code.

This is a good point - setting A5 in the callback (or callback wrapper) means you're not having to keep track of special execution environments.

 

[David Cook]

Reading the text you've posted, that looks like it's for convenience from assembly? Is there another reason why before, rather than after?

Yes, this results in simple and short assembly that works regardless of the assembler/compiler. So, good for a technical note sample. But, I also assume it was so that the ParamBlock structure could be expanded in the future without affecting the offset to A5.

 

[David Cook]

The more I think about it, the more I believe Apple could have fixed this in System software when MultiFinder was introduced. If they had simply created a structure that held the caller's ParamBlock pointer (pointer, not structure) and the A5, they could have stored the A5 when the async call is made and swapped it at callback. Even if the caller restored A5 on their own (to support older System software), it would have done no harm.

 

[jmacz]

Reading the text you've posted, that looks like it's for convenience from assembly? Is there another reason why before, rather than after?

Thanks @David Cook. This looks like what I was referring to when I mentioned hacking it by setting aside some memory (the saved A5) and accessing it via an offset. But sounds like it's the official solution?

If I'm understanding the Apple suggestion correctly, the difference between the two approaches:

The Param Block Extension Discussed Earlier

• Create a new struct that has the Param Block first followed by the saved A5.
• Allows you to cast and use the new struct as a Param Block.
• Save the A5 in an instance of the new struct and pass the new struct instance to the PBxxxx async call.
• Cast the param block as your new struct inside the completion handler to access the saved A5 (and call SetA5 appropriately).

Apple's Suggestion

• Create a new struct that has the saved A5 first and the Param Block second.
• Save the A5 in an instance of the new struct.
• Pass just the actual Param Block (within the new struct instance) to the PBxxxx async call.
• Access your saved A5 using a 4 byte offset from the beginning of the param block.

They basically accomplish the same thing though.

This is really good to know, thanks guys!

 

[David Cook]

They basically accomplish the same thing though.

Yes. Use whichever you find more convenient.

 

[cheesestraws]

If I'm understanding the Apple suggestion correctly, the difference between the two approaches:

Yup. The Apple suggestion is more convenient from assembler, less convenient from C. That's about the only difference. I'm also relatively sure I saw the 'stick it on the end' recommended somewhere else in Apple's docs which was more C-focussed, but I can't remember where, so that may be a false memory.

 

[David Cook]

A couple of other things worth mentioning:
1. The stack (A6/A7) may be pointing to another application's stack. This is usually fine to borrow, as you are going to destack and return it to the original point. However, if you have increased the size of your application stack and intend on large stack allocations in your completion routine, you may cause corruption in some random application.

2. The memory manager may be pointing to another application's memory heap. Same warning as above.

3. Low level global variables (including the application event queue) can also be another application. So, no easy posting an event to your application to indicate an async job has completed.

4. If you change A5, you need to restore it before you return.

So, @cheesestraws recommendation of just appending fields you need rather than messing with A5 (and stack and heap etc) has strong merit.

I always wondered why Apple didn't add memory protection to avoid one application writing into another application's memory space. I guess this is an example of the work they would need to do to not break existing software.

 

[cheesestraws]

As a sideways note, if your software is running on A/UX, your interrupt handler may also find itself running in a different UNIX process from the one you registered it from, because lol.

 

[jmacz]

Jeez…

 

[Snial]

To some degree, programming for the Classic Mac Toolbox is a lot like programming an embedded system, especially when it comes to interrupts, drivers and callbacks. Lots of constraints, lots of creativity required. For that reason, many of Apple’s guidelines map fairly well to good practice on constrained, embedded systems too.

Thanks for the alternative prefixing with A5 approach @David Cook !

 

[David Cook]

FYI. Basilisk II stutters badly when calling the serial port async. I don't know if the same thing will happen for file access. If you are coding in an emulator and experience that problem, you can detect if an emulator is running and switch to sync calls in those environments.