Open system call failing on System 6 but working on System 7

[atommclain]

Hello, I am working on porting an old open source project that was originally made to run on System 7 and up, to run on System 6. This is my first foray into classic Mac development, and at best I would consider myself a novice C programmer, but I am slowly making headway. I'm stuck on an issue that I've spent countless hours trying to figure out and would appreciate any insights on how to resolve it.

Where I need help is with the open system call. When the binary is run on System 6, open fails with error number -50, when the same binary is run on System 7 on the same machine open succeeds.

void testSave() { char *path = "Macintosh HD:testFile"; int flags = O_RDWR|O_CREAT|O_EXCL; int fd; fd = open(path, flags); if (fd < 0) { // Failing here on System 6 with CodeWarrior Pro 2 & 5 // Works on System 7 } }

I'm testing with Mini vMac which is compiled with Abnormal Situation Reports enabled. The project was/is based on CodeWarrior and seems somewhat reliant on it. I've upgraded Universal Headers to MPW GM version, which itself has a version of open, but I can't seem to get it to prioritize over the CodeWarrior standard C version. I also explored seeing if I could use fopen, but it looks like the return value from open is used and open itself is littered throughout the project.

During my attempts at debugging the issue I created a little scratch file to experiment with different C compilers. Open seems to work fine with Think C 5 & 6, MPW, but not with CodeWarrior Pro 2, CodeWarrior Pro 5. CodeWarrior 4 doesn't seem to have open just __open which crashes on System 6, but works on System 7. I tried to test CodeWarrior Pro 6, but couldn't get a scratch application to work. Again, the same binary will work on System 7, just not 6.

I'm hoping that this is an issue where I am not importing the correct headers or libraries or there some project setting I need to change. Any help or suggestions would be GREATLY appreciated!

 

[cheesestraws]

open() isn't a Mac System call: it's presumably being implemented by your clib in terms of the File Manager calls actually provided by the OS. That's why you're seeing different results in different compilers: it's not so much the compiler as the C library.

 

[Crutch]

I would suggest looking at how your compiler (CW) implements open(). There is probably source included. Inside somewhere it’s certainly calling FSOpen(), the high-level Mac OS routine that opens a file, or its low-level equivalent PBOpen(). See if it’s handing FSOpen() something weird in your use case. (Though a quick check in THINK Reference reports that FSOpen is allegedly not supposed to return paramErr == -50, so something else may be amiss.)

 

[atommclain]

open() isn't a Mac System call: it's presumably being implemented by your clib in terms of the File Manager calls actually provided by the OS.

I would suggest looking at how your compiler (CW) implements open(). There is probably source included. Inside somewhere it’s certainly calling FSOpen()

Thank so much for these responses! I'm both ecstatic and equally embarrassed to say that I had not realized the source for the c library was included with the complier... so much time wasted. But I'm able to pursue this new path, thanks!

 

[atommclain]

Here is the solution I ended up with. CodeWarrior Pro N, does seem to have a broken implementation of open for System 6 but luckily we do have access to the source code which is located in Code Warrior:MSL:MSL_C:MSL_MacOS:Src:fcntl.mac.c.

In the open function there are two blocks of code where a check is done for System 7, in the else condition of those blocks a call is made to PBOpenDFSync(), which is a System 7.0 call; this needs to be changed to PBOpenSync()after which it should work.

 

[cheesestraws]

PBOpenDFSync(), which is a System 7.0 call; this needs to be changed to PBOpenSync()after which it should work

That'd work.

PBOpenDFSync fixes a long-standing wart (in my opinion; the fact they fixed it suggests they also thought this) where the Open family of functions decided whether to open a file or a device based on the name, which means that Open can't open files with names that begin with full stops (.), whereas the OpenDF family can.

 

[Crutch]

I never knew that. Then again, I guess I never knew how _Open distinguished between files and drivers. I guess I must’ve thought it was … magic?

 

[twelvetone12]

That'd work.

PBOpenDFSync fixes a long-standing wart (in my opinion; the fact they fixed it suggests they also thought this) where the Open family of functions decided whether to open a file or a device based on the name, which means that Open can't open files with names that begin with full stops (.), whereas the OpenDF family can.

I was reading about this just yesterday in Daniel Allen's book and thinking it was bonkers. Good to know I'm not the only one (and it was fixed)!

 

[Crutch]

Which book is this now?

 

[twelvetone12]

"On Macintosh Programming Advanced Techniques", 1990

 

[Crutch]

OH! I have that book. Need to look this up, cheers.

 

[cheesestraws]

Yeah. There are bits of the earlyish Mac where you look at it and go 'oh, that's nice and sane', and there are bits where you ... don't.

_Open is definitely under the "don't".

 

[atommclain]

That'd work.

PBOpenDFSync fixes a long-standing wart (in my opinion; the fact they fixed it suggests they also thought this) where the Open family of functions decided whether to open a file or a device based on the name, which means that Open can't open files with names that begin with full stops (.), whereas the OpenDF family can.

Oh, that might explain why my virtual disk image stoped working after implementing this change; the program was writing temporary files starting with a period...

 

[cheesestraws]

Oh, that might explain why my virtual disk image stoped working after implementing this change; the program was writing temporary files starting with a period...

Yurp.

I think this is why A/UX's mac environment makes UNIX hidden files start with a bullet (•) instead of a full stop...

 

[atommclain]

"On Macintosh Programming Advanced Techniques", 1990

I currently reading through this myself, the chapter 4 FKeys to allow window and app switching are a god send, my only remaining hot key wish is to have a key command to move the current selection in Finder to the trash.

 

[cheesestraws]

I haven't read this book and it sounds like I should.

 

[Crutch]

It’s okay. It’s pretty MPW-tool-writing heavy which I suspect you (like me) will not find uber-fascinating. However the bibliography is maybe the best I’ve ever seen in a book. It inspired multiple additional purchases on many topics having nothing to do with Mac programming. The guy is clearly a polymath!

The discussion of the topic referenced above is really brief FWIW, on p. 75 of the PDF https://vintageapple.org/macprogramming/pdf/On_Macintosh_Programming_Advanced_Techniques_1990.pdf

 

[cheesestraws]

Wow, you're right, that's an amazing bibliography. I think my book shopping list just expanded...

 

[atommclain]

Curious to hear what books piqued everyones interest from the bibliography.

The one I found that I'm slowly going through is Operating System Design: The Xinu Approach (Macintosh Edition) which walks the reader through creating an operating system on the Macintosh. My hope is that after I finish up my current project, I'll switch over to implementing Xinu with a more modern compiler since the version in the book is tied the Aztec C compiler which seems to be lost to time.

http://512K Mac Blog has a bitmore info.

 

[Crutch]

Yes! @atommclain I am reading that exact book right now and had a similar long term plan.

That Xinu book is terrific. I was really excited to find a book on OS design based on the 68K architecture. Really cool.

I think Aztec C is actually available here though I have never tried it and think it might be entirely missing documentation (it’s a command line interface apparently). https://macintoshgarden.org/apps/aztec-c-aztec-c68k

But pretty much everything in the book (aside from some use of Aztec specific names for various memory locations, which one could reconstruct) looks like boilerplate K&R C so I think getting Xinu to run on another compiler shouldn’t be too hard.