Progress on our new Mac App Compatibility Environment

[Toni_]

Hi again,

Here's some replies to some earlier comments:

This is why I was thinking of the nanoKernel. Apple had a solution for running older cooperative code along side preemptive memory-protected code (I don't understand the details of how they did that but my understanding is that the nanoKernel was the key that made that happen.) Rather than using the host OS to do the program-to-program communication, reproduce the parts (nanoKernel?) that Apple made to do that and keep it inside the emulation environment.

Is that a reasonable idea?

I think the basic idea how Apple did it Mac OS 9 nanokernel, was abstracting the lower-level hardware through the nanokernel interface, running Mac OS 9 on top of it as a single task (and thus normal applications runnning on it would not directly benefit of the multitasking and memory protection features). I read that this abstraction was one of the key elements running the actual Mac OS 9 into Classic environment, where the low-level nanokernel was replaced with the Classic runtime, allowing most of the Mac OS 9 running on top of it to remain intact. There was some discussion here: https://groups.google.com/forum/#!msg/comp.sys.mac.programmer.help/tO0iuTNETGc/oTwfHPfuqXoJ

Yes, it is! I think there is a community-wide yearning for a smaller simpler system that's both performant on a range of hardware AND doesn't corral us in to Walled Gardens !! 6 months ago I was doing some research into SK8 (Hypercard on steroids) and I was BLOWN AWAY on how FAST emulating MacOS 7.6.1 on 68K was !!! It blew my mind! The ONLY reason I'm not primarily running classic MacOS today is that the emulators are not reliable.

In the old days the hardware limitations kind of forced people to create performance- and space-efficient code, which with current hardware is just blazingly fast and small. However, that comes with the downside, that a lot of the old System software (Toolbox APIs, etc), and many applications, bypass a lot of sanity checks, and tolerate a lot of even buggy behaviour without immediately crashing the system or having side effects. One example was "BattleCruiser" game which I was trying recently, which does DisposeHandle on a resource handle, which does not surface as bug until the resource handle in resource map is accessed next time (in that case, doing ExitToShell -> CloseResFile). Another example is actually from one of my very early shareware games, aWorm, which I noticed does not set a valid GrafPort using SetPort before doing CopyBits on the window. This by luck happens to work most of the time, but if that port is set to the screen port, it will crash in attempt to use the non-window GrafPort as WindowPeek. Good catch 21 years after after making that game...

This "dream" is certainly the core of my earlier post!

Perhaps diverging off topic but maybe "the dream" needs to be defined.

1) A reliable emulator for running existing software

(The M.A.C.E. project is VERY exciting! I'm actually considering using Hypercard for REAL projects again once it is up and running

2) Independent Mac OS 7/9 operating system. capable of running original software (FULL compatibility not required). NanoKernel included to provide a preemptive, memory protected space for "new" community created applications.

3) Poor man's Copland. Simplified versions of "some" of the Copland features. Document-Centric computing would be my #1 vote! OpenDoc compatibility NOT required.

Like I said, it's the "dream".

What do you guys think? Does it match your thoughts?

Something akin to this would be truly a dream OS. One of the motivations for writing the boot loader I mentioned in one post earlier. However, have to see how soon we will get there... now that Apple is, sadly, making it more difficult to publish independent software even on MacOS, there is growing motivation for having possibility to host the emulation on also other systems than Apple's own.

Cross application communication could be done easily using Apple Events.

Each application could run in its own Toolbox environment.

It seems like Elliot Nunn is disassembling the Classic ROM.

It is true that Apple Events can be easily abstracted. However, they are not the only means of sharing information between Applications. There's also the PPC Toolbox, Process Manager APIs, and as Mac has linear memory with no memory protection, any process could share information with other process by simply finding a way to pass valid Pointers/Handles to other application's zone, which there is no way to prevent on Mac OS. There may be also other cases, I haven't looked for example much into CFM shared library mechanism, but I'm sure those will come up eventually.

The MacBinary format does not have a CRC checksum.

Actually the header does have. But true, technically this is not a big challenge to recalculate it.

This explanation does not make sense. Do you not keep both data and resource fork in RAM? MacBinary is 3 parts: 128-byte header, followed by data fork, followed by resource fork. Surely writing these 3 parts in one combined operation is no worse than writing 2 separate files to disk. You're not reducing the size of data written to disk by using AppleDouble instead of MacBinary.

The file operations always operate on data on disk, as a 1:1 mapping to Mac OS Toolbox APIs. So when for example Mac application does _SetEOF call, we only do ftruncate call on the native side (which appropriate error checking, etc). I think the feature you are referring to might be disk caching, which caches file contents in RAM. All this however is delegated to the host system, because handling additional RAM cache in emulator would be unnecessary and complex work - and might even risk adding data corruption, for example if emulator crashes before data is flushed on to the disk. The aim is not to reduce data size, but to reduce complexity of operations and thus reliability.

---

In other news, here's some latest advancement in the development - we can now run PC emulator inside M.A.C.E., so basically the emulator can emulate emulators  

https://mace.software/2019/11/27/softpc-and-the-quest-for-the-xt-memory-map/

Sorry for the slow development speed, real-life work is slowing down us at the moment, but we're making progress every week. The goal is still to reach some sort of generic release of Phase 1 emulator (in addition to the bundled apps), but there will be a bunch of unfinished Phase 1 stuff still in it, which may limit application compatibility. I think the next target is to implement "Shared" filesystem module for M.A.C.E., which would allow cross-process - and also native HFS+ - file access. Just want to make everything perfect, so you guys won't be disappointed because of the certain bugs and certain unimplemented features that are still in progress.

 

[Toni_]

Hello,

There's another minor update on the progress. Pukka mentioned somebody on Emaculation forums was asking if we could run Apple's Finder - so we attempted it, and made some fixes so that the original, unmodified Finder 6.1.8 from System 6.0.7 will (mostly) work on M.A.C.E.:



This could be considered as another step towards the generic M.A.C.E. release. Of course, we cannot distribute Finder ourself, but it is nice to know that once we get to that point, people could use it if they want to  

Here's the full briefing on the recent development: https://mace.software/2019/12/12/finding-the-finder/

 

[Toni_]

Happy New Year (and belated Merry Christmas) from the M.A.C.E. team!

There were not much news at Christmas time, so I did not post a link to the Christmas update here earlier (at https://mace.software/2019/12/26/happy-holidays/ ) - mostly it consists of Christmas themed Dark Castle near-playthrough, and MacTCP teaser. However, if you are using Windows (or Raspberry PI) platform besides Mac OS, you might be interested to know, that we have made significant progress on the ports of M.A.C.E. on both of those platforms: https://mace.software/2020/01/01/happy-new-year/

To sum up, we expect to be able to provide Windows versions of the test applications as soon as during January, with only polishing and tweaking to be done. I will let you guys know more later once we get them ready for testing!

 

[Toni_]

And now the Windows version is ready for testing! For people who prefer to use that system   

https://mace.software/2020/01/07/windows-version-and-screen-mode-control/

We've package all the eight current test applications for Windows, and they are available in the downloads section at https://mace.software/files/ . It took some time figure out that WordPress (silently) does not allow linking EXE files (or ZIP files containing them), although it allows uploading them...weird. Anyway, for the time being, I put the Windows version ZIPs into my Dropbox and shared the downloads through it.

It should be noted, that like Mac bundles, the Windows EXE files are also unsigned, so (depending on security settings) the Windows may also complain about that a bit.

Theoretically everything *should* work, but we've only tested the new version on two PC computers (plus one virtual machine), and we haven't found any Windows-specific issues anymore. For now, the command key is mapped into ALT key on windows keyboard, and option key is mapped into left CTRL key.

Another new feature is the pixel double and full-screen mode toggling, with Cmd+Shift+1 and Cmd+Shift+2 respectively. This new feature has been updated also to the existing Mac OS X app bundles.

I hope you guys have fun with this, let me know if it works or if there are any issues   

 

[krum110487]

This is amazing, I played with the windows version and it worked really well, the only real issue is the crackling sound, is that an issue on both versions? Many of these games I didn't play as a kid, so I can't speak on accuracy as I was too poor for earlier Macs. 

When or if you get to fungus, power Pete, pathways into darkness, spelunk, gold digger, space station theta... I can verify that era

This brings back so many memories

Fantastic job! I can't wait to start seeing color games I check on your blog more times a week than I should haha.

Keep up the good work, 

 

[Toni_]

This is amazing, I played with the windows version and it worked really well, the only real issue is the crackling sound, is that an issue on both versions?

I know of the sound issue you mention, we looked briefly into it one year ago, and it may be problem with how SDL handles our (or classic Mac's) 22255 Hz sound frequency, which we use as the output format. It seems to be worse on higher CPU usage, but in most of our tests the "popping" has been minimal. It is however on the list of things to improve, and we will give it definitely a more profound look into when we start work on Sound Manager this year.

When or if you get to fungus, power Pete, pathways into darkness, spelunk, gold digger, space station theta... I can verify that era

Fantastic job! I can't wait to start seeing color games I check on your blog more times a week than I should haha.

Thank you  

Actually the color support is one of the next major features in our task list, and I posted just a moment ago a status update about it in the blog:

https://mace.software/2020/01/31/first-steps-into-a-more-colorful-emulation/



It's still far from being ready for test applications, but we have started the work on Color QuickDraw emulation a couple weeks ago, and we will be improving it (hopefully) a lot during the spring. I think we should have first color test application(s) before summer, unless there are any sudden major obstacles or changes in schedule.

This brings back so many memories

Keep up the good work, 

Thank you for the feedback It is highly appreciated.

 

[LaPorta]

Slightly O/T, but Pathways is available in the Mac AppStore as well. Very nice emuport.

 

[trag]

I didn't notice this thread almost a year ago when you were asking for game suggestions.  Very cool concept and execution, BTW.

If it's not too late, I would like to suggest the old game, "Dungeon of Doom".    I'm not sure about the providence at this point.

I bring it up, because when running it under miniVMac it has a problem with, IIRC, a keyboard function.   I want to say for dropping items, but it's been a long time ago.  When it didn't work right, I stopped trying to play it.   Anyway, I'd love to see an emulator that works better.

I can post/email a copy if you can't find it otherwise, if you are interested.

 

[Toni_]

I didn't notice this thread almost a year ago when you were asking for game suggestions.  Very cool concept and execution, BTW.

Thank you  It is always great to hear there are people who are interested in the project.

If it's not too late, I would like to suggest the old game, "Dungeon of Doom".    I'm not sure about the providence at this point.

Actually, Dungeon of Doom (or more accurately, version 5.4 demo of it) is one of the games we already tested during last year (it's on the status page at https://mace.software/status/ ). There were some problems getting it to work in the past - but I saw your message, and decided to give it a try - and after a couple tweaks and two bug fixes in Toolbox emulation a minute ago, the game seems to work now okay :-D  At least I managed to get to level 2, with save/load, the "demo version" sound effects, etc. working. Really fun game, used to play this also a lot when I was young. We should probably update the status page soon to reflect the fact it is working now, but I'll do it later when we get some more progress with the color quickdraw and other applications...

I bring it up, because when running it under miniVMac it has a problem with, IIRC, a keyboard function.   I want to say for dropping items, but it's been a long time ago.  When it didn't work right, I stopped trying to play it.   Anyway, I'd love to see an emulator that works better.

I can post/email a copy if you can't find it otherwise, if you are interested.

It would be interesting to know more about the keyboard issues, I would assume Minivmac would have most accurate implementation (at least on Mac), because it has such low-level hardware emulation - and all the modifier keys map directly to Mac keyboard (I know that on windows default minivmac has a bit oddly F1/F2 mapped to command/option). I did not find any problems at least on M.A.C.E. with quick testing though, but I'm not sure if I tried all the buttons :-D  

I'm not sure if it would be okay to bundle the demo version as a M.A.C.E. test application in the downloads - the about/other docs point out it is a free demo, but there is no mention about distribution permissions. To be safe, I would want to reach out to John Raymonds to check permission for this before adding it. However, when we get the generic M.A.C.E. environment (and possibly the MA.C.E. app bundle creator) soon working, this game and others should be testable by anybody.

 

[trag]

Wow, Toni_, thank you.  Talk about responsive.        I'll try to figure out exactly what problem I was having.   The trick is remembering to do it after I get home, where older Mac is.    The problem was pretty specific and darned annoying.

 

[Tom2112]

I just tried out two of your MACE executables for Windows. Both had a very high pitch whine over my speakers while playing, and a very loud pop when I ended the games. (Missile and Gunshy). I realize tracking down issues like this can be difficult. Perhaps you'd consider a key command to mute the sound? That would give a temporary fix.

As for games I'd like to see, a few of my favorites from my early college days working in the campus Mac lab:

PT 109 (https://macintoshgarden.org/games/pt-109)

Nettrek (https://macintoshgarden.org/games/nettrek there's a couple versions, I'd go for the early B&W one)

Dark Castle (https://macintoshgarden.org/games/dark-castle)

Shadowgate (https://macintoshgarden.org/games/shadowgate)

Thanks!

 

[trag]

I had time this weekend to experiment with my old miniVMac installation.   

I have the full version of Dungeon of Doom, but it says it is version 2.0 in the about screen.   I didn't know versions got up to 5.4.  

Anyway, the specific problem is that no keyboard inputs work at all.   Things like cmd-d for drop of cmd-g for get have zero effect.    I end up having to do everything with the mouse and the menus.

I never experimented much with minivmac so I am a totally unsophisticated user.   It's entirely possible I'm doing something wrong, or have a out of date version of minivmac or something.    At the time I did post to some forums asking about the issue I experienced, and no one ever responded.

Anyway, point being, it would be great if the keyboard commands work on your emulator.   Maybe that's not a major issue.  Maybe my problems are a user issue.

I'd be happy to send you my disk image of Dungeon of Doom.

 

[Toni_]

Hi again, and sorry for late reply. Been super busy recently!

I had time this weekend to experiment with my old miniVMac installation.   

I have the full version of Dungeon of Doom, but it says it is version 2.0 in the about screen.   I didn't know versions got up to 5.4.  

Anyway, the specific problem is that no keyboard inputs work at all.   Things like cmd-d for drop of cmd-g for get have zero effect.    I end up having to do everything with the mouse and the menus.

I never experimented much with minivmac so I am a totally unsophisticated user.   It's entirely possible I'm doing something wrong, or have a out of date version of minivmac or something.    At the time I did post to some forums asking about the issue I experienced, and no one ever responded.

Anyway, point being, it would be great if the keyboard commands work on your emulator.   Maybe that's not a major issue.  Maybe my problems are a user issue.

I'd be happy to send you my disk image of Dungeon of Doom.

What I understand (based on the "About" box contents), the author made Dungeon of Doom a commercial game with the 5.4 upgrade, it being previously a shareware game. It seems that Dungeon of Doom 5.4 is actually a demo, and "Dungeon Revealed" is the commercial version of this demo. So the 2.0 version is probably an older shareware release before they changed the purchasing/distribution model. If you have the disk image, you could consider uploading it to the websites which currently archive old mac games (I am not sure if their names are allowed to be mentioned here, so I'll just play safe and not name them). This way it might benefit other people who might also be interested in trying out that older shareware version.

The reason why I have been quite busy is that, besides of having a lot things to do at the daily "real" job, we have been busy improving the color support, which I wanted to have something to write about now that our project has officially reached the second anniversary:

https://mace.software/2020/02/23/2-year-anniversary-of-mace-and-some-color-progress/

I'm glad that we were able to squeeze already some color gameplay video into this update, but there is still a huge amount of work to be done. But I will keep you guys up-to-date as we get more progress done during the spring.

 

[trag]

Just wanted to say "Thank you" for addressing my issue/concern.   I'll try to get that disk up.  I need to check the Shareware agreement, and make sure that form of distribution is allowed.   And if the author is still around, maybe I can actually send in the shareware fee -- if I get to a point where I can play the game.

 

[Toni_]

Just wanted to say "Thank you" for addressing my issue/concern.   I'll try to get that disk up.  I need to check the Shareware agreement, and make sure that form of distribution is allowed.   And if the author is still around, maybe I can actually send in the shareware fee -- if I get to a point where I can play the game.

No problem, glad to be of help. If you manage to figure out a way to get contact with the author (he's got IMDB profile btw at https://www.imdb.com/name/nm4184021/ but his contact info is behind a paywall), let me know, and I could ask him for permission for making a MACE bundle of the game (either demo or full) too. In the recent past I've managed to contact a couple authors (most notable Missile and Continuum developers) and received positive responses, so perhaps also he might be sympathetic for the initiative.

 

[Toni_]

I hope everybody here is safe & healthy, now with the epidemic going on in the world...

As for the good news, we have made some minor progress with color support on our spare time in the past 2-3 weeks: https://mace.software/2020/03/18/more-color-support-palette-manager/

Basically, we are now starting to have Palette Manager support, and one by one we are getting more and more color drawing features functional. Here's a screenshot of the latest status:



I also made a short video clip of the demo running: https://www.youtube.com/watch?v=YdeiB91bNwo (I hope the youtube post-processing will improve the video quality, right now it is a bit messy being really low-res)

 

[Builder1]

I just found out the creator of the Einstein Newton emulator is gathering the troops to get Netwonscript running on as many platforms as possible.

I wanted to mention your project and ask if The Newton Toolkit will work in MACE yet?

Any chance it could be tested?

http://www.unna.org/view.php?/apple/development/NTK/macntk

J

 

[Builder1]

Also, when is the plan for Serial support. I don't remember hearing about that (I assume it's later).

j

 

[Toni_]

I just found out the creator of the Einstein Newton emulator is gathering the troops to get Netwonscript running on as many platforms as possible.

I wanted to mention your project and ask if The Newton Toolkit will work in MACE yet?

Any chance it could be tested?

http://www.unna.org/view.php?/apple/development/NTK/macntk

J

That looks interesting, I searching information about the project, and found youtube video of the NTK basilisk2 demo through newtontalk mailing list, and it looks like something fun to definitely try out at some point. I did quick snoop around the SDK, and I believe at least Edition Manager and Communications Toolbox are required for it, both which are not yet implemented in the emulator. They will be of course added someday, but as of now, they are still on the To-Do list, and thus the NTK will not - yet - run.

Also, when is the plan for Serial support. I don't remember hearing about that (I assume it's later).

j

The plan is to support serial ports both physically (for example, USB serial adapters), and as virtual devices (simple unix input/output streams). We don't have at this point much of that implemented, but the Device Manager DRVR driver code is ready to support serial devices when we implement them. I actually have two USB-to-serial adapters I have been playing around with, but for some reason I am having difficulty getting even native OSX serial applications to work properly with them (they're so outdated that actually I cannot get native OSX drivers for them updated). I really would like to get the native devices to work, so that we can implement the proper status & control codes and verify their functionality at the same time. The input/output stream support would be really trivial to add on top of that, because it could ignore the hardware-specific serial protocol configuration stuff (baud rates/stop/parity/etc bits, etc...). Also Pukka said he was looking into some serial port stuff a while back, but he has been busy recently, especially now with the epidemic causing rearrangements in the daily lives. But this is definitely something we have on the roadmap. We already have tested adding some MacTCP driver functions, mostly DNR/icmp echo, but that is another topic...

Ps. Here's latest thing we got recently running on the emulator, with (sadly) fitting message for these times...I will write later a blog post about it, there's still issues that need fixing before that (i.e. outline text rendering crashes 10% of time because of unfinished color QD code, color QD menus are still leaving artefacts around screen because of more unfinished color QD code, and the Palette Manager seems to be misplacing a couple entries which causes those color indices to not display properly in this particular game)

 

[Builder1]

Looks like Matthias likes your project!

I just happened to stumble over a note he recently made in one of his git repos (mosrun).

Looks like he's been doing some work on this lately

mosrun: Run m68k Classic MacOS binaries in an emulated MPW environment. Runs Apple NewtonOS related tools on modern macOS, Linux and MSWindows systems, but many other MPW tools will also run just fine.

https://github.com/MatthiasWM/mosrun

j