Jump to content

Tech Demo: MacFlim gets sound for May the 4th


Recommended Posts

To be honest, I didn't plan to spend that much time on it, but so many people requested it and @Crutch cornered me into implementing it by gaving me the crucial info I needed. Thank you so much!

 

I rewrote the whole thing maybe 5 times, but at the end, it was *totally* worth it. It is much better than the original MacFlim on almost every level (sound, better quality, consistent framerate, smaller files, etc.).

 

This is only a tech demo, it may or may not be integrated in the main app (because it is completely different and shares zero code. even the encoder is completely different. the file format is completely different. the playback logic is different too and simple app features like seek would be a major challenge). I rushed a bit to get something out for May the 4th, 'cause I could spend months continuing to improve it :-)

 

Neither binary nor the source code are yet released -- format still in flux and quite a few rough edges and stuff to finish. I'll put a binary version and a few files on the macflim.com website later (probably over the week-end at latest) so folks can test (and delivering via youtube is a huge pain due to the copyright AI).

 

Also, if you have a favorite clip, just let me know, because I spent several weeks with "Sweet Dreams" on a loop (for the MacPlus playback), and "Gangnam Style" (for the SE/30), and I could use some variations there...
 

 

Link to post
Share on other sites

@fstark Absolutely fantastic - now with sound, we have the killer "show off" app so show all our friends!  I'd love the "Bad Apple" demo with sound, but I'm sure I can encode my own eventually.  

Link to post
Share on other sites
5 hours ago, Byrd said:

@fstark Absolutely fantastic - now with sound, we have the killer "show off" app so show all our friends!  I'd love the "Bad Apple" demo with sound, but I'm sure I can encode my own eventually.  

 

 

Weirdly, Bad Apple is both a best and a worst case: it is a best case as there is generally very little change from frames to frames. However, it curently triggers two edge cases:

 

First, by having 100% white or black parts, *any* missed pixel is visible. As of now, the algorithm concentrate on parts where there are the most differences, and won't care about a single wrong pixel somewhere, unless eveything else is perfect. This leads to occasional visible artifacts in bad apple. I know how to fix that, but it will end-up as yet-another-subtle-parameter to the encoder, as concentrating on lost pixels for some video will render others worse. I have the same issue when encoding scenes from 2001, as some of the spaceships may leave pixels behind...

 

Second, there are a few moments in the video where the image completely reverts itself. This is an absolute worst case, as every pixel have changed and the whole image have to be updated. Furthermore, the large black and white areas makes it very visible even if the artifact is only during a single frame. Also, I know how to fix this, but it needs work.

 

At the end, I can currently have a somewhat correct Bad Apple on a SE/30. It should be perfect later. I doubt it will look too good on a Plus.

 

 

 

Link to post
Share on other sites
6 hours ago, risickwinters said:

@fstark I gotta second @Byrd, this looks really awesome! Looking forward to seeing some more and interested in the process :) wondering how I maybe able to do this myself with videos that I may have.

 

I unfortunately cannot upload most sample videos to youtube due to the copyright police.

 

I'll publish the encoder at some point, so don't worry, you'll be able to have your familiy holiday movie on your Mac :-)

 

I also may try to do a video on the process. My editing skills are not to good, so it may take some time to come up with something watchable...

 

Link to post
Share on other sites
1 hour ago, slomacuser said:

@fstark is that original NeXT jumper or you made it? :)

 

It is an original jumper I got in the early 90's, when I was a NeXT developer. I should probably make new one to keep this one as pristine as possible...

Link to post
Share on other sites
11 hours ago, PowerMac_G4 said:

@fstark You are an absolute genius. I don't have the words to praise this work highly enough. I can't wait for the video encoding tools to go public!

 

Do you have a Patreon/donation link where we can support your hard work?

 

 

That's really kind of you, and your words of support are more than enough motivation.

 

The only thing I really struggle with is getting my hands on hardware. In particular, I'd love to get some network card for the SE/30 ( to try streaming :-) ), but even after a couple of years looking at ebay, I can't get one (the few I saw didn't ship to Europe, and I missed them). Or if you know someone that would like to sell a NeXT Cube Turbo...

Link to post
Share on other sites

@Crutch, @risickwinters ,  I just released a beta test http://www.macflim.com/techdemo/00-README.txt (note to future readers : this link will only stay up for a few weeks, those video will find a better stable home)

 

There is the binary of the player and a set of sample flims for plus/se and se30. There are also silent mp4s so you can check the theorical result before downloading files.

 

I will release the encoding tools later, there is still quite some work.

 

@Byrd : there is a somewhat ok version of Bad Apple for the se30. Not perfect yet, but okay.

Link to post
Share on other sites

Words simply cannot describe how stunned I am that this is now a thing. Literal movies on a classic Mac, and now with sound?! 8-o

Keep up the good work Fred! This is definitely the Mac's new killer app!

 

I took the liberty to do a bit of testing. Nothing proper and since most of the collection is in storage, I gave it a go with mini vMac. 

Playback of Plus files on an emulated Plus works great. No video glitches to report. But there seems to be something wrong with the audio. Maybe this has to do with the emulation but I figured I'd better let you know. 

Basically there are a few pops and scratches. It doesn't seem to depend on the CPU speed (tried with 1x, 8x and all out) or on vMac's autoslow feature. 

 

WARNING: Audio might be a bit too loud! Adjust your device's volume before the demo starts!

https://www.youtube.com/watch?v=ktY3TUO_92E

 

With that said, the app's till awesome and I can't wait for the final product!

Edited by BadGoldEagle
Link to post
Share on other sites
Posted (edited)

Thanks for you enthusiasm!

 

The cracks are due to emulation. Here is the adrian video playing on real hardware (Mac128K with memory and scsi upgrade) :

 

https://youtu.be/CVmNsYHbwIo

 

For comparison, the SE/30 version:

 

https://youtu.be/MlvKIY-kRIc

 

I lost some time trying to get things running smoothly on minvmac before I discovered it was an emulation problem. It makes a lot of testing harder, but if it was simple, it would not be fun.

 

However, I have crashes on real SE/30, as you can see in the second video. The first time it happend was on a Star Wars video, where it felt like an hommage to the General Ackbar...

 

On the final product side, I will release that tech demo + encoding tools standalone, so people can show off a bit. The integration into the MacFlim GUI is a *major* rework (but is mandatory for completion...).

Edited by fstark
typo
Link to post
Share on other sites

Great to hear it's just the emulation acting up. FYI the Adrian SE/30 file also crashes with the emulated Plus, but it just freezes (no unimplemented trap error). Tested on 6.0.8 and on 7.5.3. 

It always happens here in my case:

output-onlinepngtools.png.ca67d36167ed7741143ee0ec5397d11a.png

Both Star Wars SE/30 files work fine on my emulated Plus.

 

By the way, how does one quit out of the app once you reach the console window? There's no mouse cursor and the Mac doesn't respond to Cmd-Q or Cmd-W... Is it meant to work that way or is vMac misbehaving again?

Edited by BadGoldEagle
Link to post
Share on other sites

It crashes on the emulation? That's good intel, it looks like it is a real bug in my code, and not some weird timing issue. Excellent news.

 

By the way, how does one quit out of the app once you reach the console window

 

Click the mouse. You can also abort playback by clicking the mouse (you may need to leave it pressed a couple of seconds, I check the mouse when I don't have anything else to do, + once every 300kb of flim, so if you are playing on a real MacPlus, it can take time...)

 

Link to post
Share on other sites
1 hour ago, fstark said:

Click the mouse. You can also abort playback by clicking the mouse (you may need to leave it pressed a couple of seconds, I check the mouse when I don't have anything else to do, + once every 300kb of flim, so if you are playing on a real MacPlus, it can take time...)

 

Thanks! There seems to be a problem though. The mouse appears to freeze (ie it does not move or register a click) after a few seconds of playback. It can be reproduced but the delay depends on the video. 

  • Adrian (Plus): It happens midway through his intro (roughly 33-35s after the start, ie as the "Adrian's Digital Basement" text disappears). Sometimes you can get out (at the 30-33sec mark) but then the mouse freezes a few seconds afterwards at the finder. Your app is greyed out as if it was still running and the desktop has borders. The clock is still running so the CPU isn't frozen. When it doesn't freeze (ie if I click before the 30s mark), I still get the greyed out app and the borders but nothing shows up in the multifinder (apart from the finder).
    Webp.net-resizeimage.png.8ab292f18bb60dca7f911ad243400a8e.png
  • Death Star (Plus): ~25 seconds after the start.

 

If you're lucky, it might be the same problem as the one you're experiencing with your SE/30. 

Edited by BadGoldEagle
Link to post
Share on other sites

I am not sure I fully understand you: as there is no mouse cursor during playback, you have no way of seeing if the mouse moves or not. The potential delay in the click is "normal", but I don't think you should see it on an emulator (as it is linked to read performance). If the mouse is jerky when you get back to the finder, it means you should reboot asap :-)

 

Anyway, I believe I found the race condition: I send the last chunk of data to be played by the sound driver during the sound callback, but that data (I suspect the sound headers) can gets immediately overwritten by the disk reading code before it gets a chance to reach the driver. My assumption of the timing of the PBWrite callbacks seems wrong.

 

After that, everything is possible, and the stability of the machine is compromised. I've seen the compiler stop working (ie: hapilly compiles, but doesn't change the binary, how fun...), or the sound stop playing until I restarted the emulator. You may lose the mouse, or get plenty of weird behaviors.

 

Still no idea on how to fix that, but we'll get through that !

Link to post
Share on other sites

@fstark This is working superbly well. The sound now works perfectly with vMac, but most importantly, there are no more crashes!

Great work!

 

Here's what I tested in more details:

- [vMac Plus 1x Speed, 6.0.8/7.1/7.5.3] Race conditions are fixed. No more crashes (even with SE/30 flims at 1x speed).

- [vMac Plus 2x Speed, 6.0.8/7.1/7.5.3] This should emulate a Mac Portable as far as speed/cpu goes. Most SE/30 flims work at 2x with some occasional slowdowns*.

- [vMac Plus Max Speed, 6.0.8/7.1/7.5.3] Everything works. 

- [vMac Plus Max Speed, System 2.0] Glitchy audio. Not worth investigating IMO (it's most certainly an HFS issue from the emulator and the files are larger than 400k anyway, but interesting).

- [vMac Plus Max Speed, System 3.2] Works good.

- [vMac II, 640x480 B&W, 1x Speed] Higher resolutions are indeed working. Occasional slowdowns with SE/30 flims*. The others are fine though.

- [vMac II, 1080p B&W, 1x Speed] I didn't know this was a thing but it works!

- [Basilisk II 040/030 8.1/7.5] Crashes instantly. Emulator issue. I can't see why it wouldn't work on SE/30s with 040 accelerators but you never know... @Bolle?

 

(*) Based on the above, it might be worth creating an extra class of machine between SE and SE/30. It would dramatically enhance the performance on machines like the Portable, PB100, LC and II (fast 68000s and 020s).

 

Apart from that, you just need an icon and perhaps still tweak the way you quit the app. On 7.1 and above it's still active (greyed out yet not available in Multifinder). Not happening on 6.0.8 and lower. But it's not a big deal.

 

Thanks again for creating this masterpiece!

Edited by BadGoldEagle
Link to post
Share on other sites

Thx for the report! Yeah, I tracked down the crash properly, I am pretty confident that this is solid, and behave properly under stress.

 

Note I don't think working in vMac always translates to working on real hardware, I have a feeling that the disk access have different impacts. But i'm checking on +, se and se30, and it seems ok.

 

That issue when quitting the app is weird, I'll see if I can reproduce and fix it.

 

The demo flims are on macplus/se and se30, because those are the machines I have (my Portable lacks a battery), and the original goal was to support only 512x342 machines (with maybe an April fool's version for Macintosh XL). However, the code knows nothing about this, the only differences are encoding parameters, and people will be free to tweak them as they like, to create better/worse combinations depending on the processor power and the disk subsystem speed.

 

Like I added the support for playback on screens greater than 512x342, I may add the support for *flim* of higher resolution (ie: having a 640x480 flim that only plays on higher end machines, even if that's not the orginal design goal).

 

Right now, I am focusing on low-end machines, I really want squeeze some extra quality out of the macplus version, even if it means cheating a bit...

Link to post
Share on other sites

I have a spontaneous question: do you know what the effective bitrate of the flims is? Is it fixed or dynamic? 

 

I'm asking purely based on my fascination with what you have accomplished. I thought it was literally impossible to make a compact Mac play video in this way, yet here we are. 

Link to post
Share on other sites
Posted (edited)

@PowerMac_G4 I want to make a video at some point describing the details.  There is nothing rocket science, just tiny bits of insight, stubborness and many many full rewrites.

 

 

Simplifying, the core thing is that it takes CPU time to read from disk and CPU time to write to screen. This time is roughly proportionate to the amount of bytes in the flim file. A higher end mac (or a mac with a better drive, or a ram disk), will be able to move more bytes via this reading + drawing process than another). Also, the quantum of time is 1/60th of a second (a tick), I cannot "borrow" time, so at the end, the idea is to design an encoder that is able to "do its best" with a given budget in bytes for the quantum (it should be a cpu-cycles budget, but they are roughly equivalent). Ie: "I give you $byterate bytes per tick, do your best".

 

I am not sure how to interpret the concept of "fixed bitrates" and "dynamic bitrates" in your question. If you look at that from a mp3-like encoding perspective, the macflim bitrate is fixed, it does not change depending on the part of the flim. This is because of that "non-borrow" clause: if I do better in an early part of the file, it doesn't free me cycles for the later part. This is not a problem mp3 players have: they do variable bitrate just to save space (space saved in the "silent" part of the file can be allocated to be used later). I can't do that, as I am not saving space but CPU. (I could support some sort of dynamic byterate later, for the case where you are ok to get a slightly worse video but care a lot about file size). However, the encoder don't always use all the allocated space, if the image is perfectly good, it cut short, so the effective byterates can actually vary.

 

So, they key to my design is an encoding that can a) manage arbitrary loss and b) be displayed fast. If I can't do a), I have no way to be sure a flim will play on a device, and b) is important, because of the original cpu usage equation.

 

A lot of calculations (in reality just trial and errors on real hardware), told me that my current algorithm supports 1600 bytes per tick on a macplus, 2500 on a mac se, and 6000 on a se30.

 

 

Improving on that (ie most of the work now) consist of:

 

* adding options to the encoder so it can be more efficient in throwing away data (examples: quantization of the source image, bluring, anything that can make successive images more similar, skipping frames), options to generate smaller files (example: reducing the image by adding black borders around)

* making the encoding smaller for common changes (example: compress vertically, make sure changes are with 255 items from each other so I can encode offsets in bytes)

* making the encoding perform faster (example: work on 16 bits words for the plus and 32 bits words for the se30)

* whatever weird idea I can have

 

(of course, some of those things may have an impact on that magical 1600/2500/6000 byterate, so it is a very iterative process)

 

I thought it was literally impossible to make a compact Mac play video in this way, yet here we are. 

I didn't it was possible on the plus either :-)

 

"Point n'est besoin d'espérer pour entreprendre, ni de réussir pour persévérer (One need not hope in order to undertake, nor succeed in order to persevere)." - Guillaume Ier d'Orange-Nassau

 

That said, it is the enthousiasm of the community that made me to do it. Didn't expect such interest and good feedback.

 

 

Edited by fstark
typo and clarification
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...