OPulence: A Software Synth/Sequencer for a 128kB Mac

[Snial]

From time to time I design or start to code Synthesiser/ Sequencers. OPulence is one I'm currently toying with. I might not complete it (many of my projects don't complete, sorry), but here's an introduction to the design and a specific question about how to allocate memory for waveforms and sequences, for which I think the solution is: SetHandleSize.

Introduction

I've done a bit of feasibility analysis on what I think I can get out of a 128kB Mac's sound capabilities. It will have similar functionality to Studio Session, and I intend to use 2-operator FM sounds.

I think I can get 5 voices (giving me 6 to 7 note polyphony, a-ha!) + a 4 piece 'polyphonic' drum kit out of an original Mac. This is close to an OPL2 chip in percussive mode. Don't get your hopes up too much, a signal to noise ratio for its 8-bit DAC with 5 voices will be pretty poor (super-crunchy), but at least you'd be able to play more sophisticated music than a classic MOD player could.

Architecture

Each FM instrument uses up to 56x 128-byte waveforms, a set of 8 FM tones x 6 amplitudes (so we don't have to calculate volumes on the fly). Envelopes will appear to step crudely between tone and amplitude. We can have up to 5 voices, but a voice can by bi-tonal, namely the note and its 5th. This means practical pieces can contain a chord (using 2 voices) + bass, melody and counter-melody. A drum kit is an array of 32 waveforms indexed by a 5-bit value where each bit represents a percussive wave that can be on or off. Combinations of bits correspond to combinations of the waveforms. eg. if bit 0 is a tom and bit 2 is a hi-hat, the value 5 will trigger a waveform that combines both.

Sequences consist of patterns at the lowest level, so, first there's a set of patterns in memory. Then the 5 (or 6) tracks list the patterns to use in each track.

Interaction

It's an actual Mac app, but incredibly crude. I have a VBL function for playback, where some ideas are copied from the MOD player discussion, except, based on counting cycles assuming a net 6.5MHz CPU, I can get 5 voices + the drums out of it. I've written a playback routine, which would just be triggered by a menu option. Finally, I have an awful non-Macintosh function for converting text in an RPN-format into synthesiser sound/drum kit definitions and sequences.

The Problem

The problem is a classic Mac OS memory allocation issue. There's no VM, and only 80kB available. Sound definitions will actually only take 42x6x128b=27kB, so playback code and sequences must fit in 53kB. Sound definitions are built up as we go along, so the application won't know the memory requirements until the end. With modern computers that's not a problem, you'd just allocate memory as you see fit, but here it is a problem.

A classic Macintosh app's memory consists of:

{ MasterPointers[forHandles], HeapSpaceForCodeAndData[both pointer-based blocks and handles + code and resources], spare space, HandleBasedBlocksThatHaveBeenMovedHi, a gap, the stack, A5World, App global variables }

So, when we're compiling instruments and sequences, there will be a couple of code segments at the bottom and the data we're allocating (to handles in my case) will go above that. And that's a bit annoying, because memory might get a bit fragmented. Pre-allocating pointers won't work, because I might need to change what's allocated to instruments and sequences to make way for larger code segments, but moving segments high is likely to help me manage it better.

So, my initial thought was to allocate blocks in 256b..512b (which would work easily for wave forms); and link them as a simple linked list. Patterns are a bit more complex as I'd want blocks to contain an integral number of patterns as long as each one is >=256b. So, then I thought I'd take this approach:

NewHandle ;to allocate the longer block.​

BlockMove ;to copy the old one to the new one.​

DisposHandle ;to deallocate the old one.​

MoveHHi ;to move it back.​

So, the reason for the 256b unit is so that I can work out the maximum number of blocks I need and generate enough MoreMasters accordingly. But I think that method would be slow and the linked list would add a long handle-ref for each block which is a bit of a waste on a 128kB machine.

But then I thought I'd take another approach, which is what I want to run past experienced Mac programmers. And this would be to pre-allocate all the memory I think I need; fill it with all the data I want, and then call SetHandleSize to the actual block's size. So, my question is, when I call SetHandleSize to make the block shorter, will it (a) create another block to occupy the remainder or will (b) the memory manager just waste the remainder (so the physical size remains the same, but the logical one is smaller). I think that it's case (a) because if you increase the size of a handle, it will shuffle handle-based blocks around if possible to ensure the larger sized block can be properly resized. And I guess that my proposed use of SetHandleSize is exactly the kind of use-case they were trying to support.

But I could be wrong.

Sound - Macintosh Garden

Frequency Modulation Sounds DL #1 Sound v1.0

macintoshgarden.org

MIDIMAC Sequencer 2.5 - Macintosh Garden

MIDIMAC Sequencer v2.5 is an early MIDI sequencer for the Macintosh, and a direct predecessor to Opcode's Vision. DL #1 This download is reconstructed from the contents of the original diskette. It is provided as a bootable 400k DiskCopy 4.2 disk image, compressed using StuffIt 1.5.1, and...

macintoshgarden.org

 

[joevt]

I would guess that SetHandleSize won't move the handle if the size is decreasing or increasing, unless increasing the size doesn't fit.

Can you do a two-pass algorithm where the first pass just calculates the size?

int total_size = 0;
for (int pass=0; pass < 2; pass++) {
    if (pass == 1) // second pass
        allocate(total_size);
    int object_position = 0;
    for each object {
        if (pass == 0) // first pass
            total_size += object_size;
        else {
            use_allocation(object_position, object_size);
        }
        object_position += object_size;
    }
}

 

[Snial]

I would guess that SetHandleSize won't move the handle if the size is decreasing or increasing, unless increasing the size doesn't fit.

Can you do a two-pass algorithm where the first pass just calculates the size?

int total_size = 0;
for (int pass=0; pass < 2; pass++) {
    if (pass == 1) // second pass
        allocate(total_size);
    int object_position = 0;
    for each object {
        if (pass == 0) // first pass
            total_size += object_size;
        else {
            use_allocation(object_position, object_size);
        }
        object_position += object_size;
    }
}

Thanks for the reply! I thought about a two-pass approach at one point and it may be the best solution. It's OK if SetHandleSize doesn't move the handle if I reduce the size as long as the memory no longer used by it is now available to the application heap. i.e. this kind of code sequence is possible:

available=FreeMem();
waveForms=NewHandle(available);
MoveHHi(waveForms);
realSize=BuildWaveForms(waveForms, file);
SetHandleSize(waveForms, realSize);
MoveHHi(waveForms);
available=FreeMem();
patterns=NewHandle(available);
// etc...

Although handles are an unusual memory management approach forced by a lack of an MMU, I genuinely like them. Modern systems will have highly fragmented memory with zillions of holes that only works statistically, because running out of virtual address space is so unlikely. Handles are error-prone and can be awkward, but they can make better use of RAM in constrained environments if used judiciously.

-cheers from Julz

 

[Snial]

A bit of an update. I wrote a simple THINK C based app whose periodic DoNull function performed a repaint that allocated all the available free memory-1kB using NewHandle, then released all but 1kB of it using SetHandleSize . I linked all the allocated blocks as a linked list of handles ( so (**(Handle**)head) would give the next block in the list (rather than (*(Ptr*)head) would for a pointer). When the application ran out of space (or Master pointers) or FreeMem() returned <1kB or NewHandle() returned nil, I could then run through the entire list, calling DisposHandle() to free each block in turn.

If SetHandleSize() didn't create a new, free block in the heap when a Handle was made smaller, then I'd only be able to allocate a single block. In fact I was able to allocate hundreds. Therefore my technique for allocating blocks of variable sizes without knowing how much space is needed beforehand should work .

 

[adespoton]

I think I can get 5 voices (giving me 6 to 7 note polyphony, a-ha!) + a 4 piece 'polyphonic' drum kit out of an original Mac.

This is going WAY back for me, but I seem to recall squeezing 6 voices out, and then using the trick of using interference on those 6 voices to include a 5 piece 'polyphonic' drum kit (interference on the 6th voice was used for something else, but I can't remember what).

For the memory... a modern technique would be to store blocks on disk and then read them into memory in sequence without having to grow your heap. But that's not really going to work here, as a) disk I/O is way too slow, and b) the routine to manage this would likely eat up all your spare memory.

So I look forward to seeing your final solution!

 

[Snial]

This is going WAY back for me, but I seem to recall squeezing 6 voices out, and then using the trick of using interference on those 6 voices to include a 5 piece 'polyphonic' drum kit (interference on the 6th voice was used for something else, but I can't remember what).

Intriguing. I think I can only do 5 voices + kit. But as said earlier, an instrument (and therefore a voice) can be bi-tonal (where the second tone is a 5th above the first). Thus we can get 6 note polyphony when playing major, minor or suspended 4th chords; and 7 note polyphony from Major 7th (e.g. C+G, E+B) and Major 6th chords (e.g. C+G, A+E).

For the memory... a modern technique would be to store blocks on disk and then read them into memory in sequence without having to grow your heap. But that's not really going to work here, as a) disk I/O is way too slow, and b) the routine to manage this would likely eat up all your spare memory.

Indeed it would, but it's OK, all the waveforms and sequences will fit in RAM even on a 128kB Mac. Instruments are described in the opening post. Wave patterns are stored as a 7x6 matrix of 128b waveforms. The X-axis is for tone, and the Y-axis amplitude. Two ADSR envelopes then select a tone and amplitude on a given VBL tick, which selects a given wave. Thus, we basically only need 42x128=5,376b for an instrument, 5376x5=26,880b for all 5. A set of 6 sine waves pads out the waveforms by 768b and a drum kit occupies 32*128=4,096b. A 128kB Mac has about 75kB spare for an application, and my playback routine is currently about 250 lines (about 2500b), so if I assume it grows 2x, we get: 75*1024-26880-768-4096-5000=40,056B left for sequences.

Every note-on/off event will take about 3 bytes on average I think, so across a 5 track+percussion 5 minute piece we have 40056/3/6/5/60=7.418 events/s per track. That would be every track being full of semiquavers at 120bpm and no repeating patterns. There should be a reasonable amount of RAM for credible pieces.

Musical track events contain a note (0..60, 5 octave range), velocity (0..3) and length. Selecting the right waveform using ADSR is done using the same, simple, fractional add on each tick, so that will be quick. But the upshot is that I don't generate a full sample per instrument, like Studio Session or a MOD Tracker expects; it's more like a crude Wavetable synth. But I guess you knew that.

So I look forward to seeing your final solution!

Thanks!

 

[Snial]

Tone Dithering

As I was driving home on cruise control from my Dad's on Saturday I entered a pleasant accident-free reverie, which solved a major issue with OPulence.

Introduction

OPulence uses a 5.25kB matrix of waveforms per instrument. 8 waveforms with different modulation depths (the tone) and for each waveform of a given modulation depth, there are 6 carrier amplitude waveforms (the volume). So, basically, there are 8 modulation amplitudes x 6 carrier amplitudes in FM terms. As a note is played, the amplitudes of the modulation and carrier ADSR envelopes at any one 60.15Hz tick select a tone and volume waveform. We pre-calculate because it isn't possible for a humble 7.5 MHz 68000 to calculate waveforms in realtime.

But the problem is that for many envelopes, particularly ones with a slow attack or a slow decay/release, the steps between different tones will be easily noticeable. You'd need about 32 or 64 different tones to create a smooth transition, which is why wavetable synths from the early PPG designs had at least 64 waves to choose from. In a sense, OPulence is a crude wavetable synth modelled using FM. It's what I would jokingly call a wave-step synth.

Insight (Part 1)

I realised that in the same way we can create different pitches by indexing samples in a waveform by adding a fixed-point frequency to a fixed-point phase, we can do the same thing to select between different waveforms on successive ticks. To recap, a waveform has 128 samples and they are indexed as:

phase=(phase+frequency)&0x7ffff; // 7 whole + 16 fractional bits representation.
sample=wave[phase>>16]; // index the whole part

So the above calculation is performed 370 times per 60.15Hz tick. An envelope rate can be represented as a 3:16 value added to the envelope level:

envLevel+=envRate; // the real code would handle overflow too.
wave=waves[envLevel>>16]; // Select one of 8 waveforms.

This calculation is performed once per 60.15Hz tick, so the specific waveform is updated relatively infrequently, but still much faster than human perception. The trick is to treat the lower 16-bits of the envLevel as a 'frequency' which determines how often we select the following waveform instead. I call this wave dithering, though in fact it's a simple application of the Bresenham line-drawing algorithm, or Pulse-Density Modulation.

envLevel+=envRate; // the real code would handle overflow too.
envPhase+=(envLevel&0xffff); //
wave=waves[(envLevel>>16)+(envPhase>>16)]; // Select one of 8 waveforms.
envPhase&=0xffff; // clear the overflow.

We can consider the effect of this for different values of [COLOR=rgb(41, 105, 176)](envLevel&0xffff)[/COLOR]. If (envLevel&0xffff) is 0, then envPhase never becomes >=0x10000 so it's always just waves[(envLevel>>16)] , the standard wave. If (envLevel&0xffff) is 0x8000, then envPhase crosses 0x10000 50% of the time so it alternates between waves[(envLevel>>16)] and waves[(envLevel>>16)+1]. If (envLevel&0xffff) is 0xffff, then envPhase crosses 0x10000 99.998% of the time so it's nearly always waves[(envLevel>>16)+1] very occasionally just waves[(envLevel>>16)].

Intermediate values, generate different proportions of the two waves. Alternating waves every 60.15Hz produces a tonal tremolo effect at 30.075Hz, but this wouldn't really be perceived - instead our ears ought to perceive it as a single tone, the average of the two waves. It should also be the case for one (or 2) in 3 waves being an alternate waveform - a tonal tremolo effect at 20.05Hz and I think even at 1/4 (or 3/4) waves being the alternate one, a tonal tremolo effect at 15.0375Hz would be just under the perceptual threshold.

But this wouldn't be true indefinitely. A 1/8th ratio would be at 7.02Hz, which we would definitely hear as a bit of a warble. Thus wave dithering is practical down to about 1/4 waves.

Still, this would be as a whole a massive improvement in tonal control (an extra 2 bits, equivalent to 32 levels) and without costing virtually anything in terms of CPU.

Insight (Part 2)

It turns out we can do significantly better than the above! The main VBL loop generates samples as per the first code fragment in red, above. In practice, for speed, the loop is unrolled 37 times and executed 10 times per VBL, per pair of voices. Thus, we could support wave dithering within a single VBL, potentially alternating between two waves 10x per VBL, an effective wave dithering frequency of 300.8Hz! This certainly wouldn't be perceived.

Thus, on entry to a VBL each voice points to a pair of waves instead of 1.

Combining both would give us a single bit of tonal control within a VBL and 2 more bits at the tick level, a total of 3 bits. With the 3 bits of control we have from the static waves, that gives us a 6-bit resolution, 64 levels, as good as anything we could wish for.

The same can be done for carrier volume levels: dithering volume levels again, gives us a high (but still LFO) frequency tremolo effect, up to another 3 bits. This gives us 6*8=48 volume levels instead of 6.

Conclusion

OPulence is forced to have a tiny (5.25kB) matrix of waveforms per instrument so that it can fit in a 128kB Mac (which is a fun part of the challenge). This kind of wave-step synthesis would normally have a lot of annoying discrete artefacts, but would still be a worthwhile method for generating conventional synthesiser envelope dynamics. By adding some simple, low-CPU tricks we can increase the tone and amplitude control to close to the ideal.

Thanks for reading this!

 

[esselfortium]

I love this project, it's been exciting to read about your progress on it!

 

[ymk]

I think what you're describing as tone dithering is called time-division multiplexing. To pull it off, you need a fairly high sample rate to start with.

I don't understand what the wavetables are for. In a true FM synth, all you should need is a cosine table and maybe some amplitude lookups.

You can pick up some speed by aligning lookup tables and eliminating some AND operations.

This is one of those applications which begs for assembly.

It's a neat project for sure, great job!