Pixel Bender Synthesizer

This experiment is CPU intensive BY DESIGN! If the audio breaks up, un-check effects or close other tabs. It was also originally designed to run in an AIR app, I'm positing it here for convenience. Source code and some explanation is below. I originally wrote in the summer of 2009 as an idea of something I wanted to show at MAX, but never did. Follow my personal blog for more info.

The delay-based effects are written in Actionscript, but all the other audio manipulation code is written in Pixel Bender.

Feel free to use any code that you'd like, but please provide attribution back to me and a link to kevingoldsmith.com.

Some General Notes

Generating Audio

This is the function that handles all audio generation and processing:

private function processSound(event:SampleDataEvent):void
{
    synth.SetVolumes( sineVolumeL.value, 
                      sineVolumeR.value,
                      squareVolumeL.value,
                      squareVolumeR.value );
    synth.SetDetunes( sinePitchOffsetL.value,
                      sinePitchOffsetR.value,
                      squarePitchOffsetL.value,
                      squarePitchOffsetR.value );
    
    var synthBuffer:ByteArray = synth.GenerateBytes( BUFFER_SIZE );
    var waveshaperBuffer:ByteArray = null;
    var leslieBuffer:ByteArray = null;
    var pitchFunBuffer:ByteArray = null;
    var delayBuffer:ByteArray = null;
    var reverbBuffer:ByteArray = null;
    
    if ( waveshaperEffectCB.selected )
    {
        waveshaper.effectAmount = waveShaperAmountSlider.value;
        waveshaper.effectMix = waveShaperMixSlider.value;
        waveshaperBuffer = waveshaper.GenerateBytes( synthBuffer, BUFFER_SIZE );
    }
    else
    {
        waveshaperBuffer = synthBuffer;
    }
    
    if ( pitchFunEffectCB.selected )
    {
        pitchFun.frequency = pitchFunFrequencySlider.value;
        pitchFun.SetDetunes( pitchFunFrequencyOffsetL.value, pitchFunFrequencyOffsetR.value );
        pitchFun.effectMix = pitchFunMixSlider.value;
        pitchFunBuffer = pitchFun.GenerateBytes( waveshaperBuffer, BUFFER_SIZE );
    }
    else
    {
        pitchFunBuffer = waveshaperBuffer;
    }
    
    if ( leslieEffectCB.selected )
    {
        leslie.frequency = leslieFrequencySlider.value;
        leslie.SetDetunes( leslieFrequencyOffsetL.value, leslieFrequencyOffsetR.value );
        leslie.effectMix = leslieMixSlider.value;
        leslieBuffer = leslie.GenerateBytes( pitchFunBuffer, BUFFER_SIZE );
    }
    else
    {
        leslieBuffer = pitchFunBuffer;
    }
    
    if ( delayEffectCB.selected )
    {
        delay.mix = delayMixSlider.value;
        delay.delayLength = delayLengthSlider.value;
        delayBuffer = delay.GenerateBytes( leslieBuffer, BUFFER_SIZE );
    }
    else
    {
        delayBuffer = leslieBuffer;
    }
    
    if ( reverbEffectCB.selected )
    {
        reverb.SetDelayVolumes( delayLine1VolumeSlider.value,
                                delayLine2VolumeSlider.value,
                                delayLine3VolumeSlider.value,
                                delayLine4VolumeSlider.value,
                                delayLine5VolumeSlider.value,
                                delayLine6VolumeSlider.value,
                                delayLine7VolumeSlider.value,
                                delayLine8VolumeSlider.value );
        reverb.SetMixerVolumes( delayLine14VolumeSlider.value,
                                delayLine58VolumeSlider.value,
                                sourceVolumeSlider.value );
        event.data.writeBytes( reverb.GenerateBytes( delayBuffer, BUFFER_SIZE ) );
    }
    else
    {
        event.data.writeBytes( delayBuffer );
    }
}
                

ProcessAudio is called by registering a callback as such:

 
output = new Sound();
output.addEventListener(SampleDataEvent.SAMPLE_DATA, processSound);
                

Pixel Bender Kernels and processing audio

I do a lot of tricks where I using image4 as inputs so that I can process multiple samples simultaneously, this makes the logic a lot harder, but it allows me to compute 2 stereo samples at a time which improves performance a lot(theoretically), so sorry if that makes the code harder to read. In general, you'll see in the kernels that I assume a Pixel4 value is: Sample1-Left Channel, Sample1-Right Channel, Sample2-Left Channel, Sample2-Right Channel. At some point in the future I'll check if the extra math incurred by doing my processing this way outweighs the benefits of the SSE code gen for when I actually process all four values at once.

Another thing that you'll notice is that I use square buffers instead of linear buffers. This also complicated the math (especially when dealing with sin/cos), but Pixel Bender in Flash uses rows to allocate blocks for multi-threading. So while a single Array of bytes or a single-row buffer would have made a lot easier, the performance would have suffered greatly. This is why I pass in the size of the buffer to any of the filters that use the sin operation.

Code for a single synth voice

A single voice combines a stereo square wave and a stereo sin wave which can be tuned separately by mixing them (although it would simple to change this to have the sin wave or square wave modulate the other). The Actionscript file is SynthVoice.as, the Pixel Bender kernels it uses are sinegenerator2.pbk, squaregenerator2.pbk and mixer2channel.pbk.

Code for 4 voice synth

A monophonic synth is nice, but a 4 voice synth is 4 times better. This lets you play simple chords and a lead line. The UI of my synth isn't multi-touch aware, so I had to add some other ui to let you add in other notes. The FourVoiceSynth class does a simple round robin as you add voices once the 4 slots are filled up. It also has simple setters for individual volumes of all the voices. It wouldn't be hard to allow you to do some more complex mixing of the voices, but I wanted to keep this part simple. It also would be pretty easy to expand the number of voices if you wanted. Depending on the number of active voices, I switch in and out different mixers to save processing power (rather than always using the 4 channel mixers and passing in empty buffers/muting channels).

The Actionscript file for the class is FourVoiceSynth.as, it also references the SynthVoice class and the Pixel Bender Kernels it uses are mixer2channel.pbk, mixer3channel.pbk and mixer4channel.pbk.

Code for the Leslie effect

A leslie is a spinning speaker that was used with organs, it produces a cool doppler effect [wikipedia entry on leslie speakers]. I did a simple version ('natch), a true Leslie would take into account the frequency shifts of an actual spinning speaker. Instead I just use a sine wave to modulate the audio signal, which is good enough for what I was trying to do.

The Actionscript file for the Leslie effect class is Leslie.as, and the Pixel Bender Kernel it uses is leslie.pbk.

Code for the Waveshaper effect

A waveshaper effect is sort of a simple digital distorition which works by simply altering the incoming signal [wikipedia entry on waveshaper filters]. My version is based on the one written by Andre Michelle and Joa Ebert in their popforge project.

The actionscript file for the Waveshaper effect class is WaveShaper.as and the Pixel Bender Kernel is users is waveshaper.pbk.