number crunching with as3 and pixelbender

I'm currently working on a project where we want to create an audio synthesis tool similar to max/msp or pd in Flash with AS3. One of the problems, as always, is to try and stop the code from eating too much CPU power.

With the release of Flashplayer 10 and the CS4 bundle Adobe has introduced a new shading language called Pixelbender, previously codenamed Hydra. The primary use of this toolkit is to manipulate images at a high speed in tools like Photoshop, After Effects and Flash. Adobe Engineers state, that PB can also be used to compute cpu costly algorithms (number crunchers) on float arrays. Here's how to do it.

Pixelbender for Vectorprocessing

Getting a Pixelbender shader (or kernel as Adobe has named Pixelbender Shaders) to operate on such datatypes as the DataArray or the new Vector in AS3 is a bit tricky, but, as I have found while trying around, possible with a few limitations. At this point I cannot say whether using a Pixelbender Shader for number crunching actually is faster than iterating your Vector, you'll have to test for yourself. Any comments are very welcome.

To develop and export Pixelbender kernels for use with AS3, you'll have to download the Adobe Pixel Bender Toolkit. With the toolkit come a language specification and a developer guide for Pixelbender. There are certain limitations of the Pixelbender language that apply when developing for use with AS3. This is documented in the developer guide.

The Code

In this example we want to hand a Vector of the type Number and a Number to the kernel and expect the shader to multiply each value of the Vector with the Number. This is what our kernel looks like.

Your Kernel for this example

```
<languageVersion : 1.0; >
```
```
//kernel definition
```
```
kernel NewFilter
```
```
<   
```
```
    namespace : "huesforalice";
```
```
    vendor : "Dave";
```
```
    version : 1;
```
```
    description: "Gain Control";
```
```
>
```
```
{
```
```
    //input datatype must be image3
```
```
    input image3 src;
```
```
    //output datatype must be pixel3
```
```
    output pixel3 dst;
```

    //this is our gain parameter, each value will be multiplied with gain

```
    parameter float gain;
```
```
 
```

    //this is a mandatory function in pixelbender which does the computation

```
    void
```
```
    evaluatePixel()
```
```
    {
```
```
        //calculate output
```
```
 
```

        dst = sampleNearest ( src, outCoord() ) * gain;

```
    }
```
```
}
```

Now we have to try and make our flashprogram use this shader for the purpose we've built it. Don't forget to export your kernel for use with Flashplayer. This is what our code could look like:

Your AS Code for this example

```
public class KernelTest
```
```
{
```
```
    //embed your filter as octet-stream
```

    [Embed("../../../../../../pb/MulNode.pbj",

                mimeType = "application/octet-stream")]

```
    private var filter:Class;
```
```
 
```
```
    //constructor
```
```
    public function KernelText () 
```
```
    {
```

        //this is our inputvector, should be filled by you, the length has

```
        //to be divisible by 3 
```

        var inputVector:Vector.<Number>;

```
 
```
```
        //our gain Value
```
```
        var gain:Number = 0.4;
```
```
 
```
```
        //create a shader object
```

        var sh:Shader = new Shader ( new filter() as ByteArray);

```
 
```

        //create a shaderjob which will actually run the shader on a vector

        var sj:ShaderJob = new ShaderJob ( sh );

```
 
```

        //tell the shaderJob to write the output back into the inputvector

        //you could use any other vector. this vector may not have a fixed

```
        //size for some reason
```
```
        sj.target = inputVector;
```
```
 
```

        //tell the shaderjob the dimensions of your output data

        //this is a tricky bit. height should be 1, because a vector

        //is and one dimensional array. width has to be the length of the

```
        //input vector divided by 3
```

        sj.width = int (inputVector.length / 3);

```
        sj.height = 1;
```
```
 
```

        //now for the input values, first gain

```
        sh.data.gain.value = gain;
```
```
 
```

        //and the input vector, which is the same as the output vector

```
	sh.data.src.input = inputVector;
```

	sh.data.src.width = int (inputVector.length / 3);

```
	sh.data.src.height = 1;
```
```
 
```

	//now start the calculation, the shaderjob should be executed now

```
	sj.start(true);
```
```
    }
```
```
}
```

Limitations

As you may have noticed, the input and output vector have to be divisible by 3. The background is, that the Pixelbender operates pixelwise, and each pixel has 3 values for red, green and blue, so the Vector is sliced up into pixels. Every evaluation of the evaluatePixel() method in the kernel processes 3 Numbers of your Vector. Theoretically you could use pixels of different dimensions, but Pixelbender doesn't allow 1 or 2 dimensional pixels for output, and using 4 dimensional pixels (which would be more adequate for an audio Vector consisting of 2048 samples) the Pixelbender produces errors.

To work around this problem you'll have to push missing values into a vector if it is not divisible by 3 and just disregard the extra values in the result.

There is also a limit on the size of a Vector which can be passed into Pixelbender. I haven't quite figured out what it is, but I have managed to pass Vectors which apparently where to big.

End

If you have any addiontal experiences or performance tests using Pixelbender this way, please post me a comment.

References
Flexible Factory
Tinic on audiomixing

30.11.2008 | back to top | write comment | 4 comments

comments

mulletchuck

Sat 03 January 2009 @ 05:35

Hey, I have a few questions on using PixelBender. Is it possible to have counters? like a global variable that is incremented for every pixel processed?

dave

Sat 03 January 2009 @ 13:42

Hi mulletchuck,
as you can see in my example, I use a function called OutCoord(). You can use this to extract the current outputcoordinate you are processing. From this you could calculate a value which increments for each pixel processed.
Dave

dub

Fri 09 July 2010 @ 16:03

Your code gives me an error saying that gain is expecting an array not a number

Dave

Sat 10 July 2010 @ 12:02

hey dub, it's been a while since I've written this. It worked then, but I'm not sure what the state is on the current non-beta flashplayers.