Canvas pixel manipulation optimization

I'm trying to do some real-time pixel manipulation (mode7-like transformations if you are interested) with canvas.

It works basicallyas following: It takes a source imageData that works like a framebuffer, it does some transformations and writes the result into a another 320x240 destination imageData.

Here's the code:

function transform() {

    var src_pointer, dest_pointer;
    var destData = dest_context.getImageData(0,0, 320, 240);
    var imgdata = destData.data;
    var sourceData= source_context.getImageData(0,0,320,240).data;


    for (var y = 0; y< 240; y++) for (var x = 0; x< 320; x++) {

        //*** DOING SOME TRANSFORMATIONS HERE, AND WRITE THE RESULT AT SX AND SY 
        //... Doesn't have impact in perfomance. Suspicious, huh?


        dest_pointer = getPointer(x,y);
        src_pointer = getPointer(sx, sy);

        imgdata[dest_pointer] = sourceData[src_pointer];
        imgdata[dest_pointer +1] = sourceData[src_pointer +1];
        imgdata[dest_pointer +2] = sourceData[src_pointer +2];
        // Alpha? Sad thing that canvas just handle 32bit image data. I don't really need it.
        imgdata[dest_pointer +3] = sourceData[src_pointer +3]; 


    }

    dest_context.putImageData(destData,0,0);
}

//Function to help map a coordinate into image data array
function getPointer(x,y) {
    return ( ( y * 320 ) + x ) * 4;
}

That works, yes but that have a poor perfomance if you excecute it continuously (about 12 frames per second). Doing some profiling I discard an specific method bottleneck (getImageData and putImageData have a really little loadtime).

I used to think that issue was in the transformation section but the profiling throws me that the bottleneck was specifically in the pixel assignment. Yes, maybe optimizing math operations can be possible (think that's hard, beacuse that's in the borderline between pure javascript and browser engine), but in array assignment is possible to optimize?

Any ideas? It makes sense? I think I'm doing something obvious really wrong, but I don't figure it.