Web Audio API gaindrops

I've run into a problem using the Web Audio API.

Gain drop example

When I connect a gain-node to the chain and change the gain on it, say -6.0dB, it will immediately decrease the gain much more than 6.0dB and then slowly bring the volume back up. Similar to having a slow-release compressor somewhere in the chain. I've experienced the same thing with the BiquadFilterNode and its more noticeable for bass-heavy content.

I'm using the following code to setup the gain-node:

var gainNode = context.createGain();
source.connect(gainNode);
gainNode.connect(context.destination);

and this to change the gain:

gainNode.gain.value = -6.0;

It's not just a dB or two of extra drop, at times the audio is completely silent on a 6dB drop and fades back in over 2-3 seconds.

I don't see this problem on other sites using Web Audio API so I'm assuming I'm missing something obvious here but I can't figure it out. And yes, I'm new to this API.

Any ideas would be very helpful! It there an auto-gain or something by default that I need to disable?

Cheers! / Mattias