HTML Audio element with mediastream as source manipulate stereo

I'm using webRTC(voip) to feed a Mediastream to an audio element on my webpage.

The users of the voip on the website should be able to have some sort of 3D Audio, they can place other users around them and maybe have one user on their right and another one on their left.

Every User you hear is one audio element on the page, in order to simulate 3d audio I have to control how loud one audioelement is on each side of the headphones. Is there a way to achieve this, control an audioelements volume on each side? I already looked in the Web Audio API and its panning functions but I couldn't get it to work with the Mediastream and the audio element.

//stream is the MediaStream
audio.src = URL.createObjectURL(stream);

audio.play();

Hope someone can help me, best regards.