Android font-size brakes UI

On my web project I encountered a problem with android phones. If you just browse a page with default settings all is fine, but if in Settings > Display > Font > I change font-size to e.g. Huge then my UI elements become scaled.

I style my elements using em's mostly.

I tried fixes on topic of font boosting:

html * {max-height:9999999px}

or

text-size-adjust: none

However, nothing works.

Please help. :)