How to set a minimum width for responsive web design and then let device zoom to fit below that point?

I'm working on a semi-responsive web design that adapts (responsive behaviour) to any screen width above 480px. However, it can't adapt to a screen width under 480px. Below that point I want the device to zoom out the page content (scale it down) so it fits the device width without need for horizontal scrolling.

I've read both Android and IOs documentation on the ViewPort meta tag and tryied many different combinations. Belows code will let the layout's responsive behaviour work when above 480px and then it will zoom out to fit screen width when below 480p, as expected.

<meta name="viewport" content="width=480">

However, if zoom is applyied by the user it will save that state for the current mobile browser window and horizontal scrolling will be shown and will not go away unless user manually zoom out. It won't reset to the correct zoom (zoom out enough to fit the screen width) even if user reloads the page. At least, thats the behaviour when tested on my iPhone. I don't know where to test it for different Android resolutions.

<meta name="viewport" content="width=480, initial-scale=1">

If I add initial-scale zoom will reset when page reloads, but since its set to 1 it won't scale down to fit the device width without horizontal scrolling. Instead, it will have 100% width (480px) and shown horizontal scrolling on screens under that resolution. I want to keep it responsive above 480px and let the mobile device scale it down below that so it fits the device width without horizontal scrolling. I would prefer a solution without having to use any JS library, if possible.