Why does the Same Origin Policy prevent reading GET responses?

I've done a bit of research on the web and searched through a few questions about SOP and what kinds of abuse it mitigates, but most answers are focused on preventing stolen credentials. This makes sense to me.

What doesn't make sense to me is why browsers following SOP rules block the response outright, rather than blocking cookie and local storage access.

In other words, if cookies and local storage didn't exist, would there still be a need to prevent reading GET responses? Presumably this is already what happens to some degree with <img>, <script>, and <iframe>.