Flexbox gallery layout without stretching images of different sizes

I've been trying to achieve something like this gallery style layout using flexbox: Example of desired outcome

I want images (regardless of size) to sit next to each other in a two column layout until mobile layout where it's only 1 image for each line. When the two images sit next to each other I want them to scale to the size of the largest one of the two without stretching or distorting either picture.

My current attempt can be found here: Codepen

You can see that I've tried two methods, each have resulted in different problems.

I'll stick to the stretched image issue as that's currently closest to my desired outcome.

    <div class="wrapper">
  <header>
    MY HEADER
  </header>
  <section>
    <a href="" class "unitie">
      <img src="http://ift.tt/2lHBR4r" />
    </a>
    <a href="" class "meow">
      <img src="http://ift.tt/2lMWPej" />
</section>
  <footer>
    2016
  </footer>
</div>

CSS:

.wrapper {
  display: flex;
  flex-direction: column;
}

header {
  display: flex;
}

section {
  display: flex;
  flex-flow: row wrap;
}

a {
  width: 48%;
  margin-left: 10px;
}

img {
  max-width: 100%;
  height: 100%;
}

I found this idea online: Using aspect ratio for flex property But I have no idea how to find the aspect ratio of any image and convert it into the flex grow property value like he did.

Any help would be much appreciated.