Woocommerce swatch variations: filter gallery on select

I am using Variation Swatches For WooCommerce plugin to add variation images to product colors.

It's working fine for now, but the problem is that I am also showing all the product's gallery beside it. When I select a specific color, the corresponding image is showing but the gallery is not filtered to the image(s) of this specific color only.

I want to show the user only the images of the color they choose.

Is there an easy way to filter the gallery by choice of variation or should I do it programmatically?