Jquery Click Outside To Close a Filter

Ok I have a Side Filter, where a User can fill out some information and filter a Webpage.

I want this menu to appear when clicking a button and disappear when I click outside of it. This is what I currently have to make the Filter disappear once the user clicks outside of it:

        $(window).click(
                function(e) {
                    if ((!$(e.target).hasClass("filterOverlay"))
                            && (!$(e.target).hasClass("toggleFilterButton"))) {
                        $('.filterOverlay').hide();
                        $('.darkBackground').hide();
                    }
                });

This works. However only if the Filter is empty. Because the fun thing is, when I add an Input-Field and click it. Of course Jquery doesn't recognize it as being the Filter and closes it.

What's the best way to go with such a thing? All I find is the solution above, but as pointed out, this doesn't really fit.