When is if modified used over cache control max age?

In my apache server, I am configuring max age in the cache control header. But, some clients are still hitting the server and getting 304s that the resources is not modified. I'm just wondering when does a client decide to use this approach before max-age?

Thanks