Using PHP to force image download doesn't work correctly

I have an image link that is 100% fine. I can use it to link to the image, and I can use it to display the image. However whenever I try to use PHP's headers to force an image download, it doesn't work. Here's what I'm using:

            $file = $link;
            header('Content-Type: application/octet-stream');
            header('Content-Disposition: attachment; filename="'.basename($file).'"'); 
            header('Content-Length: ' . filesize($file));
            readfile($file);

The download starts and asks me where to save the image, as it should. However after the image is downloaded, it simply won't open. What am I doing wrong here?