Screaming Frog and other online checker returning 200 status code but while debugging from c# returning 500

When i scan the URL through screaming frog as well as other online checker it returns status code 200 (OK) but when i do the same from C# code for same URL , it always return 500 (Internal Server Error) without any detail error, but while browsing the URL is running smoonthly.

try
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    {
        if (response.StatusCode == HttpStatusCode.NotFound)
        {
            EventLogProvider.LogInformation("Test", "Not found", "Not found");
        }
    }
}
catch (WebException ex)
{
    var httpResponse = (HttpWebResponse)ex.Response;
}

I would like to test if the status code is 404 or not but everytime i gets 500