I'm trying to make a crawler for the Best Buy website, however when I try to run the DownloadString function I get a 403 forbidden error. I've tried adding a header but the results remain the same. I've also run Fiddler and see what header is being sent by my browser and used the header there. What could possibly be causing the issue?
WebClient client = new WebClient();
client.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36");
client.DownloadString("http://www.bestbuy.com");
Aucun commentaire:
Enregistrer un commentaire