samedi 26 mars 2016

Retrieve web page content like a browser

After I learned some things about differents technologies, I wanted to make a small project using UWP+NoSQL. I wanted to do a small UWP app that grabs the horoscope and display it on my raspberry Pi every morning.

So I took a WebClient, and I do the following:

WebClient client = new WebClient();
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
string downloadString = client.DownloadString("http://ift.tt/1LHqoDg");

But it seems that it detect that this request isn't coming from a browser, since the interesting part is not in the content(and when I check with the browser, it is in the initial XML, according to fiddler).

I also tried with ScrapySharp but I got the same result. Any idea why?

(I've already done the UWP part, so I don't want to change the topic of my personal project just because it is detected as a "bot")




Aucun commentaire:

Enregistrer un commentaire