I'm having troubles authenticated my first scraper against the login page for my hockey pool (just looking to grab the stats once a night for dislaying in a nicer format).
//my variables
string userName = "My Pool";
string password = "hockey";
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "ctl00$MainContent$txtPoolName=" + userName + "&ctl00$MainContent$txtPassword=" + password;
byte[] postDataBytes = encoding.GetBytes(postData);
string url = "http://ift.tt/1LqS4fo";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.Headers.Add("Cache-Control: max-age=0");
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
request.Headers.Add("Accept-Encoding: gzip,deflate");
request.Headers.Add("Accept-Language: en-GB,en-US;q=0.8,en;q=0.6");
request.Headers.Add("Upgrade-Insecure-Requests: 1");
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36";
request.ContentLength = postDataBytes.Length;
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("Origin: http://ift.tt/1iG84xe");
request.Referer = "http://ift.tt/1LqS4fo";
request.Host = "www.pickuphockey.com";
request.AllowAutoRedirect = false;
request.CookieContainer = new CookieContainer();
using (var stream = request.GetRequestStream())
{
stream.Write(postDataBytes, 0, postDataBytes.Length);
stream.Close();
}
//cookie container to save for the next request
var cookieContainer = new CookieContainer();
//get the cookies from the login page
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
foreach (Cookie cookie in response.Cookies)
{
cookieContainer.Add(cookie);
}
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
var r = readStream.ReadToEnd();
}
I've been working with various examples throughout Stack Overflow and am using fiddler to analyse the web calls made; trying to match all the headers in the browser login with the headers that I'm adding to my code.
Right now when I code break and look at the variable r, I see the html for a page saying that I need to enter a value for the username/password. I assume this means that I'm not setting the parameters correctly in this call but nothing I read or tweak seems to get past this error. I've masked my actual user/password but I assume that even incorrect values (if someone tries the code) should result in a different error message saying something about a bad password. Any web call gurus out there that can spot what I'm doing wrong?
Once the code works I'll wrap it up into some nicer helper class, I know it's a tad messy at the moment.
Appreciate the time.
Aucun commentaire:
Enregistrer un commentaire