jeudi 1 décembre 2016

python scraping school's webpage which requires user login

I'm using python to scrape my school's webpage, but in order to do that I needed to simulate a user login first. here is my code:

import requests, lxml.html
s = requests.session()

url = "https://my.emich.edu"

login = s.get(url)

login_html = lxml.html.fromstring(login.text)
hidden_inputs = login_html.xpath(r'//form//input[@type="hidden"]')
form = {x.attrib["name"]:x.attrib["value"] for x in hidden_inputs}
form["username"] = "myusernamge"
form["password"] = "mypassword"

response = s.post("http://ift.tt/2fMuzdg",form)

f = open("result.html","w")
f.write(response.text)

print response.text

i am expecting that response.text will give me my own student account page instead of that it gives me a log in requirement page. Can any one help me with this issue? BTW this is not a homework




Aucun commentaire:

Enregistrer un commentaire