Without requiring credential, I tested out how to use urllib2 to grab files from the web server. But I am facing an authenticated website which even requires you to click an 'agree' button to their terms and condition after you provide the login credential. I have searched around in Stackoverflow and found some relevant posts, but seems none of them mentioned this type of authentication.
Further, the url for login is something like 'http://ift.tt/2q437vZ', but the zip file download page after you provide the credential and also click the 'agree' button is different than the login page, something like "http://ift.tt/2oHm3f6". This makes me confusing which url shall I use in the code.
I tested using the following code (from Download a file from https with authentication):
import urllib2
username = 'user1'
password = '123456'
baseurl = 'http://ift.tt/2oHl2Ux'
dataurl = 'http://ift.tt/2q48n2z'
#Create a password manager
manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
manager.add_password(None, baseurl, username, password)
#Create an authentication handler using the password manager
auth = urllib2.HTTPBasicAuthHandler(manager)
#Create an opener that will replace the default urlopen method on further calls
opener = urllib2.build_opener(auth)
urllib2.install_opener(opener)
#Here you should access the full url you wanted to open
response = urllib2.urlopen(dataurl)
after running the code, it did not give me error, but I did not get the file neither.
Any comments/suggestions? thanks a lot!
Aucun commentaire:
Enregistrer un commentaire