dimanche 24 janvier 2016

Make a bot to navigate a site?

I want to make a script that will just go to my college course scheduling listing page, and scrape each course being offered with information like who is teaching it, the time, the course id, etc and store it into a database.

Problem is I don't know much about web scraping, but also I think this may be a little different given that you have to enter in forms and select drop down options to get to each page. So I'd have to select the term Spring 2016, then select the department Computer Science, selecting the course say Databases, and then selecting the section for example.

Is there anything out there that does something similar to this already? Or anyways anyone could point me into the right direction for what I'd need to know to do this.

Preferably in Python.

There are no captchas or anything like that I'd have to deal with.




Aucun commentaire:

Enregistrer un commentaire