jeudi 27 août 2015

Need to scrape information from a webpage with a "show more" button, any recommendations?

Currently developing a "crawler" for educational reasons,

Everything is working fine, i can extract url's & information & save it in a json file, everything is all fine and dandy... EXCEPT

the page has a "load more" button that i NEED to interact with in order for the crawler to continue looking for more urls.

This is where i could use you amazing guys & girls!

Any recommendations on how to do this?

I would like to interact with the "load more" button and re-send the HTML information to my crawler.

Would really, appreciate any amount of help from you guys!




Aucun commentaire:

Enregistrer un commentaire