jeudi 3 janvier 2019

Pagination fails using Web Scraper.io

I am trying to scrape this website. When I try to paginate over the pages

https://timesofindia.indiatimes.com/topic/Hiv/news/1 https://timesofindia.indiatimes.com/topic/Hiv/news/2 etc..

I find that I am re-directed back to the starting URL

https://timesofindia.indiatimes.com/topic/Hiv/news/

Since I use Web Scraper the chrome extension, I am not able to test response by sending in cookies for the session. I am guessing it could be either that or some locally stored variable that gets through pages. If there is a work-around for the same using the extension, please let me know.




Aucun commentaire:

Enregistrer un commentaire