I am trying to scrape this website. When I try to paginate over the pages
https://timesofindia.indiatimes.com/topic/Hiv/news/1 https://timesofindia.indiatimes.com/topic/Hiv/news/2 etc..
I find that I am re-directed back to the starting URL
Since I use Web Scraper the chrome extension, I am not able to test response by sending in cookies for the session. I am guessing it could be either that or some locally stored variable that gets through pages. If there is a work-around for the same using the extension, please let me know.
Aucun commentaire:
Enregistrer un commentaire