Im trying to scrape the url and would like to get all apartments on all the pages possible. In this example, there are only two pages and I want to:
- "Click on the next button" / Go to page 2
- Go to the last page if there is no next-button
In the tutorials I've seen such as this one there is a href-link to the next button on the page he is scraping. In my case the HTML-code for the page-list does not contain any href link but looks like this:
In other tutorials, they could find the href link to the next-button by finding looping over the links in the webpage. When I do this, I get only the main links for the site (despite loading a nested url) and not find any next button.
nav = soup.nav
for url in nav.find_all('a'):
print(url.get('href'))
Do you have any ideas on how to access the url of the next-button in this case?
Aucun commentaire:
Enregistrer un commentaire