dimanche 29 décembre 2019

Scrape dynamic Website where content is loaded as you scroll. Python

I dont know the right term but I think "dynamic website" might do the trick.
By that I mean that as I scroll, resources are being loaded. I searched for solutions and I came across webdrivers, personally I dont like a whole browser being loaded, just for the purpose of scrolling down.
A different approach would be looking at the network tab and scraping the url, that I find there.

https://www.immowelt.de/liste/hamburg/wohnungen/mieten?prima=700&sort=relevanz&cp=1

However the content really gets loaded as I scroll.

the link that gets shown when I open the network tab and scroll down: https://www.immowelt.de/liste/getlistitems

Im new to web development so I dont get how these links that I see in the network tab can be named exactly identical but hold different values.




Aucun commentaire:

Enregistrer un commentaire