mardi 7 janvier 2020

Why did I get an empty list with response.css("li.ais-pagination--item ais-pagination--item__next a").xpath("@href") using scrapy shell?

The website is "https://www.jbhifi.com.au/collections/laptops". I'm trying to crawl the href for the "next page".

Area I'm trying to crawl

But why scrapy shell returns an empty list? I'm using the statement:

"response.css("li.ais-pagination--item ais-pagination--item__next a").xpath("@href")".

Please show me how to scrap this using scrapy. I suspect this is because the class starts with "ais" (but don't know why it causes the problem). This happened to me in the past. Any solutions? Cheers!




Aucun commentaire:

Enregistrer un commentaire