jeudi 9 juillet 2020

Web Scraping Python Beautiful Soup 4 'article'

I'm trying to do a web scrapper with python by using web driver and beautiful soup 4. I'm trying the code on decathlon.fr and the problem is that the class encompassing the product is a 'article' class. And when i start the program it is taking nothing because i need that the class to be a 'div' class or 'a' class.

from bs4 import BeautifulSoup
driver = webdriver.Chrome("chromedriver.exe")

products=[] #List to store name of the product
prices=[] #List to store price of the product
ratings=[] #List to store rating of the product

driver.get("https://www.decathlon.fr/search?Ntt=t+shirt+anti+uv+b%C3%A9b%C3%A9")

content = driver.page_source
soup = BeautifulSoup(content)
for a in soup.findAll('article', attrs={'class':'dkt-product.js-product-slider-init.product-printed'}):
    name=a.find('a', attrs={'class':'dkt-product__title__wrapper'})
    price=a.find('div', attrs={'class':'dkt-product__price'})
    #rating=a.find('div', attrs={'class':'hGSR34 _2beYZw'})
    rating=a.find('span', attrs={'itemprop':'name'})
    products.append(name)
    prices.append(price)
    ratings.append(rating)


print(products)
print(prices)
print(ratings)



Aucun commentaire:

Enregistrer un commentaire