mardi 3 novembre 2020

How to scrap the website properly and getting all td texts from website

I am new to python. is anyone know {sum(int(td.text) for td in soup.select('td:last-child')[1:])} what is use of [1:] in this or [0] or [1]. i saw it in many scraping examples below for in loop. As i was practicing i build this code and don't able to scrap all data in csv file. thanks in advance, sorry for two question at one time.

import requests
from bs4 import BeautifulSoup
import csv

url= "https://iplt20.com/stats/2020/most-runs"

r= requests.get (url)

soup= BeautifulSoup (r.content, 'html5lib')

lst= []

table=soup.find ('div', attrs = {'class':'js-table'})



#for row in table.findAll ('div', attrs= {'class':'top-players__player-name'}):
#    score = {}
 #   score['Player'] = row.a.text.strip()
#    lst.append(score)

for row in table.findAll (class_='top-players__m top-players__padded '):
    score = {}
    score['Matches'] = int(row.td.text)
    lst.append(score)

filename= 'iplStat.csv'
with open (filename, 'w', newline='') as f:
    w= csv.DictWriter(f,['Player', 'Matches'])
    w.writeheader()
    for score in lst:
        w.writerow(score)



print (lst)



Aucun commentaire:

Enregistrer un commentaire