mardi 18 septembre 2018

python web scraping beautiful soup and adding to a list

I am trying to learn to web scrape using Python and BeautifulSoup. My problem is when attempting to add "scraped" items to a new list, only the final entry in the relevant tags is displaying when I print the lists. How do I add each combination as a list item?

import requests

    standings = requests.get('http://games.espn.com/ffl/tools/finalstandings?leagueId=379978&seasonId=2012')


from bs4 import BeautifulSoup

soup = BeautifulSoup(standings.text, 'html.parser')

## Ask BeautifulSoup to find all of the records

pat = soup.find_all('tr', attrs={'class':'sortableRow evenRow'})
teams = []
for x in pat:
        name1 = x.find('a').text
        record1 = x.find('td', {'class':'sortableREC'}).text
        pf1 = x.find('td', {'class':'sortablePF'}).text
        pa1 = x.find('td', {'class':'sortablePA'}).text
        pfg1 = x.find('td', {'class':'sortablePFG'}).text
        pag1 = x.find('td', {'class':'sortablePAG'}).text
        diff1 = x.find('td', {'class':'sortableDIFF'}).text


 teams.append((name1, record1, pf1, pa1, pfg1, pag1, diff1))

odd =soup.find_all('tr', attrs={'class':'sortableRow oddRow'})

teams2 = []
for team in odd:
        name2 = team.find('a').text
        record2 = team.find('td', {'class':'sortableREC'}).text
        pf2 = team.find('td', {'class':'sortablePF'}).text
        pa2 = team.find('td', {'class':'sortablePA'}).text
        pfg2 = team.find('td', {'class':'sortablePFG'}).text
        pag2 = team.find('td', {'class':'sortablePAG'}).text
        diff2 = team.find('td', {'class':'sortableDIFF'}).text
teams2.append((name2, record2, pf2, pa2, pfg2, pag2, diff2))




Aucun commentaire:

Enregistrer un commentaire