lundi 4 janvier 2021

Python: Web Scraping Weird Output

from bs4 import BeautifulSoup
from urllib.request import urlopen as uReq
import requests

url = 'https://en.wikisource.org/wiki/Main_Page'
r = requests.get(url)

Soup = BeautifulSoup(r.text, "html5lib")
List = Soup.find("div",class_="enws-mainpage-widget-content", id="enws-mainpage-newtexts-content").find_all('a')
ebooks=[]
i=0
for ebook in List:
    x=ebook.get('title')
    for ch in x:
        if(ch==":"):
            x=""
    if x!="":
        ebooks.append(x)
        i=i+1
        

inputnumber=0
while inputnumber<len(ebooks):
    print(inputnumber+1, " - ", ebooks[inputnumber])
    inputnumber=inputnumber+1
input=int(input("Please select a book: "))
selectedbook = Soup.find("a", title=ebooks[input-1])
print(selectedbook['title'])
url1 = "https://en.wikisource.org/"+selectedbook['href']
print(url1)
r1 = requests.get(url1)
Soup1 = BeautifulSoup(r1.text, "html5lib")
List1 = Soup.find("div", class_="prp-pages-output")
print(List1)

This is my code. I want to get the paragraghs in the html code at the last part. But as output I get:

1  -  The Center of the Web
2  -  Bobby Bumps Starts a Lodge
3  -  May (Mácha)
4  -  Animal Life and the World of Nature/1903/06/Notes and Comments
5  -  The Czechoslovak Review/Volume 2/No Compromise
6  -  She's All the World to Me
7  -  Their One Love
Please select a book: 4
Animal Life and the World of Nature/1903/06/Notes and Comments
https://en.wikisource.org//wiki/Animal_Life_and_the_World_of_Nature/1903/06/Notes_and_Comments
None

Why is the List1 returns as one? It shouldn't. Can someone tell me where I am doing wrong.




Aucun commentaire:

Enregistrer un commentaire