dimanche 18 juillet 2021

Python - BeautifulSoup wont scrape the rest of the li in the ul

I'm looking to scrape information from an exam website that stores dates and exam files in English. Link provided here

For those who dont wish to inspect themselves, this is the relevant html:

<ul class="sub-terms">
<li class="single-sub-term">
<div class="single-sub-term-heading">
<h3>מועד א׳ - 26/05/2021</h3>
<p class="single-sub-term-description"></p></div>
    <ul class="pdf-list">
    <li class="pdf-item">
    <div class="pdf-info">
        <p><span class="first-line">16481</span></p>

Problem: There are 3 li's in the ul 'sub-terms' - "single-sub-term" - where the exam name and date are stored. I need to access all 3 li's, but my code currently outputs on the first li and stops.

Code:

from bs4 import BeautifulSoup
import requests

html_text = requests.get('https://www.geva.co.il/solution_term/english_2021_summer/').text
soup = BeautifulSoup(html_text, 'lxml')


entirePage = soup.find('ul', class_ = 'sub-terms')
lists = entirePage.find_all('li', class_ ='single-sub-term')
for list in lists:
    date = list.find('h3').text
    pdf_info = list.find('span', class_='first-line').text

    print(f'''Welcome! 
    Current date - {date}        Term Number : {pdf_info} ''')
    print('')

Output:

    Welcome! 
    Current date - מועד א׳ - 26/05/2021        Term Number : 16481 

Welcome! 
    Current date -  - 27/05/2021        Term Number : 16382 

Welcome! 
    Current date - מועד ב' - 05/07/2021        Term Number : 16381 - A 



Aucun commentaire:

Enregistrer un commentaire