samedi 5 décembre 2020

getting data from pages of a site

Im trying to get a site data with requests and then i wanna go to next page and again get the data. the simplest way is that i put page numbers at the end of site url but the problme is the url doesnt reload for next page and it doesnt have page number in url i can click on the next page button with selenium but i dont know how to get the data cause of driver in selenium doesnt have .text or any other functions as i know what can i do?

this is the part of my code to trying access to the site data:

from selenium import webdriver
import requests
import re
from bs4 import BeautifulSoup as bfs
import time
driver = webdriver.Chrome(executable_path='/Users/payasystem1/w/samane_tadarokat/chromedriver')

# URL of website
url = "https://etend.setadiran.ir/etend/index.action"


r = driver.get(url);
time.sleep(5) # Let the user actually see something!
search_box = driver.find_element_by_id('next_tendersGrid_pager')
search_box.click()
print (str(r))
time.sleep(5) 
driver.quit()

as you see i can access to the site and next page but i dont know how to get the shown table data in my program if you know pls help me!




Aucun commentaire:

Enregistrer un commentaire