if I open a website with Selenium (headless or not), a blank page gets opened and the output is:
<html><head></head><body></body></html>
if I open the page manually in my browser, the website loads normal. I tried it with time.sleep(10), removed the random and I played around with different arguments like:
options.add_argument('--remote-debugging-port=9222')
options.add_argument("--no-sandbox")
...
even reinstalling chromedriver did not help.
No success so far.
Here is my code:
url = "https://www.arket.com/de_de/men/knitwear/product.alpaca-blend-jumper-grey.0937502001.html"
options = Options()
ua = UserAgent()
userAgent = ua.random
options.add_argument(f'user-agent={userAgent}')
#options.add_argument("headless")
driver = webdriver.Chrome('/path/to/chromedriver', options=options)
time.sleep(5)
driver.get(url)
time.sleep(5)
soup = BeautifulSoup(driver.page_source, 'html.parser')
time.sleep(0.5)
driver.quit()
soup
Does anyone know how to solve this issue?
It might be remarkable that the page sometimes loads correctly with selenium, but usually it doesnt.
yesterday the output of the code was different:
<html xmlns="http://www.w3.org/1999/xhtml"><head></head><body></body></html>
Aucun commentaire:
Enregistrer un commentaire