mardi 28 avril 2020

How do I scrape a randomly generated sentence from this website

Im using python 3.8x to try and scrape a randomly generated sentence from this website. https://randomwordgenerator.com/sentence.php Except when I read it, the generated sentence isn't in the HTML. Can any one help me find a way to scrape the generated sentence? I found the HTML tags when the sentence is generated but it doesn't generate when I request.

Here is my code.

random_sentence_webpage = 'https://randomwordgenerator.com/sentence.php'

# The HTML tag for the generated sentence
start_marker = '"support-sentence">'
end_marker = '</span>'

from urllib.request import urlopen, Request
headers = {'User-Agent': 'Chrome/81.0.4044.129'}
reg_url = random_sentence_webpage
req = Request(url=reg_url, headers=headers) 
html = urlopen(req).read()
html_text = html.decode('utf-8', 'backslashreplace')

starting_position = html_text.find(start_marker)
end_position = html_text.find(end_marker,starting_position)

random_generated_sentence = html_text[starting_position + len(start_marker):end_position]

print(random_generated_sentence)



Aucun commentaire:

Enregistrer un commentaire