samedi 12 août 2017

BeautifulSoup can't find video or certain div tags

I don't know why it doesn't let me access the video tag. I'm trying to scrap the video source but it doesn't let me access the 'video' tag at all.

<video class="jw-video jw-reset" disableremoteplayback="" webkit-
     playsinline="" playsinline="" jw-loaded="data" 
     src="randomsrc2" jw-played="" style="object-fit: 
     fill;"></video>
    #web scraping stuff
    #web scraping stuff
    import bs4 as bs
    import urllib.request

    url = 'http://ift.tt/2fAaQy5'
    user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; 
    rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'

    headers={'User-Agent':user_agent,}

    q = urllib.request.Request(url, headers=headers)
    sauce = urllib.request.urlopen(q).read()
    soup = bs.BeautifulSoup(sauce,'lxml')
    print(soup)

    f=open('testd2.txt','w+')
    kuk = str(soup)
    f.write(kuk) #When I search for 'video' in the file it doesn't give me anything
    video = soup.find('video')
    print(video) #gives None




Aucun commentaire:

Enregistrer un commentaire