mercredi 17 octobre 2018

Indeed how to extract the job title href link?

I have the code to extract the job information from Indeed, but now I want to extract the link form the job title so I can open a new page and pull out the job description information.

I can see the link on the html page with the reference to the job posting, within the href tag but not sue how to extract it?

import requests  
import time
from random import randint
from bs4 import BeautifulSoup
import urllib, requests, re, pandas as pd

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import TimeoutException

webdriver.DesiredCapabilities.CHROME["unexpectedAlertBehaviour"] = "accept"

webdriver.Chrome(chrome_options=options,executable_path=CHROMEDRIVER_PATH)
options = Options()
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options,executable_path='chromedriver')

driver.get("https://www.indeed.co.uk/automotive-engineer-jobs-in-uk")

soup=BeautifulSoup(driver.page_source, "lxml")

title = [tag.text.strip() for tag in soup.select('.jobtitle')]
company = [tag.text.strip() for tag in soup.select('.company')]
location = [tag.text.strip() for tag in soup.select('.location')]

for y in range (len(title)):
    tmpstring = (title[y] + ',' + company[y] + ',' + location[y] + ",0")
    tmpstring = tmpstring.encode("utf-8")
    f = open('FileDump','a')
    f.write(tmpstring)
    f.close




Aucun commentaire:

Enregistrer un commentaire