lundi 23 novembre 2020

Using puppeteer to scrape from multiple select boxes

So I'm writing this web scraper to scrape values out of a bunch of select boxes, Each select box is populated based on the value of the previous select box. So i've managed to to set the value of the first select box but now when i get the second boxes. value it just shows the initial value, as if its not waiting for the first box to be set.

const puppeteer = require('puppeteer');
async function Main(){
    const browser = await puppeteer.launch({ headless: false }); // for test disable the headlels mode,
    const page = await browser.newPage();
    await page.setViewport({ width: 1000, height: 926 });
    await page.goto("website");
      // Get a list of all 'li a' text
      await page.select('.vehicle-form>.field>select', 'Abarth')
      const listElements = await page.evaluate(() => Array.from(document.querySelectorAll('select option'), element => element.textContent));
      const listElements1 = await page.evaluate(() => Array.from(document.querySelectorAll('select[name="model"'), element => element.textContent));

      console.log('list elements', listElements1);
}

Main();
``



Aucun commentaire:

Enregistrer un commentaire