vendredi 20 juillet 2018

How to find out that there is a search filter implemented in the page?

In my project I need to parse webpages.I use Jsoup for parsing.There are some pages which couldn't be parsed if there are search filters.For these kind of webpages I use Selenium to extract the data.

In my project I need to scrape the career pages of companies we search for on the internet.Since the whole process is automated and volume of data is high,it's really hard to find out which page isn't getting parsed.

Is there a way to identify that there is a search filter in the webpage ,so that I can redirect the same to Selenium without doing it manually(need not search that there is a search filter present).




Aucun commentaire:

Enregistrer un commentaire