I'm in the process of setting up test cases for an app I've written. That app communicates with an external partner who offers a website interface to check and review data submitted to them. Now, I obviously don't want to manually go on their website and check whether everything is in order, instead I'd like to automate the whole process. The data is slightly altered by the partner's platform, so I have to go through their website to check data integrity.
To that end I was looking into preferrably open source/permissive license web scrapers and/or similar that support HTTPSC.
I've looked around a bit but the sheer amount of web scrapers is daunting and the one web scraper I've tried getting into, Scrapy for Python, comes with relatively big technical hurdles just to run (libxml2, compilation of libs for twisted) and I'm not even sure it's the right tool for the job.
Am I on the right track with Scrapy or are there better alternatives?
Aucun commentaire:
Enregistrer un commentaire