jeudi 23 avril 2015

How to retrieve all the images, js, css urls

I was going through all the scrapy examples and tutorials I can find and I couldn't find an example where I can go and get all the urls of the images, css, and js files being sent from the server.

Is there a way to do that with scrapy? If not with scrapy, then is there a way to do it with something else?

I basically want to go through my website and get all the urls/resources and output them to a log file.




Aucun commentaire:

Enregistrer un commentaire