dimanche 23 juillet 2017

Is there a way of capturing cross-site requests?

I have written a bash script that uses tools such as wget, sed and awk to extract data on all the climbing routes at a particular location from a website so I can have a hard copy when I'm in the middle of nowhere with no phone signal.

Some of the routes have an accompanying photo or two (for example) and I would like to extract them into my outputted file too but I've got no idea how or even if that's possible.

The photos are being served from a different domain (https://cdn.ukc2.com) and the names of the photo files have no bearing on the name of the climb. When I look through the source of the page I've linked to, there are references to the cdn domain but that is only for logos and other page elements. I can't see where the photos are being requested to replicate the request in my script. I wonder if it's something that's happening server side.

Is there a way I can use curl or wget or similar to identify the photos related to a particular climb and download them?

Note: I know it might be useful if I pasted the script, but as this could encourage potential copyright infringement I will decline. I am keeping the use of this script for my own personal use and therefore I believe it to be fair-use. All I'm doing is automating me clicking through every climbing route on a crag and printing it manually.




Aucun commentaire:

Enregistrer un commentaire