What is the most efficient way of scanning entire internet for particular files?

What could be the most efficient ways of scanning the entire internet for say JavaScript files and parse them to get all the Relative URLs & Full URLs (domain-name.com/*) present in them?