mardi 16 juillet 2019

Node.js request not returning hmtl on specific websites

I am trying to write a web scraper for a NYC database of building and I am trying to get the html of the actual website. For whatever reason, when I put the url of the website I am trying to scrape, my program does nothing. Whenever I put the url of almost any other website, I actually get the html i requested. Is this because I am trying to scrape a government site?

var request = require("request");

request(
    { uri: "http://a810-bisweb.nyc.gov/bisweb/JobsQueryByNumberServlet?requestid=3&passjobnumber=123768556&passdocnumber=01" },
    function(error, response, body) {
        console.log(body);
        console.log("hello")
    }
);

I expected to recieve the html as a string printed in my console, instead, I get nothing. The "hello" is not even printed. However, when I try any other site, I get the actual html string.

Aucun commentaire:

Enregistrer un commentaire