dimanche 7 janvier 2018

How to anti-web crawler using Ruby on Rails?

As you can try the following link. I want to crawl the historical data.

I thought there must be an Api. like xxx/api/v1/tron.json which will return you a json data as the response. In this way, a malicious user could crab bulks of data easily.

However, I found the response from the server is an html structure.

What's the name of this technique? Server rendering? or ?

In Ruby on Rails, The similar way to act like this is that uses ERB to render the HTML page, right?

http://ift.tt/2qB3kZb

{"ads":
 {
  "version": 1,
  "count": 2,
  "pixels":"http://ift.tt/2m9sjOn",
  "oxt": 184.025,
  "ad": [
        {
         "adunitid":539181151,
         "adid":537153255,
         "type":"html",
         "html":"<div id='beacon_QmWzibqKt2xuqrAU316bjIV8ID' style='position: absolute; left: 0px; top: 0px; visibility: hidden;'><img src='http://ift.tt/2qEuCgW"
             }
           }
         ]
        }]
 }
}




Aucun commentaire:

Enregistrer un commentaire