samedi 25 juin 2016

ruby web scraping nokogiri from "Load more" button

I wish to get list of links given on contact name from page http://ift.tt/28Yo04I

Shows only 10 results and wish to extract more then need to click on "Load more"

Can you give bit idea how to do it?

I already code like as following and its showing result of 10 records. Wish to get all links given on contact name by clicking "Load more"

Please help me if you can.

require 'nokogiri'

require 'open-uri'

require 'csv'

require 'openssl'

require 'json'

data_array=[]

url='http://ift.tt/28Yo04I'

nokogiri_object = Nokogiri::HTML(open(url).read)

fparse = nokogiri_object.xpath("//h3/a[@class='search-result-link']/@href")

fparse.each do |a|

post_name = a.text

furl= 'http://ift.tt/13R3BmF' + post_name

data_array.push(furl)

end

CSV.open('yp.csv', 'w', {:col_sep => "\n"}) do |csv|

csv << data_array

end




Aucun commentaire:

Enregistrer un commentaire