mardi 24 avril 2018

how can i build a website that can scrap another website giving it URL ?

i already made my own spider using scrapy , and i don't know how to use it in a website so i can control it from the website.

    import scrapy


class QuotesSpider(scrapy.Spider):
name = "quotes"
start_urls = [
    'http://quotes.toscrape.com/page/1/',
    'http://quotes.toscrape.com/page/2/',
]

def parse(self, response):
    for quote in response.css('div.quote'):
        yield {
            'text': quote.css('span.text::text').extract_first(),
            'author': quote.css('small.author::text').extract_first(),
            'tags': quote.css('div.tags a.tag::text').extract(),
        }




Aucun commentaire:

Enregistrer un commentaire