dimanche 14 juillet 2019

Where are my mistakes in my scrapy codes?

I want to crawl a website via scapy but my codes come up with an error.

I have tried to use xpath but it seems I can not define the div class in the web site.

The following codes were already tested but this part (("h2 ::text").extract()) of the code comes with an error.

import scrapy
from scrapy.selector import Selector
from artistlist.items import ArtistlistItem

class MySpider(scrapy.Spider):
    name = "artistlist"
    allowed_domains = ["baltictriennial13.org"]
    start_urls = ["https://www.baltictriennial13.org/artist/caroline-achaintre/"]

    def parse(self, response):
        hxs = Selector(response)
        titles = hxs.xpath("//div[@class='artist']")
        items = []
        for titles in titles:
            item = ArtistlistItem()
            item["artist"] = titles.select("h2 ::text").extract()
            item["biograpy"] = titles.select("p::text").extract()
            items.append(item)
        return items

I want to crawl the web site and store the data in a .csv file.

Aucun commentaire:

Enregistrer un commentaire