mercredi 29 avril 2015

Distributed Web crawling using Apache spark

It's an interesting question asked me when I attended one interview about Apache spark,as a part of web mining. The question was, is it possible to crawl the Websites using Apache spark?. Then I was confused. I answered possible, just guess. Then next question how? It's because it supports distributed processing capacity of spark. After the interview I had searched it, but couldn't find any interesting answer. Is that possible with spark any help?




Aucun commentaire:

Enregistrer un commentaire