jeudi 12 octobre 2017

Querying data from hadoop dfs from web interface

Im stuck with a problem related to hadoop, What im doing is that i have a scrapper which is extracting data from an online source. This data is converted into json form. I have to store this data on hadoop.Now there will a web interface which will have search option. Now there will be two kinds of search options, One of them is that when i search some word, it will contact extractor and extract online data related to it. this data will be stored in json form in its specific repository for example it came from google then it will be stored in a folder specific for google on hadoop using redis server and apache flume in real time. Now other search option on web interface is that it will search some word or parameter which will in-turn seach it on hadoop dfs and have to return a result(json string) related to that search, for example lets say i have a data in hadoop with json string with key as {name:arslan, edu:cs, height,5.11, age,24} so if that web interface search name == arslan then it should get this json from there,now the problem is that i dont know how to do this part. Ill be grateful for help. thanks




Aucun commentaire:

Enregistrer un commentaire