mercredi 1 juillet 2015

Web scraping with snowfall fails?

I'm lapplying over a bunch of URLs to get some data out and in this the readLines(<URL>) command works fine. When I switch to sfLapply the code is unable to read the webpage. Anyone know why? Example below

library(snowfall)
library(rlecuyer)
# looping through each combination of fruit and dish 

sfInit(parallel = T, cpus  = as.integer(Sys.getenv('NUMBER_OF_PROCESSORS')) - 1)
dtData = # lapply(
sfLapply(
   c('apple','mango','banana'),
   function(fruit) {
      cat(fruit,'\n')
      lapply(
         c('pie','shake'),
         function(dish) {
            # getting the data 
            vcTemp = readLines(paste0('http://ift.tt/1BYMhzw',dish,' ',fruit))
         }
      )
   }
)

sfStop()




Aucun commentaire:

Enregistrer un commentaire