building a product web directory with scrapping

is there any use of a web spider in building a web direcity website either in Java/Php or python ? confused can you please enlighten me on why and when you need to spider? Is building a web directory all a human based task?

thanks