Is it possible to BFS through hyperlinks?

I was thinking about web navigation, and I came up with an interesting idea. I'm wondering if it is possible for a program to navigate through webpages via hyperlinks (in the fashion of a souped-up-BFS)?

Theoretically, I think it would be possible, but I'm trying to think up of an approach where multiple computers would be running the program and writing the webpage names to a server. Then, the server data could be read and we'd have a massive graph of all (almost) the websites in the world.