Create a website Crawler

[treeleaf20]

All,
I'm looking to do something similar to how Google indexes web pages. For example, if I find a link that says Contact Us I'd like to get that information from the website. Any ideas or links out there that people have used to obtain this type of information?

Thanks in advance!

[ajay1831]

can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

[Jinxy]

can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

This might get you started. This will check all the links on a page and tell you if they are good.

http://pastebin.com/5uaRuDG2