Web Analytics Made Easy -
StatCounter Create a website Crawler - CodingForum

Announcement

Collapse
No announcement yet.

Create a website Crawler

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Create a website Crawler

    All,
    I'm looking to do something similar to how Google indexes web pages. For example, if I find a link that says Contact Us I'd like to get that information from the website. Any ideas or links out there that people have used to obtain this type of information?

    Thanks in advance!

  • #2
    can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..

    Comment


    • #3
      Originally posted by ajay1831 View Post
      can any one tell me the source code in php for a web crawler that crawl for all links ...thanks in advance..
      This might get you started. This will check all the links on a page and tell you if they are good.

      http://pastebin.com/5uaRuDG2

      Comment

      Working...
      X