Web Analytics Made Easy -
StatCounter Getting HTML from other webistes - fopen/fsockopen - CodingForum

Announcement

Collapse
No announcement yet.

Getting HTML from other webistes - fopen/fsockopen

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Getting HTML from other webistes - fopen/fsockopen

    I am trying to retrieve information (HTML) from other sites using fopen and fscokopen.

    in the past, its all worked fine but i have found a couple of sites that I am having problem reading information from. eg:

    http://www.google.com/search?hl=en&l...afe=off&q=here

    or

    http://www.amazon.co.uk/exec/obidos/...277180-4903802

    I either get blank information or just random rubbish

    I can get this all to work in ASP using a similar method...

    any ideas? just looking for example code that willread either of those links

    Thanks
    Fatman
    Andrew Hall

  • #2
    my experience with amazon is that they dont want data grabbing , e.g. if they make it hard then you probably should not be doing it, many sites now require a cookie or use sessions to prevent remote data grabbers , in saying that this worked for me for google.

    PHP Code:
    <?
    $fp 
    fsockopen ("www.google.com"80$errno$errstr30); 
    if (!
    $fp) { 
       echo 
    "$errstr ($errno)<br>\n"
    } else { 
        
    fputs ($fp"GET /search?hl=en&lr=&ie=UTF-8&oe=UTF-8&safe=off&q=here HTTP/1.0\r\nHost: www.google.com\r\n");
       
    fputs ($fp"\r\n"); 
       while (!
    feof($fp)) { 
           echo 
    fgets ($fp,128); 
       } 
       
    fclose ($fp); 

    ?>
    Last edited by firepages; Feb 22, 2004, 09:41 PM.
    resistance is...

    MVC is the current buzz in web application architectures. It comes from event-driven desktop application design and doesn't fit into web application design very well. But luckily nobody really knows what MVC means, so we can call our presentation layer separation mechanism MVC and move on. (Rasmus Lerdorf)

    Comment


    • #3
      Not to mention copyright issues.

      Be very careful if your pulling in content from other sites without their permission. If it's for your personal site, you're probably safe with the "Fair Use" clause, but IANAL, so just be careful. It's also a really good idea to at least give credit to where you get the information.
      Create accessible online surveys -::- Koobten.com - compare netbook prices and reviews -::- Affordable, reliable hosting for less than $20 per year
      Zend Certified Engineer

      Comment


      • #4
        yep - im fine about things like that!

        its just the actual process and purely for personal use - but the amazon one is the one i want to work most

        any easy way to get it?
        Andrew Hall

        Comment


        • #5
          Amazon's got all sorts of neat tools you can use including XML feeds and the like. Go and sign up as an "amazon Associate" to get more info.
          Create accessible online surveys -::- Koobten.com - compare netbook prices and reviews -::- Affordable, reliable hosting for less than $20 per year
          Zend Certified Engineer

          Comment


          • #6
            i already am an associate!

            but cant see anything for wish lists....
            Andrew Hall

            Comment


            • #7
              There are two PHP technologies that I am somewhat familiar with, namely: cURL and Snoopy.

              Do a search for those online and you'll probably be able to use one of them to solve your problem.

              Good luck,
              Sadiq.

              Comment


              • #8
                tried snoopy - that did the same ?

                need to install curl - would any kind soul be willing to try this as i dont wanna ask my host to install it if it does nothing,,,,,
                Andrew Hall

                Comment

                Working...
                X