advanced code snippet search

We Recommend

Learning Perl Learning Perl
In this smooth, carefully paced course, a leading Perl trainer teaches you to program in the language that threatens to make C, sed, awk, and the Unix shell obsolete for many tasks. This book is the "official" guide for both formal (classroom) and informal learning. It is fully accessible to the novice programmer.

Posted By

hxseven on 08/14/06


Tagged

parse html example request perl lwp


Versions (?)


LWP request example

Published in: Perl 

URL: http://codedump.jonasjohn.de/snippets/lwp_example.htm

Shows how to load a text file Another good resource: Web Client Programming with Perl.

Plain TextExpand 
  1. #!/usr/bin/perl
  2.  
  3. # load LWP library:
  4. use LWP::UserAgent;
  5. use HTML::Parse;
  6.  
  7. # define a URL
  8. my $url = 'http://www.jonasjohn.de';
  9.  
  10. # create UserAgent object
  11. my $ua = new LWP::UserAgent;
  12.  
  13. # set a user agent (browser-id)
  14. # $ua->agent('Mozilla/5.5 (compatible; MSIE 5.5; Windows NT 5.1)');
  15.  
  16. # timeout:
  17. $ua->timeout(15);
  18.  
  19. # proceed the request:
  20. my $request = HTTP::Request->new('GET');
  21. $request->url($url);
  22.  
  23. my $response = $ua->request($request);
  24.  
  25.  
  26. #
  27. # responses:
  28. #
  29.  
  30. # response code (like 200, 404, etc)
  31. my $code = $response->code;
  32.  
  33. # headers (Server: Apache, Content-Type: text/html, ...)
  34. my $headers = $response->headers_as_string;
  35.  
  36. # HTML body:
  37. my $body =  $response->content;
  38.  
  39.  
  40.  
  41. # print the website content:
  42. # print $body;
  43.  
  44.  
  45. # do some parsing:
  46.  
  47. my $parsed_html = HTML::Parse::parse_html($body);
  48. for (@{ $parsed_html->extract_links(qw(a body img)) }) {
  49.  
  50.     # extract all links (a, body, img)
  51.     my ($link) = @$_;
  52.  
  53.     # print link:
  54.     print $link . "\n";
  55. }

Report this snippet 

You need to login to post a comment.