Posted By

rowntreerob on 11/03/09


Tagged

perl bookmarks linkcheck


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

wirenaught


run 'linkcheck' on bookmarks 2 of 2


 / Published in: Perl
 

source for perl file that is step 2

  1. #!/usr/bin/perl
  2.  
  3. require LWP::Parallel::UserAgent;
  4. use HTTP::Request;
  5.  
  6. my $pua = LWP::Parallel::UserAgent->new();
  7. $pua->in_order (1); # handle requests in order of registration
  8. $pua->duplicates(0); # ignore duplicates
  9. $pua->timeout (8); # in seconds
  10. $pua->redirect (1); # follow redirects
  11. $pua->max_hosts(5); # sets maximum number of locations accessed in parallel
  12. $pua->max_req (5); # sets maximum number of parallel requests per host
  13.  
  14. my(@reqs) = <>; # read input file from step 1
  15.  
  16. foreach my $req (@reqs) {
  17. # print "Registering $req \n";
  18. if ( my $res = $pua->register (HTTP::Request->new('GET', $req) ) ) {
  19. print STDERR $res->error_as_HTML;
  20. }
  21. }
  22. my $entries = $pua->wait();
  23.  
  24. foreach (keys %$entries) {
  25. my $res = $entries->{$_}->response;
  26. print "Answer for '",$res->request->url, "' was \t", $res->code,": ",
  27. $res->message,"\n";
  28. }

Report this snippet  

You need to login to post a comment.