advanced code snippet search

We Recommend

bash Cookbook: Solutions and Examples for bash Users bash Cookbook: Solutions and Examples for bash Users
bash Cookbook teaches shell scripting the way Unix masters practice the craft. It presents a variety of recipes and tricks for all levels of shell programmers so that anyone can become a proficient user of the most common Unix shell -- the bash shell -- and cygwin or other popular Unix emulation packages.

Posted By

noah on 11/02/07


Tagged

http download commandline pipe web perl lwp shortcut tools productivity textonly grabit numbered sequential scraping


Versions (?)


Who likes this?

2 people have marked this snippet as a favorite

basicmagic
vali29


Download linked JPEGs from a Web page, on the command line

Published in: Bash 

The following command will download all the files with a JPG extension that are linked from http://flickr.com.

Plain TextExpand 
  1. lwp-request -o links http://flickr.com/ | grep jpg | perl -pe "chomp; s/.*?(\S+jpg)/$1 /;" | xargs wget
  2.  
  3. #or
  4.  
  5. lwp-request -o links http://flickr.com | grep jpg | perl -pe "chomp; s/IMG\s*(.*jpg)/$1 /;" | xargs wget
  6.  
  7.  
  8. #The following will download the numbered images site.com/gallery/01.jpg through sitem.com/gallery/16.jpg
  9.  
  10. perl -e "$i=0;while($i<16){open(WGET,qq/|xargs wget/);printf WGET qq{http://site.com/gallery/%02d.jpg},++$i}"

Report this snippet 

You need to login to post a comment.