Posted By

alemani on 04/27/10


Tagged

web mirror wget Website


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

armanx


Mirror a website using wget


 / Published in: Bash
 

If this command seems to run forever, it's because there may be parts of the site that generate an infinite series of different URLs. You can solve this in many ways, the simplest being to use the -l option to specify how many links “away” from the home page wget should travel. For instance, -l 3 will refuse to download pages more than three clicks away from the home page. You’ll have to experiment with different values for -l. Consult man wget for additional workarounds.

  1. wget -m -k -K -E http://url/of/web/site

Report this snippet  

You need to login to post a comment.