Snipplr

(Bash) Copy a dynamic website using wget - alexaivars

Tue, 14 Aug 2012 16:57:32 GMT

Creating a static copy of a dynamic website

(Bash) WGET Wordpress SALT Keys - paullombard

Tue, 17 Jul 2012 19:17:32 GMT

WGET the SALT keys from Wordpress' API

(Bash) WGET Resources From Remote Host - paullombard

Wed, 13 Jun 2012 01:04:01 GMT

Fetches resources from a remote host with specified crawl depth.

(Bash) mirror a website with wget - archvi

Wed, 07 Dec 2011 12:03:43 GMT

Mirroring only from allowed sub-dir -m : mirror website -k : relative urls for downloaded files, otherwise your mirrored link point to original site -I : allowed dir --domains : allowed domains

(Bash) Get files (wget) - klovera

Tue, 18 Oct 2011 13:31:48 GMT

Used to get file. First one is done via http. Second is done via FTP.

(Bash) wgetm3u.sh - cruelfate

Sat, 22 Jan 2011 07:52:45 GMT

(Bash) Save entire web site with wget - minipark

Wed, 20 Oct 2010 01:01:31 GMT

(Bash) How to Build and Install GNU wget on Apple OS X 10.5 Leopard - armanx

Wed, 06 Oct 2010 21:06:52 GMT

**Download**\r\n\r\n1. Download the latest source code from the [GNU wget website](http://www.gnu.org/software/wget/ \"GNU wget website\"). You can download the latest version from [this link](http://ftp.gnu.org/gnu/wget/wget-latest.tar.gz \"Latest wget version\").\r\n\r\n2. Extract the downloaded archive to a directory of your choice. For this example, we will extract to a directory on the desktop.\r\n\r\n**Configure**\r\n\r\n1. Once extracted, open up the terminal, and `cd` into the directory where you extracted the wget source code. Adjust the example below accordingly.\r\n\r\n `cd /Users/admin/Desktop/wget-1.11.4/`\r\n\r\n2. Before compiling, we must configure the source code for our system by running the configure script that is included with the source code. The configure script lets us specify options on how we want to build the program.\r\n\r\n For example, we can control where to install the program. By default, wget will install into `/usr/local/bin`. If we wish to change the install location, we can run the configure script with the `--prefix=/path/to/install/location` flag.\r\n\r\n In the example below, we will install all the wget files in its own directory (/usr/local/wget).\r\n\r\n **If you want to install wget into the default location (`/usr/local/bin`), *omit* the `--prefix=/usr/local/wget` when running the command below.**\r\n\r\n `./configure --prefix=/usr/local/wget`\r\n\r\n**Compiling and Installing**\r\n\r\n1. Once the configure script is done, type in `make` to start building the wget binaries. It will take several seconds to compile wget depending on the speed of your system. Once the compilation is complete, install the binaries (as root) by typing in `sudo make install`.\r\n\r\n `make`\r\n \r\n `sudo make install`\r\n\r\n2. If you used the default installation path, restart your terminal session and type in `wget` to run the program.\r\n\r\n\r\n3. If you used the location in our example, you must use the full path to the binary:\r\n\r\n `/usr/local/wget/bin/wget http://www.example.com`\r\n\r\n * To use `wget` instead of the full path, you can add wget to your `.profile` path, or add it system-wide by editing `/etc/paths`.\r\n\r\n `sudo nano /etc/paths`\r\n\r\n * Add `/usr/local/wget/bin` to the end of the file as shown. When done, press CTRL+X, answer \"y\" to save the changes, and press RETURN to update the file. Restart your terminal session to reload the paths.\r\n\r\n `/usr/bin`\r\n \r\n `/bin`\r\n \r\n `/usr/sbin`\r\n \r\n `/sbin`\r\n \r\n `/usr/local/bin`\r\n \r\n `/usr/local/wget/bin`

(Bash) Using wget to download a single web page and all its requisite files, displaying properly locally. - armanx

Wed, 06 Oct 2010 20:40:32 GMT

from the wget manual (1.12):\r\n\r\n\"Actually, to download a single page and all its requisites (even if they exist on separate websites), and make sure the lot displays properly locally, this author likes to use a few options in addition to ‘-p’: \"

(Bash) get_external_ip.sh - ksaver

Sun, 22 Aug 2010 16:03:50 GMT

(Bash) bash [script] - download website using wget - kentoy

Mon, 09 Aug 2010 16:31:37 GMT

save as filename.shchmod u+x filename.shusage : ./filename.sh\"url\"

(Bash) Mirror a website using wget - alemani

Tue, 27 Apr 2010 09:56:12 GMT

If this command seems to run forever, it's because there may be parts of the site that generate an infinite series of different URLs. You can solve this in many ways, the simplest being to use the -l option to specify how many links “away” from the home page wget should travel. For instance, -l 3 will refuse to download pages more than three clicks away from the home page. You’ll have to experiment with different values for -l. Consult man wget for additional workarounds.

(Bash) Watch a page for changes - benpjohnson

Thu, 04 Feb 2010 06:29:14 GMT

NB not yet working as I want it too as timeout isn't working correctly

(Bash) Commandline oneliner to get your public IP adress - berkes

Thu, 04 Feb 2010 05:58:52 GMT

Use wget and sed to get your public IP adress (instead of firing up firefox and visting whatismyip.com). Usefull in scripts where you need your public IP adres. There are probably better ways, maybe even a GNUtool that I missed?

(Bash) print a list of bookmarks on delicous in a certain tag. - berkes

Sun, 31 Jan 2010 15:59:52 GMT

Simple wget - grep - sed pipe to print a list of bookmarks on delicous in a certain tag. Replace {USERNAME} with your username. Replace {TAG} with the tag you want a list of urls for.

(Bash) Downloading an Entire Web Site with wget - abhiomkar

Fri, 27 Nov 2009 12:24:00 GMT

Source: Linux Journal This command downloads the Web site www.website.org/tutorials/html/. The options are: * --recursive: download the entire Web site. * --domains website.org: don't follow links outside website.org. * --no-parent: don't follow links outside the directory tutorials/html/. * --page-requisites: get all the elements that compose the page (images, CSS and so on). * --html-extension: save files with the .html extension. * --convert-links: convert links so that they work locally, off-line. * --restrict-file-names=windows: modify filenames so that they will work in Windows as well. * --no-clobber: don't overwrite any existing files (used in case the download is interrupted and resumed).

(Bash) Check for 404s with wget - benpjohnson

Wed, 04 Nov 2009 12:26:09 GMT

(Bash) Archive & Timestamp Webcam Images - cshaiku

Tue, 30 Jun 2009 01:38:38 GMT

(Bash) (HTML) Tutorial Page Download with 'wget' - abhiomkar

Fri, 05 Jun 2009 15:14:26 GMT

Download all tutorial pages of html format recursively. Ex: http://www.moraware.com/help/ Get the url of tree frame (left), use it in wget. Have fun :)

(Bash) RFC Downloader Enhanced - zxeem

Fri, 15 May 2009 20:33:27 GMT

THERE IS A BUG AFTER COUNTING 100