PDA

View Full Version : How to download / mirror a website easilly from linux?



Fli
04-15-2014, 10:17 PM
Hi,

if you need to save some website for future / offline browsing, it is quite easy from Linux.

execute this command on command line:


wget -mk http://yourwebsite.com/

If your linux machine is a webserver, you may try to cd into web directory like:


cd /var/www/html

create new directory named mirror and go to that directory


mkdir mirror;cd mirror

then do the wget above

PS:
add "-w 2" (without quotation marks) after the wget command if you want to delay each request for 2 seconds
add "-q" (without quotation marks) after the wget command if you want to disable verbose output

if the website is large, alot of files and process can take many hours and you working on remote server via SSH, you should better run the wget via "screen (http://internetlifeforum.com/linux-forums/1214-linux-command-screen-keep-commands-running%3B-work-separate-window/)" so closing terminal wont interupt operation

===========
Option 2 - using well known advanced web coppier - Httrack


yum install zlib-devel
wget http://download.httrack.com/cserv.php3?File=httrack.tar.gz -O httrack.tar.gz
tar xvfz httrack.tar.gz
cd httrack-3.47.27
./configure
make && sudo make install

Then this start mirroring

httrack --ext-depth=1 http://yourwebsite.com
(add depth if needed)