OVH Community, your new community space.

Using Wget to create a site copy/mirror?


Iray
08-03-2012, 19:31
Quote Originally Posted by jonlewi5
Not tried myself, but take a look here

http://www.linuxjournal.com/content/...-web-site-wget

This is the example given

Code:
wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.org \
     --no-parent \
         www.website.org/tutorials/html/
Thanks, tried this but I believe -mirror option does most of this already.

Quote Originally Posted by yonatan
do you have access to the server which runs the sites?

why use http protocol for the mirror creation?
Using HTTP for the sites I have in mind is perfect for what I need, they are mainly static but in high demand always.

I think so far -mirror with wget is still the best way to quickly grab a static copy. I'll keep searching though.

jonlewi5
07-03-2012, 11:49
Not tried myself, but take a look here

http://www.linuxjournal.com/content/...-web-site-wget

This is the example given

Code:
wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.org \
     --no-parent \
         www.website.org/tutorials/html/

yonatan
07-03-2012, 10:50
do you have access to the server which runs the sites?

why use http protocol for the mirror creation?

Iray
07-03-2012, 06:51
Hello,

I need to create a full site mirror (static mirror) of some sites and so far I have found wget to be useful, particularly in -mirror mode.

However it has missed some files, specifically some CSS linked images and flash files, the site looks incomplete.

Does anyone know of a better command than my current Wget command? Perhaps with added functions to literally download a perfect mirror / copy?

Any help appreciated.