Question

As the question states how to instruct WGET to only download the same file if the existing one is older

e.g. fileA has a date / file stamp of 9.00AM 10/10/2011

e.g. fileA on the remote server has a date / file stamp of 11AM 10/10/2011

so wget will download FileA on the server as its newer (and overwrite the local file)

Any help would be greatly appreciated, I have heard this is possible, but after looking around for a while I havn’t come up with anything

Answer 1

Take a look at the timestamping section in the wget manual:

Time-Stamping

One of the most important aspects of
mirroring information from the
Internet is updating your archives.

Downloading the whole archive again
and again, just to replace a few
changed files is expensive, both in
terms of wasted bandwidth and money,
and the time to do the update. This is
why all the mirroring tools offer the
option of incremental updating.

Such an updating mechanism means that
the remote server is scanned in search
of new files. Only those new files
will be downloaded in the place of the
old ones.

A file is considered new if one of
these two conditions are met:

1. A file of that name does not already
   exist locally.

2. A file of that name
   does exist, but the remote file was
   modified more recently than the local
   file.

To implement this, the program
needs to be aware of the time of last
modification of both local and remote
files. We call this information the
time-stamp of a file.

The time-stamping in GNU Wget is
turned on using ‘–timestamping’
(‘-N’) option, or through timestamping
= on directive in .wgetrc. With this option, for each file it intends to
download, Wget will check whether a
local file of the same name exists. If
it does, and the remote file is not
newer, Wget will not download it.

If the local file does not exist, or
the sizes of the files do not match,
Wget will download the remote file no
matter what the time-stamps say.