Home/ Questions/Q 8145069
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-06T13:32:59+00:00

I’m using wget to download new files from a FTP server. The new files

  • 0

I’m using wget to download new files from a FTP server. The new files that have been downloaded need to be processed by another script. 

wget -N -r ftp://server/folder

So my question is: How do i get a list of all files that wget has been downloaded?

Thanks in advance!

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You can use wget’s -o flag to pipe the standard output to a logfile. The logfile will have the same format as the regular output to a terminal, for example:

    --2012-06-28 17:57:13--  http://cdn.sstatic.net/stackoverflow/img/sprites.png
Resolving cdn.sstatic.net (cdn.sstatic.net)... 67.201.31.70
Connecting to cdn.sstatic.net (cdn.sstatic.net)|67.201.31.70|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16425 (16K) [image/png]
Saving to: `sprites.png'

0K .......... ......                                     100%  131K=0.1s

2012-06-28 17:57:13 (131 KB/s) - `sprites.png' saved [16425/16425]

    If you pipe this file through egrep as egrep -e "--" logfile.txt you will get only the lines specifying which files were downloaded, with timestamps. 

    --2012-06-28 17:57:13--  http://cdn.sstatic.net/stackoverflow/img/sprites.png

    If you wish, you can then pipe it to cut egrep -e "--" logfile.txt|cut -d ' ' -f 4 to get only to downloaded files.

    http://cdn.sstatic.net/stackoverflow/img/sprites.png

    You should add error-checks in here as well, but this is the basic outline.

    • 0