Question

I’m using bash shell.

Hi,ppl
Would be glad if someone could provide some kind of advice, because googling around yielded some answers
but couldn’t still get the script to work.

I’am new to using bash script and got a script to modify because it was failing to copy
a large number of files from and input directory to an output directory after the files were processed.

Description:

We have a bunch of pdf’s in a large directory.
We process a file called filename.pdf, after it’s processed an additional file is created called filename.pdf.marker
Then both files filename.pdf.marker and filename.pdf shoud be moved from input/in directory to directory output/out.
We work with about 10 -15 thousands of files.

The script should do the following:

1. select all .marker file names
2. move.marker files from input/in directory to directory output/out (done in separate line)
3. remove the .marker from the selected filename,
4. move the file filename.pdf to the output/out directory

Old script (didn’t work for a larger number of files) :

FILELIST=$(ls ${V04}/*.pdf.marker 2> /dev/null | sort)
for FILEMARKER in ${FILELIST}; do
    FILENAME=${V04}/$(basename $FILEMARKER .marker)
        mv ${FILENAME} ${VLOGDIR}/.  
        mv ${FILENAME}.marker ${VLOGDIR}/.  
    done  

Because of that I needed to use xargs command.

Problem:

I managed to move the .marker files in a separate line.
Now i need to move the .pdf files with this script line.

find /input/in -iname "*.marker" -print0 | xargs -0 -r -I {} mv `basename {} .marker`  /output/out

My problem lies in the part: `basename {} .marker`
Why isn’t the string filename.pdf extracted from the string filename.pdf.marker, and substituted into the mv command ?

Any help i’s welcome 😉

UPDATED

• Corrected description of what script should do: Both filetypes .pdf
  and .pdf.marker should be moved in my script not copied.
• Added old script that didn’t work well for larger amount of files.

Answer 1

The problem with the old script was, that you try to catch all entries in one var, which have size limits.
You can solve that, if you must not sort the entries in this way:

ls -1 "${V04}/*.pdf,.marker | while read FM; 
do 
    mv "${FM}" "${VLOGDIR}/"
    mv "${V04}/$(basename "${FM}" .marker)" "${VLOGDIR}/"
done;