Question

I’m trying to use GNU parallel to post a lot of files to a web server. In my directory, I have some files:

file1.xml
file2.xml

and I have a shell script that looks like this:

#! /usr/bin/env bash

CMD="curl -X POST -d@$1 http://server/path"

eval $CMD

There’s some other stuff in the script, but this was the simplest example. I tried to execute the following command:

ls | parallel -j2 script.sh {}

Which is what the GNU parallel pages show as the “normal” way to operate on files in a directory. This seems to pass the name of the file into my script, but curl complains that it can’t load the data file passed in. However, if I do:

find . -name '*.xml' | parallel -j2 script.sh {}

it works fine. Is there a difference between how ls and find are passing arguments to my script? Or do I need to do something additional in that script?

Answer 1

I have not used parallel but there is a different between ls & find . -name '*.xml'. ls will list all the files and directories where as find . -name '*.xml' will list only the files (and directories) which end with a .xml.
As suggested by Paul Rubel, just print the value of $1 in your script to check this. Additionally you may want to consider filtering the input to files only in find with the -type f option.
Hope this helps!