Question

I’m having trouble with braces (curly brackets) using GNU parallel (http://www.gnu.org/software/parallel/)

I have a list of four files:

file1.txt.super
file2.txt.super
file3.txt.super
file4.txt.super

If I issue: ls * | parallel "mkdir ./{.}"

I get returned four directories:

file1.txt
file2.txt
file3.txt
file4.txt

My question is, how can I simply return four directories called:

file1
file2
file3
file4

I have read http://www.linuxjournal.com/article/8919 but have been unable to implement these regex’s with gnu parallel. I think I’m missing something here. Also, any examples with much more complicated regex would be very much appreciated.

Answer 1

Yes, it appears you are missing something here. The linuxjournal article explains features of shell parameter expansion. Those braces (which are always preceded by $) are unrelated to parallel utility’s default replacement strings, which coincidently use braces. The parallel documentation shows command line options allow arbitrary strings to be used instead of its brace-enclosed defaults.

For example the replacement string {.} in your example could be changed to %foo

ls * | parallel --extensionreplace %foo "mkdir ./%foo"

More information about ${…} from the linuxjournal article can be found in the man bash page, in the Parameter Expansion section.

Since you asked in the comment on @AdamLiss answer, here is a way to (ab)use the curly braces and the --colsep parameter to perform your task:

ls * | parallel --colsep '\.' "mkdir ./{1}"

Note: this --colsep trick (like the sed proposed by @AdamLiss) will produce undesirable results if the filenames contain more than two periods (since the pathname is truncated at the first period.)

However, since the --colsep parameter is a regular expression, this should be resilient to periods elsewhere in the filename:

ls * | parallel --colsep '\.[^\.]*$' "mkdir ./{1.}"

Note: --extensionreplace isn’t working due to a bug in the current (21120422) version of parallel. But since parallel is an a perl script, you can fix it by changing:

    "extensionreplace|er" => \$::opt_U,

to

    "extensionreplace|er=s" => \$::opt_U,