Question

I have following jobs (just as example) to run in unix (bash shell) cluster computers:

### job1
mkdir file01
cp *.map  flex01
date > out
cd ..

### job2
mkdir file02
cp *.map  flex02
date > out
cd ..

### job3
mkdir file03
cp *.map  flex03
date > out
cd ..

### job4
mkdir file04
cp *.map  flex04
date > out
cd ..

If I submit these jobs, we do one by one. But I want to run them in parallel means that job1 to job4 running at the same time at the background.

How can I do it ? Sorry for the simple question, I am new to unix.

Answer 1

The cd commands don’t seem to be a good idea; you don’t cd into a directory. You’ll probably also want to append the date information to the output file, rather than always clobber it. It also seems more likely that you’d copy the map files to the directory you just created. So, you might write:

(mkdir flex01; cp *.map flex01; echo "Job 1: $(date)" >> out) &
(mkdir flex02; cp *.map flex02; echo "Job 2: $(date)" >> out) &
(mkdir flex03; cp *.map flex03; echo "Job 3: $(date)" >> out) &
(mkdir flex04; cp *.map flex04; echo "Job 4: $(date)" >> out) &

wait

This runs each sequence of commands as a separate background job, and then waits for them all to finish before proceeding. You could look at using a loop for this task, too.

for n in $(seq 1 4)
do
    (mkdir flex0$n; cp *.map flex0$n; echo "Job $n: $(date)" >> out) &
done

You could also consider using mkdir -p flex01 so you don’t get error messages when trying to create a directory that already exists. (Or you could test for errors and not copy if it exists, or test for existence before running mkdir, or clean it out before copying if it already exists, or …)