Question

Is there a way to directly pass parameters to a .pbs script before submitting a job? I need to loop over a list of files indicated by different numbers and apply a script to analyze each file.

The best I’ve been able to come up with is the following:

#!/bin/sh 

for ((i= 1; i<= 10; i++))
do
        export FILENUM=$i
        qsub pass_test.pbs
done

where pass_test.pbs is the following script:

#!/bin/sh

#PBS -V
#PBS -S /bin/sh
#PBS -N pass_test
#PBS -l nodes=1:ppn=1,walltime=00:02:00
#PBS -M [e-mail]

cd /scratch/XXXXXX/pass_test

./run_test $FILENUM

But this feels a bit wonky. Particularly, I want to avoid having to create an environment variable to handle this.

Answer 1

The qsub utility can read the script from the standard input, so by using a here document you can create scripts on the fly, dynamically:

#!/bin/sh

for i in `seq 1 10`
do
    cat <<EOS | qsub -
#!/bin/sh

#PBS -V
#PBS -S /bin/sh
#PBS -N pass_test
#PBS -l nodes=1:ppn=1,walltime=00:02:00
#PBS -M [e-mail]

cd /scratch/XXXXXX/pass_test

./run_test $i
EOS
done

Personally, I would use a more compact version:

#!/bin/sh

for i in `seq 1 10`
do
    cat <<EOS | qsub -V -S /bin/sh -N pass_test -l nodes=1:ppn=1,walltime=00:02:00 -M [e-mail] -
cd /scratch/XXXXXX/pass_test
./run_test $i
EOS
done