Parameter for Shell Scripts That Is Started with Qsub

parameter for shell scripts that is started with qsub

I just figured out how to solve it: just print the commands of the shell scrip with echo and pipe the result to qsub:

echo "./script.sh var1=13 var2=24" | qsub

Does qsub pass command line arguments to my script?

You can pass arguments to the job script using the -F option of qsub:

qsub script.sh -F "args to script" 

or inside script.sh:

#PBS -F arguments

This is documented here.

Use variables as argument parameters inside qsub script

Beware that to Bash, #$ -t 1-$[numModels] is nothing more than a comment; hence it does not apply variable expansion to numModels.

One option is to pass the -t argument in the command line: remove it from your script:

#$ -S /bin/bash
#$ -cwd 
#$ -V

model=(`ls $MODELS`)
echo "Starting ${model[$SGE_TASK_ID-1]}..."

and submit the script with

MODELS=/home/sahil/Codes/bistable/models qsub -t 1-$(ls $MODELS|wc -l) submit.sh

If you prefer to have a self-contained submission script, another option is to pass the content of the whole script through stdin like this:

#!/bin/bash
qsub <<EOT
MODELS=/home/sahil/Codes/bistable/models
numModels=(`ls $MODELS|wc -l`)
echo $numModels 

#$ -S /bin/bash
#$ -cwd 
#$ -V
#$ -t 1-$[numModels] # Running array job over all files in the models directory.

model=(`ls $MODELS`)
echo "Starting ${model[$SGE_TASK_ID-1]}..."
EOT

Then you source or execute that script directly to submit your job array (./submit.sh rather than qsub submit.sh as the qsub command is here part of the script.

Setting PBS/Torque/qsub parameters in script via command line arguments

@dbeer's answer gave me a little more insight. The solution to my problem is as follows:

#PBS args are overwritten by the command line. In such case, the args to PBS and the script itself must be separated. Therefore, rather than trying to do something like:

#PBS -l nodes=${NODES}:ppn=${PPN},walltime=${WALLTIME}

/some/command ${ARG1}

inside the script and running like

qsub script.sh -v NODES=2,PPN=2,WALLTIME=160:00:00,ARG1=2

these can all be set with args to qsub itself:

qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00

Then, any args that need to be passed to the executable can be passed through the -v argument to qsub:

qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00 -v ARGS1=2

Directly pass parameters to pbs script

The qsub utility can read the script from the standard input, so by using a here document you can create scripts on the fly, dynamically:

#!/bin/sh

for i in `seq 1 10`
do
    cat <<EOS | qsub -
#!/bin/sh

#PBS -V
#PBS -S /bin/sh
#PBS -N pass_test
#PBS -l nodes=1:ppn=1,walltime=00:02:00
#PBS -M XXXXXX@XXX.edu

cd /scratch/XXXXXX/pass_test

./run_test $i
EOS
done

Personally, I would use a more compact version:

#!/bin/sh

for i in `seq 1 10`
do
    cat <<EOS | qsub -V -S /bin/sh -N pass_test -l nodes=1:ppn=1,walltime=00:02:00 -M XXXXXX@XXX.edu -
cd /scratch/XXXXXX/pass_test
./run_test $i
EOS
done

How can I pass command line arguments containing braces using sun grid engine qsub?

I was able to solve my problem by running qsub -b y -cwd -shell no python script.py aaa{} instead of qsub -b y -cwd python script.py aaa{}. On my system, -shell yes seemed to be enabled by default, which initiated some preprocessing. Adding -shell no appears to fix this.

---

Related Topics