Use Bash Variable Within Slurm Sbatch Script

Use Bash variable within SLURM sbatch script

This won't work. What happens when you run

sbatch myscript.sh

is that slurm parses the script for those special #SBATCH lines, generates a job record, stores the batch script somewhere. The batch script is executed only later when the job runs.

So you need to structure you workflow in a slightly different way, and first calculate the number of procs you need before submitting the job. Note that you can use something like

sbatch -n $numProcs myscript.sh

, you don't need to autogenerate the script (also, mpirun should be able to get the number of procs in your allocation automatically, no need to use "-np").

Import bash variables into slurm script

Okay, I've solved it:

sbatch --export=sampleID=$sampleID myscript.sh

bash variable expansion in slurm's #SBATCH directive

No, this isn't possible. But you can overwrite the scripts default --array by giving it explicitly on the sbatch command line.

run job script with different variables found in other files

This is the typical workload suited for a job array. With a submission script like this

#!/bin/bash
#SBATCH --job-name test
#SBATCH --ntasks 8
#SBATCH --time 04:00
#SBATCH --output out
#SBATCH --error err
#SBATCH --array=0-3

readarray -t VARS < var.txt
VAR=${VARS[$SLURM_ARRAY_TASK_ID]}
export VAR

Rscript script.R

and script.R being

print(Sys.getenv("VAR"))

you will get a four-jobs job array, each one running the R script with a different value of the env var VAR, taken from the var.txt file.

Setting and passing a string variable in SLURM job script

Actually, the single quotes will be striped by Bash during the assignment.

❯ MYSTRING='blablabla'
❯ echo $MYSTRING
blablabla

You can try to escape the quotes like this:

❯ MYSTRING=\'blablabla\'
❯ echo $MYSTRING
'blablabla'

Otherwise, Matlab wrongly believe blablabla is the name of a variable or a function and not a string literal. The error message is misleading though as it adds a single quote around blablabla. But those single quotes are not the ones that you inserted in the MYSTRING=... assignment, as shown here below in an interactive Matlab session.

$ matlab
MATLAB is selecting SOFTWARE OPENGL rendering.

< M A T L A B (R) >
Copyright 1984-2019 The MathWorks, Inc.
R2019a (9.6.0.1072779) 64-bit (glnxa64)
March 8, 2019

To get started, type doc.
For product information, visit www.mathworks.com.

>> blablabla
Undefined function or variable 'blablabla'.

Pass command line arguments via sbatch

The lines starting with #SBATCH are not interpreted by bash but are replaced with code by sbatch.
The sbatch options do not support $1 vars (only %j and some others, replacing $1 by %1 will not work).
When you don't have different sbatch processes running in parallel, you could try

#!/bin/bash

touch outFile${1}.txt errFile${1}.txt
rm link_out.sbatch link_err.sbatch 2>/dev/null # remove links from previous runs
ln -s outFile${1}.txt link_out.sbatch
ln -s errFile${1}.txt link_err.sbatch

#SBATCH -o link_out.sbatch
#SBATCH -e link_err.sbatch

hostname
# I do not know about the background processing of sbatch, are the jobs still running
# at this point? When they are, you can not delete the temporary symlinks yet.

exit 0

Alternative:
As you said in a comment yourself, you could make a masterscript.
This script can contain lines like

cat  exampleJob.sh.template | sed -e 's/File.txt/File'$1'.txt/' > exampleJob.sh
# I do not know, is the following needed with sbatch?
chmod +x exampleJob.sh

In your template the #SBATCH lines look like

#SBATCH -o "outFile.txt"
#SBATCH -e "errFile.txt"

Understanding internal variable substitution with backticks in an sbatch --wrap statement

You need to escape the backticks, just like you escaped the quotes and the $ before internal variables. Otherwise they'll be interpreted by the original shell, not the shell run by sbatch, and the original shell will

$ sbatch --wrap="for((i=0; i<3; i++)); do TMP=\`echo \"${X} \$i\"\`; echo \${TMP}; done"

BTW, in modern shell scripting it's preferable to use $() rather than backticks. It's easier to see, they nest properly (you need to use additional escaping to nest backticks) and readers are less likely to mistake them for quotes. You still need to escape it to make it run in the sbatch shell.

$ sbatch --wrap="for((i=0; i<3; i++)); do TMP=\$(echo \"${X} \$i\"); echo \${TMP}; done"

Passing a parameter into a slurm script

In order to avoid possible name collisions with shell and anvironment variables, it is a good habit to always use lowercase or mixed case variables in your Bash scripts.

You were almost there. You just need to pass the variable as an argument to the second script and then pick it up there based on the positional parameters. In this case, it looks like you're only passing one argument, so $1 is OK to use. In other cases, with multiple parameters of a fixed number you could also use $2,$3, etc. With a variable number of arguments "$@" would be more appropriate.

Shell Script (myshell.sh)

#!/bin/bash
for ((fnum=24; fnum<=30; fnum+=1))
do
sbatch myscript.slurm "$fnum"
done

Slurm script (myscript.slurm)

#!/bin/bash
#SBATCH --array=1-128

fnum=$1

...
matlab -nodesktop -nodisplay r "frame=[${slurm_array_task_ID}]; filename=[${fnum}]; myfunction(frame, filename);";

For handling various timeout conditions this might work:

A=$(sbatch --parsable a.slurm)

case $? in
9|64|130|131|137|140)
echo "some sort of timeout occurred"
B=$(sbatch --parsable --dependency=afternotok:$A a.slurm)
;;
*)
echo "some other exit condition occurred"
;;
esac

You will just need to decide what conditions you want to handle and how you want to handle them. I have listed all the ones that seem to involve timeouts.



Related Topics



Leave a reply



Submit