Use Bash variable within SLURM sbatch script
This won't work. What happens when you run
sbatch myscript.sh
is that slurm parses the script for those special #SBATCH lines, generates a job record, stores the batch script somewhere. The batch script is executed only later when the job runs.
So you need to structure you workflow in a slightly different way, and first calculate the number of procs you need before submitting the job. Note that you can use something like
sbatch -n $numProcs myscript.sh
, you don't need to autogenerate the script (also, mpirun should be able to get the number of procs in your allocation automatically, no need to use "-np").
Import bash variables into slurm script
Okay, I've solved it:
sbatch --export=sampleID=$sampleID myscript.sh
bash variable expansion in slurm's #SBATCH directive
No, this isn't possible. But you can overwrite the scripts default --array
by giving it explicitly on the sbatch
command line.
run job script with different variables found in other files
This is the typical workload suited for a job array. With a submission script like this
#!/bin/bash
#SBATCH --job-name test
#SBATCH --ntasks 8
#SBATCH --time 04:00
#SBATCH --output out
#SBATCH --error err
#SBATCH --array=0-3
readarray -t VARS < var.txt
VAR=${VARS[$SLURM_ARRAY_TASK_ID]}
export VAR
Rscript script.R
and script.R
being
print(Sys.getenv("VAR"))
you will get a four-jobs job array, each one running the R script with a different value of the env var VAR
, taken from the var.txt
file.
Setting and passing a string variable in SLURM job script
Actually, the single quotes will be striped by Bash during the assignment.
❯ MYSTRING='blablabla'
❯ echo $MYSTRING
blablabla
You can try to escape the quotes like this:
❯ MYSTRING=\'blablabla\'
❯ echo $MYSTRING
'blablabla'
Otherwise, Matlab wrongly believe blablabla
is the name of a variable or a function and not a string literal. The error message is misleading though as it adds a single quote around blablabla
. But those single quotes are not the ones that you inserted in the MYSTRING=...
assignment, as shown here below in an interactive Matlab session.
$ matlab
MATLAB is selecting SOFTWARE OPENGL rendering.
< M A T L A B (R) >
Copyright 1984-2019 The MathWorks, Inc.
R2019a (9.6.0.1072779) 64-bit (glnxa64)
March 8, 2019
To get started, type doc.
For product information, visit www.mathworks.com.
>> blablabla
Undefined function or variable 'blablabla'.
Pass command line arguments via sbatch
The lines starting with #SBATCH are not interpreted by bash but are replaced with code by sbatch.
The sbatch options do not support $1 vars (only %j and some others, replacing $1 by %1 will not work).
When you don't have different sbatch processes running in parallel, you could try
#!/bin/bash
touch outFile${1}.txt errFile${1}.txt
rm link_out.sbatch link_err.sbatch 2>/dev/null # remove links from previous runs
ln -s outFile${1}.txt link_out.sbatch
ln -s errFile${1}.txt link_err.sbatch
#SBATCH -o link_out.sbatch
#SBATCH -e link_err.sbatch
hostname
# I do not know about the background processing of sbatch, are the jobs still running
# at this point? When they are, you can not delete the temporary symlinks yet.
exit 0
Alternative:
As you said in a comment yourself, you could make a masterscript.
This script can contain lines like
cat exampleJob.sh.template | sed -e 's/File.txt/File'$1'.txt/' > exampleJob.sh
# I do not know, is the following needed with sbatch?
chmod +x exampleJob.sh
In your template the #SBATCH lines look like
#SBATCH -o "outFile.txt"
#SBATCH -e "errFile.txt"
Understanding internal variable substitution with backticks in an sbatch --wrap statement
You need to escape the backticks, just like you escaped the quotes and the $
before internal variables. Otherwise they'll be interpreted by the original shell, not the shell run by sbatch
, and the original shell will
$ sbatch --wrap="for((i=0; i<3; i++)); do TMP=\`echo \"${X} \$i\"\`; echo \${TMP}; done"
BTW, in modern shell scripting it's preferable to use $()
rather than backticks. It's easier to see, they nest properly (you need to use additional escaping to nest backticks) and readers are less likely to mistake them for quotes. You still need to escape it to make it run in the sbatch
shell.
$ sbatch --wrap="for((i=0; i<3; i++)); do TMP=\$(echo \"${X} \$i\"); echo \${TMP}; done"
Passing a parameter into a slurm script
In order to avoid possible name collisions with shell and anvironment variables, it is a good habit to always use lowercase or mixed case variables in your Bash scripts.
You were almost there. You just need to pass the variable as an argument to the second script and then pick it up there based on the positional parameters. In this case, it looks like you're only passing one argument, so $1
is OK to use. In other cases, with multiple parameters of a fixed number you could also use $2
,$3
, etc. With a variable number of arguments "$@" would be more appropriate.
Shell Script (myshell.sh)
#!/bin/bash
for ((fnum=24; fnum<=30; fnum+=1))
do
sbatch myscript.slurm "$fnum"
done
Slurm script (myscript.slurm)
#!/bin/bash
#SBATCH --array=1-128
fnum=$1
...
matlab -nodesktop -nodisplay r "frame=[${slurm_array_task_ID}]; filename=[${fnum}]; myfunction(frame, filename);";
For handling various timeout conditions this might work:
A=$(sbatch --parsable a.slurm)
case $? in
9|64|130|131|137|140)
echo "some sort of timeout occurred"
B=$(sbatch --parsable --dependency=afternotok:$A a.slurm)
;;
*)
echo "some other exit condition occurred"
;;
esac
You will just need to decide what conditions you want to handle and how you want to handle them. I have listed all the ones that seem to involve timeouts.
Related Topics
How to Add Text After Last Pattern Match Using Ed
How to Sort Comma Separated Values in Bash
Installing Gnu Parallel Without Root Permission
Undefined Reference to 'Pthread_Init' When Using -Lpthread Flag:
Using Sed to Print Between Two Patterns
Will Data Written via Write() Be Flushed to Disk If a Process Is Killed
How to Monitor the Amount of Simd Instruction Usage
Reusing a Port Number in a Udp
Determine Os from a Single Command Line Operation
Stop on Newline When Using Read(...)
One-Liner: Print All Lines Except the Last 3
Which Capabilities Are Needed for Statx to Stop Giving Eperm
How to Scale Ejabberd Server MAChine on Centos to Handle 200 K Connections
How Does Gdb Start an Assembly Compiled Program and Step One Line at a Time
How to Display Filename from a Column Using Awk
Average of Multiple Files Without Considering Missing Values