Trace of Executed Programs Called by a Bash Script

Trace of executed programs called by a Bash script

Since you say you can edit the script itself, simply put a:

ps -ef >/tmp/bash_stack_trace.$$

in it, where the problem is occurring.

This will create a number of files in your tmp directory that show the entire process list at the time it happened.

You can then work out which process called which other process by examining this output. This can either be done manually, or automated with something like awk, since the output is regular - you just use those PID and PPID columns to work out the relationships between all the processes you're interested in.

You'll need to keep an eye on the files, since you'll get one per process so they may have to be managed. Since this is something that should only be done during debugging, most of the time that line will be commented out (preceded by #), so the files won't be created.

To clean them up, you can simply do:

rm /tmp/bash_stack_trace.*

Show call stack in Bash

Run the script with bash -x, which will produce output on standard error to show each command that is executed.

how to trace a shell script running the background?

Though, @hek2mgl 's answer was great help and paved the way to solution, but the actual solution is a bit different:

Type fg in the terminal.

Then type Ctrl+z.

tracking bash calls

You may use strace to track your script's execution of other scripts or programs.

You will have to run your script like: strace -q -f -e execve yourscript.sh.

This will trace all calls made to other executables.

[root@devel ~]# ./x.sh 
x
y
z
[root@devel ~]# cat x.sh 
#!/bin/bash
echo x
./y.sh

[root@devel ~]# cat y.sh 
#!/bin/bash

echo y
./z.sh
[root@devel ~]# cat z.sh 
#!/bin/bash

echo z

[root@devel ~]# strace -q -f -e execve ./x.sh 
execve("./x.sh", ["./x.sh"], [/* 28 vars */]) = 0
x
[pid 19781] execve("./y.sh", ["./y.sh"], [/* 28 vars */]) = 0
y
[pid 19782] execve("./z.sh", ["./z.sh"], [/* 28 vars */]) = 0
z
[pid 19781] --- SIGCHLD (Child exited) @ 0 (0) ---
--- SIGCHLD (Child exited) @ 0 (0) —

It will trace even calls to perl or other executables.

[root@devel ~]# cat x.sh 
#!/bin/bash
echo x
./y.sh
ls >/dev/null 2>&1
[root@devel ~]# cat y.sh 
#!/bin/bash

echo y
perl -e 'print "z\n";'
[root@devel ~]# ./x.sh 
x
y
z
[root@devel ~]# strace -q -f -e execve ./x.sh 
execve("./x.sh", ["./x.sh"], [/* 28 vars */]) = 0
x
[pid 20300] execve("./y.sh", ["./y.sh"], [/* 28 vars */]) = 0
y
[pid 20301] execve("/usr/bin/perl", ["perl", "-e", "print \"z\\n\";"], [/* 28 vars */]) = 0
z
[pid 20300] --- SIGCHLD (Child exited) @ 0 (0) ---
--- SIGCHLD (Child exited) @ 0 (0) ---
[pid 20302] execve("/bin/ls", ["ls"], [/* 28 vars */]) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
[root@devel ~]# 

How can I debug a Bash script?

sh -x script [arg1 ...]
bash -x script [arg1 ...]

These give you a trace of what is being executed. (See also 'Clarification' near the bottom of the answer.)

Sometimes, you need to control the debugging within the script. In that case, as Cheeto reminded me, you can use:

set -x

This turns debugging on. You can then turn it off again with:

set +x

(You can find out the current tracing state by analyzing $-, the current flags, for x.)

Also, shells generally provide options '-n' for 'no execution' and '-v' for 'verbose' mode; you can use these in combination to see whether the shell thinks it could execute your script — occasionally useful if you have an unbalanced quote somewhere.

---

There is contention that the '-x' option in Bash is different from other shells (see the comments). The Bash Manual says:

• -x

  Print a trace of simple commands, for commands, case commands, select commands, and arithmetic for commands and their arguments or associated word lists after they are expanded and before they are executed. The value of the PS4 variable is expanded and the resultant value is printed before the command and its expanded arguments.

That much does not seem to indicate different behaviour at all. I don't see any other relevant references to '-x' in the manual. It does not describe differences in the startup sequence.

Clarification: On systems such as a typical Linux box, where '/bin/sh' is a symlink to '/bin/bash' (or wherever the Bash executable is found), the two command lines achieve the equivalent effect of running the script with execution trace on. On other systems (for example, Solaris, and some more modern variants of Linux), /bin/sh is not Bash, and the two command lines would give (slightly) different results. Most notably, '/bin/sh' would be confused by constructs in Bash that it does not recognize at all. (On Solaris, /bin/sh is a Bourne shell; on modern Linux, it is sometimes Dash — a smaller, more strictly POSIX-only shell.) When invoked by name like this, the 'shebang' line ('#!/bin/bash' vs '#!/bin/sh') at the start of the file has no effect on how the contents are interpreted.

The Bash manual has a section on Bash POSIX mode which, contrary to a long-standing but erroneous version of this answer (see also the comments below), does describe in extensive detail the difference between 'Bash invoked as sh' and 'Bash invoked as bash'.

When debugging a (Bash) shell script, it will be sensible and sane — necessary even — to use the shell named in the shebang line with the -x option. Otherwise, you may (will?) get different behaviour when debugging from when running the script.

How to echo shell commands as they are executed

set -x or set -o xtrace expands variables and prints a little + sign before the line.

set -v or set -o verbose does not expand the variables before printing.

Use set +x and set +v to turn off the above settings.

On the first line of the script, one can put #!/bin/sh -x (or -v) to have the same effect as set -x (or -v) later in the script.

The above also works with /bin/sh.

See the bash-hackers' wiki on set attributes, and on debugging.

$ cat shl
#!/bin/bash                                                                     

DIR=/tmp/so
ls $DIR

$ bash -x shl 
+ DIR=/tmp/so
+ ls /tmp/so
$

eclipse: how to debug a Scala program called from a shell script

I'm not sure if there is a way to debug both together, but what you can do is run your script with the option -xv. So...

user@mypc$: bash -xv myscript other_args

That will show you the commands that are executed along with their parameters.

Then in Eclipse you can debug your Scala program normally and pass those parameters to it through the main method or run configuration.

Typically debuggers are language specific and won't be able to do both bash scripts and code in another language, but with this method, you should be able to figure out what's going on.

---

Related Topics