Debugging Linux process hangs, which code is it running?
gdb comes with a program called gcore
, which will generate a core file from the running process.
Many systems nowadays disable core files by default (ulimit -c
in a shell will show 0). Use the ulimit -c unlimited
shell command, then run your process in the same shell (these limits are inherited from the parent process. If you start your process some other way than directly from the shell, you will need to find out how to set them there. e.g., LimitCORE=
in a systemd unit file).
Once your process gets into the bad state, run gcore
on its process ID. You can then copy it to your workstation and load it into gdb (gdb <executable> <core-file>
). You can then view the stack trace and other state as of the moment the core dump was taken.
How to Check if Linux File System is Hanging in Shell Script
How about checking if the filesystem is mounted instead of trying to cd
into it?
grep "folder_name" /proc/mounts
will return 0 if the folder exists, 1 otherwise
Other option would be check the filesystems on your box:cat /proc/filesystems
.
The content of this file is built in real time (/proc), so when it's read it reflects the actual current status
Linux/Unix command to determine if process is running?
While pidof
and pgrep
are great tools for determining what's running, they are both, unfortunately, unavailable on some operating systems. A definite fail safe would be to use the following: ps cax | grep command
The output on Gentoo Linux:
14484 ? S 0:00 apache2
14667 ? S 0:00 apache2
19620 ? Sl 0:00 apache2
21132 ? Ss 0:04 apache2
The output on OS X:
42582 ?? Z 0:00.00 (smbclient)
46529 ?? Z 0:00.00 (smbclient)
46539 ?? Z 0:00.00 (smbclient)
46547 ?? Z 0:00.00 (smbclient)
46586 ?? Z 0:00.00 (smbclient)
46594 ?? Z 0:00.00 (smbclient)
On both Linux and OS X, grep returns an exit code so it's easy to check if the process was found or not:
#!/bin/bash
ps cax | grep httpd > /dev/null
if [ $? -eq 0 ]; then
echo "Process is running."
else
echo "Process is not running."
fi
Furthermore, if you would like the list of PIDs, you could easily grep for those as well:
ps cax | grep httpd | grep -o '^[ ]*[0-9]*'
Whose output is the same on Linux and OS X:
3519 3521 3523 3524
The output of the following is an empty string, making this approach safe for processes that are not running:
echo ps cax | grep aasdfasdf | grep -o '^[ ]*[0-9]*'
This approach is suitable for writing a simple empty string test, then even iterating through the discovered PIDs.
#!/bin/bash
PROCESS=$1
PIDS=`ps cax | grep $PROCESS | grep -o '^[ ]*[0-9]*'`
if [ -z "$PIDS" ]; then
echo "Process not running." 1>&2
exit 1
else
for PID in $PIDS; do
echo $PID
done
fi
You can test it by saving it to a file (named "running") with execute permissions (chmod +x running) and executing it with a parameter: ./running "httpd"
#!/bin/bash
ps cax | grep httpd
if [ $? -eq 0 ]; then
echo "Process is running."
else
echo "Process is not running."
fi
WARNING!!!
Please keep in mind that you're simply parsing the output of ps ax
which means that, as seen in the Linux output, it is not simply matching on processes, but also the arguments passed to that program. I highly recommend being as specific as possible when using this method (e.g. ./running "mysql"
will also match 'mysqld' processes). I highly recommend using which
to check against a full path where possible.
References:
http://linux.about.com/od/commands/l/blcmdl1_ps.htm
http://linux.about.com/od/commands/l/blcmdl1_grep.htm
Related Topics
How to Make Find and Printf Works in Bash Script
Getting Stacktrace from Core Dump
In Screen, How to Send a Command to All Virtual Terminal Windows Within a Single Screen Session
How to Pass Argument in Expect Through the Command Line in a Shell Script
Keep Remote Directory Up-To-Date
How to Set Nginx Max Open Files
Are Tar.Gz and Tgz the Same Thing
How to Copy Text from My Xterm Without a Mouse
Check Whether a Certain File Type/Extension Exists in Directory
Linux: Which Process Is Causing "Device Busy" When Doing Umount
Shell Script to Count Files, Then Remove Oldest Files
How to Call Curl Without Using Server-Side Cache
How to Set the Grep After Context to Be "Until the Next Blank Line"
Linux: Set Permission Only to Directories
How to Make Vim Play Typewriter Sound When I Write a Letter
How to Find a File/Directory That Could Be Anywhere on Linux Command Line