How to Capture Unix 'Top' Command Output to a CSV File

How To Capture Unix 'Top' Command Output to a CSV file?

If you want to trim runs of whitespace and replace them with commas, try

top -b -n 3 | sed -n '8, 12{s/^ *//;s/ *$//;s/  */,/gp;};12q'

The sed script performs the following substitutions on lines 8 through 12:

• Replace any leading space with nothing (otherwise you get an empty first column when the PID is right-aligned).
• Replace any trailing spaces with nothing (similarly to avoid empty fields after the data).
• Replace any remaining runs of adjacent spaces with a comma. Print this line.

Finally, on line 12, we are done, so we quit sed.

The shell does not pay any attention to the name of the file you are redirecting into and generally, file extensions on Unix are informal decorations, not file type specifiers like they are on some platforms.

You could do echo hello >outputfile.ps and the output would still be text, not PostScript (or a number of other possible interpretations of the .ps file extension). In any event, the echo command does not know that it is being redirected, because that is handled by the shell before the command runs, anyway. (Well, echo is a shell built-in, so in theory there could be some coordination in this case.)

How to capture the output of a top command in a file in linux?

for me top -b > test.txt will store all output from top ok even if i break it with ctrl-c. I suggest you dump first, and then grep the resulting file.

send the unix output to a csv file

Try this :

printf '%s\n' A B C | paste -sd ' ' >> file.csv

or more classical for a CSV (delimiter with a , :

printf '%s\n' A B C | paste -sd ',' >> file.csv

printf '%s\n' A B C is just an example to have the same sample input as you. My solution works with spaces in a same line too.

EDIT from your comments, you seems to need to treat with a for loop, so :

for i in {0..5}; do printf '%s\n' {A..C} | paste -sd " " >> file.csv; done

or in pseudo code :

for ...:
    unix_command | paste -sd " " >> file.csv
endfor

How to store ps command output in csv format?

You could try something like the following code I wrote:

ps -e -o %p, -o lstart -o ,%C, -o %mem -o ,%c > output.csv

Brief explanation:

The -o option can be used multiple times in a ps command to specify the format.
In order to control which separator is used we can use AIX format descriptors. We can specify our needed separators like, e.g. %p,. Since AIX format descriptors are not available for every piece of data, but only for some of the data (for example in our case there are no AIX format descriptors for %mem and for lstart), we plant %mem and lstart around the available AIX format descriptors to achieve the comma separation. For example this site provides information about the ps command for further readings.

output.csv example:

  PID,                 STARTED,%CPU,%MEM,COMMAND
    1,Mon Feb 25 00:00:01 2019, 0.0, 0.1,examplecommand1
    2,Mon Feb 25 00:00:01 2019, 0.0, 0.0,examplecommand2
                           (...)

Output sar command results to a csv file

sar -d -u -w 1 1 | grep -v Average | grep -v Linux |awk '{if ($0 ~ /[0-9]/) { print $1","$2","$4","$5","$6; }  }'

22:14:04,CPU,%nice,%system,%iowait
22:14:05,all,0.00,8.53,0.00
22:14:04,proc/s,,,
22:14:05,0.00,,,
22:14:04,DEV,rd_sec/s,wr_sec/s,avgrq-sz
22:14:05,dev8-0,0.00,0.00,0.00

outputs above enjoy

How to write the output of a bash command to a csv file?

piping to awk '{$1=$1}1' OFS="," before tee should be enough to convert to csv. e.g:

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)/\1/g')" | awk '{$1=$1}1' OFS="," | tee -a log20201113.csv
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | awk '{$1=$1}1' OFS="," | tee -a log20201113.csv
    sleep 1
done

produces:

$ head -n 5 log20201113.csv
date,time,total,used,free,shared,buff/cache,available
2020-11-14,17:30:04,3638,1663,765,177,1208,1561
2020-11-14,17:30:05,3638,1663,765,177,1208,1561
2020-11-14,17:30:06,3638,1662,766,177,1208,1562
2020-11-14,17:30:07,3638,1662,767,177,1208,1562

when awk assigns to a field ($1=$1) it will

causes $0 to be reconstructed by concatenating the $i's separated by OFS

where $0 is the input line, $i are the fields and OFS is , in our case. the terminating 1 is just an "always match" patter that always gets executed with the default action (print).

but this will also print to stdout in csv format. if this is a problem and you would like to keep nicely formatted output on the terminal and csv in the file, you can use this "trick":

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)/\1/g')" | tee /dev/tty | awk '{$1=$1}1' OFS="," >> log20201113.csv
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | tee /dev/tty | awk '{$1=$1}1' OFS="," >> log20201113.csv
    sleep 1
done

View tabular file such as CSV from command line

You can also use this:

column -s, -t < somefile.csv | less -#2 -N -S

column is a standard unix program that is very convenient -- it finds the appropriate width of each column, and displays the text as a nicely formatted table.

Note: whenever you have empty fields, you need to put some kind of placeholder in it, otherwise the column gets merged with following columns. The following example demonstrates how to use sed to insert a placeholder:

$ cat data.csv
1,2,3,4,5
1,,,,5
$ sed 's/,,/, ,/g;s/,,/, ,/g' data.csv | column -s, -t
1  2  3  4  5
1           5
$ cat data.csv
1,2,3,4,5
1,,,,5
$ column -s, -t < data.csv
1  2  3  4  5
1  5
$ sed 's/,,/, ,/g;s/,,/, ,/g' data.csv | column -s, -t
1  2  3  4  5
1           5

Note that the substitution of ,, for , , is done twice. If you do it only once, 1,,,4 will become 1, ,,4 since the second comma is matched already.

---

Related Topics