Sort Logfile by Timestamp on Linux Command Line

sort logfile by timestamp on linux command line

Use sort's -k flag:

sort -k1 -r freeswitch.log

That will sort the file, in reverse, by the first key (i.e. freeswitch.log:2011-09-08 12:21:07.282236). If the filename is always the same (freeswitch.log), then it should sort by the date.

Sort logs by date field in bash

For GNU sort: sort -k2M -k3n -k4

• -k2M sorts by second column by month (this way "March" comes before "April")
• -k3n sorts by third column in numeric mode (so that " 9" comes before "10")
• -k4 sorts by the fourth column.

See more details in the manual.

How to Sort Output from several Log Files by date

Thank you all.

I improved script from Dennis Williamson to sort errors by date. Each log file with error inside is saved in file named by the timestamp of last error occured. These files are later sorted and put together. There may be cleaner solutions for that than to use of temp files.

find log/ -iname "*debug*.log" -size +0 | while read -r file
do
    if grep -qsm 1 'ERROR' "$file"
    then
        echo -e "$i \t$file"
        errors=$(grep 'ERROR' --color=auto -C 5 "$file")
        #get the timestamp of last error occured
        time=$(echo $errors | head -n 1 | awk '{print $1" "$2}')
        timestamp=$(date -d "$time" +%s)
        #save it to temp file
        echo -e "\n$file\n$errors" > tmp/logs/$timestamp.$i
    fi
    let i++
done

#put files together
rm -f output.txt
for i in `ls tmp/logs/*|sort`;do cat $i >> output.txt ; rm  $i; done

Opinions and suggestions for improvement appreciated!

Sort lines by timestamp in Emacs

You can apply any bash command to a selected region:

Please follow these steps:

1. select your region (or the whole buffer with C-x h)

2. type C-u M-|, Emacs will prompt you for your shell command, enter sort -k1 -r for instance

And that's it! (attention, it is M-| and not M-!)

Extract from Emacs doc (you can get it with C-h k M-|):

M-| runs the command shell-command-on-region (found in global-map),
which is an interactive compiled Lisp function in ‘simple.el’.

It is bound to M-|, .

(shell-command-on-region START END COMMAND &optional OUTPUT-BUFFER

REPLACE ERROR-BUFFER DISPLAY-ERROR-BUFFER REGION-NONCONTIGUOUS-P)

Execute string COMMAND in inferior shell with region as input.
Normally display output (if any) in temp buffer ‘Shell Command
Output’; Prefix arg means replace the region with it. Return the
exit code of COMMAND.

In other terms M-| runs the shell command and displays output in the ‘Shell Command Output’ buffer. If you want this output to replace the selected region you must prefix the M-| command, this is our C-u command (step 2).

---

Answering to @Toby Speight comment here is an example. I use as initial buffer these lines (your question provided link)

freeswitch.log:2011-09-08 12:21:07.282236 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda3525c0 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!
freeswitch.log:2011-08-08 13:21:07.514261 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda354460 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!
freeswitch.log:2011-06-04 16:21:08.998227 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda356300 in queue 0x7f2ce8005990, no more room! windex == rindex == 58! 
freeswitch.log:2011-09-08 12:21:10.374238 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda3581a0 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!

I type: C-x h C-u M-| sort -k1 -r <RET> (where <RET> stands for the "return" keyboard key) as result my buffer contains now:

freeswitch.log:2011-09-08 12:21:10.374238 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda3581a0 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!
freeswitch.log:2011-09-08 12:21:07.282236 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda3525c0 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!
freeswitch.log:2011-08-08 13:21:07.514261 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda354460 in queue 0x7f2ce8005990, no more room! windex == rindex == 58!
freeswitch.log:2011-06-04 16:21:08.998227 [ERR] ftdm_queue.c:136 Failed to enqueue obj 0x7f2cda356300 in queue 0x7f2ce8005990, no more room! windex == rindex == 58! 

How to merge log files and sort by time

Ref: Merging multiple log files by date including multilines

As mentioned in the above question, if you are certain that all the log lines start with timestamp, you can do:

cat logA.log logB.log | sort -n 

This would not work when there are other lines such as stack trace which do not start with timestamp.

I think you can check out the above question and answers if your considering a similar scenario.

Linux grep and sort log files

Try this:

grep --color=always "myID" file*.log | sort -t : -k2,2 -k3,3n -k4,4n

Output:

file3.log:2015-09-26 15:39:48,788 - ERROR - bar : {'id' : myID}
file1.log:2015-09-26 15:39:50,788 - DEBUG - blabla : {'id' : myID}
file2.log:2015-09-26 15:39:51,788 - ERROR - foo : {'id' : myID}

How can I sort an Apache log file by date?

#!/bin/sh
if [ ! -f $1 ]; then
    echo "Usage: $0 "
    exit
fi
echo "Sorting $1"
sort -t ' ' -k 4.9,4.12n -k 4.5,4.7M -k 4.2,4.3n -k 4.14,4.15n -k 4.17,4.18n -k 4.20,4.21n $1 > $2

---

Related Topics