Write to a File After Piping Output from Tail -F Through to Grep

Filter followed tail to file using grep and redirect

Can you please try:

tail -f /var/log/apache2/error.log | grep "trace1" | tee -a output.txt

No output while piping tail through grep

grep is not line-buffered when its stdout isn't connected to a tty. So it's trying to process a block (usually 4 KiB or 8 KiB or so) before generating some output.

You need to tell grep to buffer its output by line. If you're using GNU grep, this works:

done < <(tail -F -n +1 "$filepath" | grep '^!' --line-buffered)
                                               ^^^^^^^^^^^^^^^

tail -f | sed to file doesn't work

It's the sed buffering. Use sed -u. man sed:

-u, --unbuffered

          load  minimal amounts of data from the input files and flush the
          output buffers more often

And here's a test for it (creates files fooand bar):

$ for i in {1..3} ; do echo a $i ; sleep 1; done >> foo &
[1] 12218
$ tail -f foo | sed -u 's/a/b/' | tee -a bar
b 1
b 2
b 3

Be quick or increase the {1..3} to suit your skillz.

How to grep output from tail and include file names?

You could use a simple regex that matches;

• All the filename's; ==>(.*)<==
• Your 'search' string (eg: foo)

tail -n +1 * | grep -E '==>(.*)<==|foo'

---

Anther option, based on my original answer by running the -exec through grep like so:

find . -type f -print -exec grep 'foo'  {} \;

---

Example using both methods in 3 files ({a,b,c}.txt) all containing a, b, and c.

bash-3.2$ tail -n +1 *
==> a.txt <==
a
b
c

==> b.txt <==
a
b
c

==> c.txt <==
a
b
c

bash-3.2$ tail -n +1 * | grep -E '==>(.*)<==|c'
==> a.txt <==
c
==> b.txt <==
c
==> c.txt <==
c

bash-3.2$ find . -type f -print -exec grep 'c' {} \;
./c.txt
c
./b.txt
c
./a.txt
c

tail -f pipe not only the single line

The error shows the problem is other place, which you don't show to us.
Below test code is fine for me.

tail -1f logfile.log | while read line
do 
   echo /usr/bin/gammu --sendsms TEXT ****** -text "$line"
done

Linux bash: grep from stream and write to file

If just grep, without writing to the file, works, you encountered a buffering "problem". I/O buffering, unless manually implemented by the program will get handled by the libc. If the program's stdout is a termial, buffering will be line-based. If not, the libc buffers output until the buffer reached a size limit.

On Linux, meaning with glibc you can use the stdbuf command to configure that buffering:

tail -f A.log | stdbuf -oL grep "keyword" >> B.log

-oL specifies that the output stream should be line-buffered.

---

Related Topics