Delete Empty Lines Using Sed

Delete empty lines using sed

You may have spaces or tabs in your "empty" line. Use POSIX classes with sed to remove all lines containing only whitespace:

sed '/^[[:space:]]*$/d'

A shorter version that uses ERE, for example with gnu sed:

sed -r '/^\s*$/d'

(Note that sed does NOT support PCRE.)

Remove blank lines in a file using sed

Use the following sed to delete all blank lines.

sed '/./!d' cou.data

Explanation:

/./ matches any character, including a newline.
! negates the selector, i.e. it makes the command apply to lines which do not match the selector, which in this case is the empty line(s).
d deletes the selected line(s).
cou.data is the path to the input file.

Where did you go wrong?

The following excerpt from How sed Works states:

sed operates by performing the following cycle on each line of input: first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed; each command can have an address associated to it: addresses are a kind of condition code, and a command is only executed if the condition is verified before the command is to be executed.
When the end of the script is reached, unless the -n option is in use, the contents of pattern space are printed out to the output stream, adding back the trailing newline if it was removed.8 Then the next cycle starts for the next input line.

I've intentionally emboldened the parts which are pertinent to why your sed examples are not working. Given your examples:

They seem to disregard that sed reads one line at a time.
The trailing newlines, (\n\n and \n\n\n in your first and second example respectively), which you're trying to match don't actually exist. They've been removed by the time your regexp pattern is executed and then reinstated when the end of the script is reached.

UNIX: using tr to delete empty lines

tr on linux, at least, can squeeze repeated characters:

echo -ne $a
the quick
brown fox

jumps over
echo -ne $a |tr -s '\n'
the quick
brown fox
jumps over

Remove empty line before a pattern using sed

You'll need to add m flag for Test 1, so that ^ and $ anchors will match every line's start and end locations, otherwise they'll match start/end of entire string. This assumes m flag is supported by your implementation, like GNU sed does.

sed ':r;$!{N;br};s/^\n\(# \[END\)/\1/mg'

Test 3 works because there's a newline just before the empty line as part of that previous line. The below example might help you better visualize it:

$ printf 'a\nb\nc\n'
a
b
c
$ printf 'a\nb\n\nc\n'
a
b

c

With perl:

perl -0777 -pe 's/\n\K\n(?=# \[END)//g'

-0777 will slurp the entire input as single string
\n\K\n(?=# \[END) will match a newline provided there's a newline character before and # \[END after that newline

Another alternative with GNU sed, doesn't need to read whole file in one go.

sed '/^$/{N; s/\n\(# \[END\)/\1/; P; D}'

/^$/ will match an empty line
- N add next line to pattern space
- s/\n$# \[END$/\1/ remove the newline if required regexp matches

P and D are crucial here, so I'll quote the manual:

P Print out the portion of the pattern space up to the first newline.

D If pattern space contains no newline, start a normal new cycle as if the d command was issued. Otherwise, delete text in the pattern space up to the first newline, and restart cycle with the resultant pattern space, without reading a new line of input.

Delete empty lines from a text file via Bash including empty spaces characters

Use character class [:blank:] to indicate space or tab:

With sed:

sed -i '/^[[:blank:]]*$/ d' file.txt

With perl:

perl -ne 'print if !/^[[:blank:]]*$/' file.txt

With awk:

awk '!/^[[:blank:]]*$/' file.txt

With grep:

grep -v '^[[:blank:]]*$' file.txt

If the tool does not support editing in-place, leverage a temporary file e.g. for grep:

grep -v '^[[:blank:]]*$' file.txt >file.txt.tmp && mv file.txt{.tmp,}

removing empty lines from a pipe using sed

The problem is the -Z flag of grep. It causes grep to terminate matched lines with a null character instead of a newline. This won't work well with sed, because sed processes input line by line, and it expects each line to be terminated by a newline. In your example grep doesn't emit any newline characters, so as far as sed is concerned, it receives a block of text without a terminating newline, so it processes it as a single line, and so the pattern /^\s*$/ doesn't match anything.

Furthermore, the -z flag would only make sense if the filenames in the output of find were terminated by null characters, that is with the -print0 flag. But you're not using that, so the -z flag in grep is pointless. And the -Z flag is pointless too, because that should be used when the next command in the pipeline expects null-terminated records, which is not your case.

Do like this:

find ~/AppData/Local/atom/ -name atom.sh -type f -print0 | grep -zF 'cli/atom.sh' | tr '\0' '\n' | tail -n 1

SED command to delete empty lines till the first occurrence of sentence

If the lines are really empty (no whitespace), I would suggest

sed -n '/./,$p', otherwise sed -n $'/[^ \t]/,$p'. (The $'..' syntax makes bash expand the \t, so you don't need a sed that understands it.)

sed + remove # and empty lines with one sed command

If you're worried about starting two sed processes in a pipeline for performance reasons, you probably shouldn't be, it's still very efficient. But based on your comment that you want to do in-place editing, you can still do that with distinct commands (sed commands rather than invocations of sed itself).

You can either use multiple -e arguments or separate commands with a semicolon, something like (just one of these, not both):

sed -i 's/#.*$//' -e '/^$/d' fileName
sed -i 's/#.*$//;/^$/d' fileName

The following transcript shows this in action:

pax> printf 'Line # with a comment\n\n# Line with only a comment\n' >file

pax> cat file
Line # with a comment

# Line with only a comment

pax> cp file filex ; sed -i 's/#.*$//;/^$/d' filex ; cat filex
Line

pax> cp file filex ; sed -i -e 's/#.*$//' -e '/^$/d' filex ; cat filex
Line

Note how the file is modified in-place even with two -e options. You can see that both commands are executed on each line. The line with a comment first has the comment removed then all is removed because it's empty.

In addition, the original empty line is also removed.

How to remove empty duplicate lines with sed?

Something along:

sed 'N;/^ *\n *$/!P;D'

Small explanation:

N - add next line to pattern space
/^ *\n *$/ - we have two lines in pattern space, if they are empty, there will only be newline with spaces. So match from beginning ^ of the pattern, * match zero or more spaces only a newline, a newline \n, then * zero or more space till the end of pattern space $
! - if the regex does not match
P - print the pattern space up until the newline
D - delete the pattern space up until newline and start over

Tested with:

$ printf "   %s   \n" "A" "" "" "" "B" "C" "D" "" "" "" "" "E" | sed 'N;/^ *\n *$/!P;D'
   A   

   B   
   C   
   D   

   E

I see the last line is not printed with my GNU sed with --posix, I was able to fix it with $q on the end, I guess what N does differs between posix and gnu sed (it does not print pattern space in posix):

sed --posix '$q;N;/^ *\n *$/!P;D;'

Delete Empty Lines Using Sed