How to Replace Just One Newline Between > and < in Unix

How to remove only one newline between [greater than] and [less than] in unix

This might work for you (GNU sed):

sed ':a;N;s/>\n</></;ta;P;D' file

Read two lines into the pattern space and if the newline is between > and < remove it. Then print and delete the first line and repeat.

Replace multiple newlines with just 2 newlines using unix utilities

You can use awk like this:

awk 'BEGIN{RS="";ORS="\n\n"}1' file

RS is the input record separator, ORS is the output record separator.

From the awk manual:

If RS is null, then records are separated by sequences consisting of a newline plus one or more blank lines

That means that the above command splits the input text by two or more blank lines and concatenates them again with exactly two newlines.

How to replace \n string with a new line in Unix Bash script

No need for sed, using parameter expansion:

$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"  
1
2
3

With bash 4.4 or newer, you can use the E operator in ${parameter@operator}:

$ foo='1\n2\n3'; echo "${foo@E}"
1
2
3

How can I replace each newline (\n) with a space using sed?

Use this solution with GNU sed:

sed ':a;N;$!ba;s/\n/ /g' file

This will read the whole file in a loop (':a;N;$!ba), then replaces the newline(s) with a space (s/\n/ /g). Additional substitutions can be simply appended if needed.

Explanation:


  1. sed starts by reading the first line excluding the newline into the pattern space.
  2. Create a label via :a.
  3. Append a newline and next line to the pattern space via N.
  4. If we are before the last line, branch to the created label $!ba ($! means not to do it on the last line. This is necessary to avoid executing N again, which would terminate the script if there is no more input!).
  5. Finally the substitution replaces every newline with a space on the pattern space (which is the whole file).

Here is cross-platform compatible syntax which works with BSD and OS X's sed (as per @Benjie comment):

sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g' file

As you can see, using sed for this otherwise simple problem is problematic. For a simpler and adequate solution see this answer.

How can I replace multiple newlines(\n) between characters using sed?

Using sed:

$ sed 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file
{
{
doSomething();
}
}

If you want to change the file in place, use the -i switch:

sed -i 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file  # GNU sed

sed -i '' -e 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file # BSD sed

How it works

  • H;1h;$!d;x;

    This is a sed idiom which reads the whole file in to the pattern space. (If your file is huge, we would want to look at other solutions.)

  • s/}\n\n*}/}\n}/g

    This is just your substitute command with one change: \n+ is changed to \n\n*. This is because + is not an active character under basic regular expressions. If we use extended regex, then braces, { and }, become active characters which has the potential to lead to other issues for this input.

    Note that this removes the extra newlines only if the second closing brace, }, occurs at the beginning of a line.

How to replace New line and ^M chars with ~~ in Unix

Try using sed

sed -i -e 's/^M//g' -e '/"$/!{:a N; s/\n/~~/; /"$/b; ba}' file

Note : To enter ^M, type Ctrl+V followed by Ctrl+M

File content after running command

"id","notes"
"N001","this is~~test.~~~~Again test~~~~"
"N002","this is perfect"
"N00345","this is~~~~having ~~problem"

Or using dos2unix followed by sed

dos2unix file
sed -i '/"$/!{:a N; s/\n/~~/; /"$/b; ba}' file

Short Description

Idea here is to remove newline character in each line not ending with "

sed -i '              # -i specifies in-place relace i.e. modifies file itself
/"$/!{ # if a line doesn't contain end pattern, " at the end of a line, then do following
:a # label 'a' for branching/looping
N; # append the next line of input into the pattern space
s/\n/~~/; # replace newline character '\n' with '~~' i.e. suppress new lines
/"$/b; # if a line contains end pattern then branch out i.e. break the loop
ba # branch to label 'a' i.e. this will create loop around label 'a'
}
' file # input file name

Refer to man sed for further details


EDIT

Sometimes data in the cell itself contains " within it.

Using sed

sed -i ':a N; s/\n/~~/; $s/"~~"/"\n"/g; ba' file

File content after running command for updated case data

"id","notes"
"N001","this is~~test. "Again test."~~~~Again test~~~~"
"N002","this is perfect"
"N00345","this is~~~~having ~~problem as it contains "~~test"

Using perl one-liner

perl -0777 -i -pe 's/\n/~~/g; s/"~~("|$)/"\n$1/g;' file

Unix replace new lines inside quotes

With GNU awk for multi-char RS and

1) the 3rd arg to match():

$ awk -v RS='^$' -v ORS= '{while ( match($0,/"[^"]+"/,a) ) {gsub(/\n/," ",a[0]); print substr($0,1,RSTART-1) a[0]; $0=substr($0,RSTART+RLENGTH)} print}' file
HEADER1, HEADER2, HEADER3, HEADER4
data1, data2, data3, "Text here with new line characters"

2) the 4th arg to split():

$ awk -v RS='^$' -v ORS= '{n=split($0,f,/"[^"]+"/,s); for (i=0; i<=n; i++) {gsub(/\n/," ",s[i]); print f[i] s[i]} }' file
HEADER1, HEADER2, HEADER3, HEADER4
data1, data2, data3, "Text here with new line characters"

Replace groups of characters by newline

You can use sed like this to use \n in replacement:

sed "s/', '/"$'\\\n'"/g" file

Here we are using $'\n' to use a newline character in replacement. We ended up using ``$'\\n'due to use of double quotes aroundsed` command.

As per man bash:

Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard

or else with multiline sed:

sed "s/', '/\\
/g" file

This will work on both gnu and POSIX sed versions on bash.

PS: If you're using gnu sed then a simplified command would be:

sed "s/', '/\n/g" file
['siteed01pg|10.229.16.153|10.229.0.0|19|test / crt|BACKUP_MUT_SD  Vlan981 (PVLAN 1981)  New Backup Subnet #1  (site SD)
siteed01pg|10.129.135.53|10.129.135.0|26|test / crt|Fmer bopreprodback Vlan 754

[...]


Related Topics



Leave a reply



Submit