How to remove only one newline between [greater than] and [less than] in unix
This might work for you (GNU sed):
sed ':a;N;s/>\n</></;ta;P;D' file
Read two lines into the pattern space and if the newline is between >
and <
remove it. Then print and delete the first line and repeat.
Replace multiple newlines with just 2 newlines using unix utilities
You can use awk like this:
awk 'BEGIN{RS="";ORS="\n\n"}1' file
RS
is the input record separator, ORS
is the output record separator.
From the awk manual:
If RS is null, then records are separated by sequences consisting of a newline plus one or more blank lines
That means that the above command splits the input text by two or more blank lines and concatenates them again with exactly two newlines.
How to replace \n string with a new line in Unix Bash script
No need for sed
, using parameter expansion:
$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"
1
2
3
With bash 4.4
or newer, you can use the E
operator in ${parameter@operator}
:
$ foo='1\n2\n3'; echo "${foo@E}"
1
2
3
How can I replace each newline (\n) with a space using sed?
Use this solution with GNU sed
:
sed ':a;N;$!ba;s/\n/ /g' file
This will read the whole file in a loop (':a;N;$!ba
), then replaces the newline(s) with a space (s/\n/ /g
). Additional substitutions can be simply appended if needed.
Explanation:
sed
starts by reading the first line excluding the newline into the pattern space.- Create a label via
:a
. - Append a newline and next line to the pattern space via
N
. - If we are before the last line, branch to the created label
$!ba
($!
means not to do it on the last line. This is necessary to avoid executingN
again, which would terminate the script if there is no more input!). - Finally the substitution replaces every newline with a space on the pattern space (which is the whole file).
Here is cross-platform compatible syntax which works with BSD and OS X's sed
(as per @Benjie comment):
sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g' file
As you can see, using sed
for this otherwise simple problem is problematic. For a simpler and adequate solution see this answer.
How can I replace multiple newlines(\n) between characters using sed?
Using sed:
$ sed 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file
{
{
doSomething();
}
}
If you want to change the file in place, use the -i
switch:
sed -i 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file # GNU sed
sed -i '' -e 'H;1h;$!d;x; s/}\n\n*}/}\n}/g' file # BSD sed
How it works
H;1h;$!d;x;
This is a sed idiom which reads the whole file in to the pattern space. (If your file is huge, we would want to look at other solutions.)
s/}\n\n*}/}\n}/g
This is just your substitute command with one change:
\n+
is changed to\n\n*
. This is because+
is not an active character under basic regular expressions. If we use extended regex, then braces,{
and}
, become active characters which has the potential to lead to other issues for this input.Note that this removes the extra newlines only if the second closing brace,
}
, occurs at the beginning of a line.
How to replace New line and ^M chars with ~~ in Unix
Try using sed
sed -i -e 's/^M//g' -e '/"$/!{:a N; s/\n/~~/; /"$/b; ba}' file
Note : To enter ^M
, type Ctrl+V followed by Ctrl+M
File content after running command
"id","notes"
"N001","this is~~test.~~~~Again test~~~~"
"N002","this is perfect"
"N00345","this is~~~~having ~~problem"
Or using dos2unix
followed by sed
dos2unix file
sed -i '/"$/!{:a N; s/\n/~~/; /"$/b; ba}' file
Short Description
Idea here is to remove newline character in each line not ending with "
sed -i ' # -i specifies in-place relace i.e. modifies file itself
/"$/!{ # if a line doesn't contain end pattern, " at the end of a line, then do following
:a # label 'a' for branching/looping
N; # append the next line of input into the pattern space
s/\n/~~/; # replace newline character '\n' with '~~' i.e. suppress new lines
/"$/b; # if a line contains end pattern then branch out i.e. break the loop
ba # branch to label 'a' i.e. this will create loop around label 'a'
}
' file # input file name
Refer to man sed
for further details
EDIT
Sometimes data in the cell itself contains " within it.
Using sed
sed -i ':a N; s/\n/~~/; $s/"~~"/"\n"/g; ba' file
File content after running command for updated case data
"id","notes"
"N001","this is~~test. "Again test."~~~~Again test~~~~"
"N002","this is perfect"
"N00345","this is~~~~having ~~problem as it contains "~~test"
Using perl
one-liner
perl -0777 -i -pe 's/\n/~~/g; s/"~~("|$)/"\n$1/g;' file
Unix replace new lines inside quotes
With GNU awk for multi-char RS and
1) the 3rd arg to match():
$ awk -v RS='^$' -v ORS= '{while ( match($0,/"[^"]+"/,a) ) {gsub(/\n/," ",a[0]); print substr($0,1,RSTART-1) a[0]; $0=substr($0,RSTART+RLENGTH)} print}' file
HEADER1, HEADER2, HEADER3, HEADER4
data1, data2, data3, "Text here with new line characters"
2) the 4th arg to split():
$ awk -v RS='^$' -v ORS= '{n=split($0,f,/"[^"]+"/,s); for (i=0; i<=n; i++) {gsub(/\n/," ",s[i]); print f[i] s[i]} }' file
HEADER1, HEADER2, HEADER3, HEADER4
data1, data2, data3, "Text here with new line characters"
Replace groups of characters by newline
You can use sed
like this to use \n
in replacement:
sed "s/', '/"$'\\\n'"/g" file
Here we are using $'\n'
to use a newline character in replacement. We ended up using ``$'\\n'due to use of double quotes around
sed` command.
As per man bash
:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard
or else with multiline sed
:
sed "s/', '/\\
/g" file
This will work on both gnu and POSIX sed versions on bash.
PS: If you're using gnu sed
then a simplified command would be:
sed "s/', '/\n/g" file
['siteed01pg|10.229.16.153|10.229.0.0|19|test / crt|BACKUP_MUT_SD Vlan981 (PVLAN 1981) New Backup Subnet #1 (site SD)
siteed01pg|10.129.135.53|10.129.135.0|26|test / crt|Fmer bopreprodback Vlan 754
[...]
Related Topics
How to Setup the Passwords-Less Authentication Between Two Different Accounts
The Gnu Screen Is Unresponsive, Seems Blocked
How to Run Vi on Docker Container
How to Set Nginx Max Open Files
How to Add My Own Software to a Buildroot Linux Package
Should I Put Trailing Slash After Source and Destination When Copy Folders
Installing Openssh on the Alpine Docker Container
How to Dump Part of Binary File
Multiple Ip Addresses on Google Cloud Compute
How to Perform a 'Git Pull' Without Re-Entering My Ssh Password
Is Usb Supported on Bash on Ubuntu on Windows 10
Given Two Directory Trees How to Find Which Files Are the Same
When to Use --Dynamic Option in Nm
Lsb_Release: Command Not Found in Latest Ubuntu Docker Container