Merge Multiple Lines to 1 Row with Awk(Or Familiar)

Merge multiple lines to 1 row with awk(or familiar)

Why not this:

awk '$0~"Nmap"{if(output!="")print output;print;output=""}$0!~"Nmap"{output=output""$0" "}END{if(output!="")print output}'

If $0 has "Nmap", print that line. If not, keep track of what is happening, then print it when a new Nmap.

The END{} block takes care of firing what was not printed. Note that you are looking for Nmap to print something out. Also note the condition $0!~"Nmap" to avoid buffering that first line as well.

How to merge two or more lines if they start with the same word?

Since this resembles SQL like group operations, you can use sqlite which is available in bash

with the given inputs

$ cat aqua.txt
AAKRKA HIST1H1B AAGAGAAKRKATGPP
AAKRKA HIST1H1E RKSAGAAKRKASGPP
AAKRLN ACAT1 LMTADAAKRLNVTPL
AAKRLN SUCLG2 NEALEAAKRLNAKEI
AAKRLR GTF2F1 VSEMPAAKRLRLDTG
AAKRMA VCL NDIIAAAKRMALLMA
AAKRPL WIZ YLGSVAAKRPLQEDR
AAKRQK MTA2 SSSQPAAKRQKLNPA

$

Script:

$ cat ./sqlite_join.sh
#!/bin/sh
sqlite3 << EOF
create table data(a,b,c);
.separator ' '
.import $1 data
select a, group_concat(b) , group_concat(c) from data group by a;
EOF

$ 

Results

$ ./sqlite_join.sh aqua.txt
AAKRKA HIST1H1B,HIST1H1E AAGAGAAKRKATGPP,RKSAGAAKRKASGPP
AAKRLN ACAT1,SUCLG2 LMTADAAKRLNVTPL,NEALEAAKRLNAKEI
AAKRLR GTF2F1 VSEMPAAKRLRLDTG
AAKRMA VCL NDIIAAAKRMALLMA
AAKRPL WIZ YLGSVAAKRPLQEDR
AAKRQK MTA2 SSSQPAAKRQKLNPA

$

Use multiple lines with Awk

You can use Perl:

use strict;
use warnings;

my %hash=();

while (<DATA>) {
    if (m/^([0-9_]+)_(?:[^_]+)_(.*?)\s*$/) {
        push @{ $hash{join(' ', split('_', $1) )} }, $2;    
    }
}

print "$_ ". join(' ', @{ $hash{$_} })."\n" for (keys %hash);

__DATA__
101_1_1_trialOutcome_found_berry            
101_1_1_trialStartTime_2014-08-05 11:26:49.510000           
101_1_1_trialOutcomeTime_2014-08-05 11:27:00.318000         
101_1_1_trialResponseTime_0:00:05.804000            
101_1_1_whichResponse_d         
101_1_1_bearPosition_6          
101_1_1_patch_9         
101_1_1_food_11 

Prints:

101 1 1 found_berry 2014-08-05 11:26:49.510000 2014-08-05 11:27:00.318000 0:00:05.804000 d 6 9 11

Or, perl one line version:

$ perl -lane '
> push @{ $hash{join(" ", split("_", $1) )} }, $2 if (m/^([0-9_]+)_(?:[^_]+)_(.*?)\s*$/);
> END { print "$_ ". join(" ", @{ $hash{$_}})."\n" for (keys %hash); }
> ' file.txt
101 1 1 found_berry 2014-08-05 11:26:49.510000 2014-08-05 11:27:00.318000 0:00:05.804000 d 6 9 11

How do I match a pattern and then copy multiple lines?

Your existing awk matches correctly the rows from the ids' file, you now need to add a condition to print N lines ahead after reading the last field of the matching row. So we will set a variable p to the number of lines to print plus one (the current one), and decrease per row printing.

awk -F'/' 'NR==FNR{id[$0]; next} $1 in id{p=$6+1} p-->0{print}' file1 file2

or the same with last condition more "awkish" (by Ed Morton) and covering any possible extreme case of a huge file

awk -F'/' 'NR==FNR{id[$0]; next} $1 in id{p=$6+1} p&&p--' file1 file2

here the print condition is omitted, as it is the default action, and the condition is true again as long as decreasing p is positive.

Faster way to merge multiple files with unequal number of rows by column in bash

You could put all in one awk script:

awk -F'|' '{if (NR==FNR) a[$1]=$2; else print $1 "|" a[$1] " " $2}' a.txt b.txt 

001|johan chu
001|johan stewart
002|mike lewis
002|mike jordan
003|adam lambert
003|adam johnson
003|adam smith
003|adam long

Merge two data sets using AWK

Read your values from file01 into one or more arrays. You can use getline in the BEGIN block or the canonical way is to use a FNR == NR loop as one of the main blocks.

FNR == NR {array[$1] = $1; ...; next } # read file01 into some arrays
{ for (item in array) { ... }     # process each item in the array(s) against each line in file02

Your script would be invoked as awk '...' file01 file02

Instead of indexing the arrays by field values, you could index them with a counter array1[c] = $1; array2[c] = $2; c++ and iterate with a counter instead of using in: for (i=0; i<c; i++).

Of course, you should choose meaningful array names.

how to concatenate lines into one string

Use paste

$ echo -e 'one\ntwo\nthree' | paste -s -d':'
one:two:three

Concatenate 2 columns within a single .txt file and separate by : using awk?

Here you go:

awk '{print $1":"$3"\t"$2}' < infile.txt 

This will print Column $1:Column $3 followed by a tab character "\t" and then Column $2.

You can save the output to a file by adding > outfile.txt at the end of the command.

You can find some more examples of simple awk syntax here: http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples/

---

Related Topics