Add Up a Column of Numbers at the Unix Shell

Add up a column of numbers at the Unix shell

... | paste -sd+ - | bc

is the shortest one I've found (from the UNIX Command Line blog).

Edit: added the - argument for portability, thanks @Dogbert and @Owen.

Bash command to sum a column of numbers

Using existing file:

paste -sd+ infile | bc

Using stdin:

<cmd> | paste -sd+ | bc

Edit:
With some paste implementations you need to be more explicit when reading from stdin:

<cmd> | paste -sd+ - | bc

Options used:

-s (serial) - merges all the lines into a single line

-d - use a non-default delimiter (the character + in this case)

Shell command to sum integers, one per line?

Bit of awk should do it?

awk '{s+=$1} END {print s}' mydatafile

Note: some versions of awk have some odd behaviours if you are going to be adding anything exceeding 2^31 (2147483647). See comments for more background. One suggestion is to use printf rather than print:

awk '{s+=$1} END {printf "%.0f", s}' mydatafile

Shortest command to calculate the sum of a column of output on Unix?

ipcs -mb | tail +4 | awk '{ sum += $7 } END { print sum }'

Or without tail:

ipcs -mb | awk 'NR > 3 { sum += $7 } END { print sum }'

Using awk with bc to have arbitrary long results (credits to Jouni K.):

ipcs -mb | awk 'NR > 3 { print $7 }' | paste -sd+ | bc

SUM up all values of each row and write the results in a new column using Bash

To add a new column, just increment number of columns and assign the new column a value:

NF++; $NF=sum

do:

awk -v OFS=, -F, 'NR>1{sum=0; for(i=1; i<=NF; i++) sum += $i; NF++; $NF=sum } 1'

Shell script to sum columns associated with a name

awk '{ map[$2]+=$1 } END { for (i in map) { print i" "map[i] } }' file

Using awk, create an array with the name as the first index and a running total of the values for each name. At the end, print the names and totals.

Shell Script add column values

Assuming consistent use of double quote characters, you can use:

awk -F\" '{s += $8} END{print "sum=" s+0}' inputFile

This will generate:

sum=10

This works because a quote delimiter gives you the fields:

1 2      3 4      5 6     7 8 ...
{"userId":"f1fcab","count":"3","type":"Stack"}

How can I quickly sum all numbers in a file?

For a Perl one-liner, it's basically the same thing as the awk solution in Ayman Hourieh's answer:

 % perl -nle '$sum += $_ } END { print $sum'

If you're curious what Perl one-liners do, you can deparse them:

 %  perl -MO=Deparse -nle '$sum += $_ } END { print $sum'

The result is a more verbose version of the program, in a form that no one would ever write on their own:

BEGIN { $/ = "\n"; $\ = "\n"; }
LINE: while (defined($_ = <ARGV>)) {
    chomp $_;
    $sum += $_;
}
sub END {
    print $sum;
}
-e syntax OK

Just for giggles, I tried this with a file containing 1,000,000 numbers (in the range 0 - 9,999). On my Mac Pro, it returns virtually instantaneously. That's too bad, because I was hoping using mmap would be really fast, but it's just the same time:

use 5.010;
use File::Map qw(map_file);

map_file my $map, $ARGV[0];

$sum += $1 while $map =~ m/(\d+)/g;

say $sum;

Shell command to sum up numbers across similar lines of text in a file

There is no need for multiple processes and pipes. awk alone is more than capable of handling the entire job (and will be orders of magnitude faster on large files). With awk simply append each of the fields 2-NF as a string and use that as an index to sum the numbers in field 1 in an array. Then in the END section, simply output the contents of the array, e.g. presuming your data is stored in file, you could do:

awk '{
    for (i=2; i<=NF; i++)
        str = str " " $i
    a[str] += $1
    str=""
}
END {
    for (i in a) print a[i], i
}' file

Above, the first for loop simply appends all fields from 2-NF in str, a[str] += $1 sums the values in field 1 into array a using str as an index. That ensures the values for similar lines are summed. In the END section, you simply loop over each element of the array outputting the element value (the sum) and then the index (original str for fields 2-NF).

Example Use/Output

Just take what is above, select it, and then middle-mouse paste it into a command line in the directory where your file is located (change the name of file to your data file name)

$ awk '{
>     for (i=2; i<=NF; i++)
>         str = str " " $i
>     a[str] += $1
>     str=""
> }
> END {
>     for (i in a) print a[i], i
> }' file
30  take a test
37  cup of coffee
75  sign on the dotted

If you want the lines sorted in a different order, just add | sort [options] after the filename to pipe the output to sort. For example for output in the order you show, you would use | sort -k 2 and the output would be:

37  cup of coffee
75  sign on the dotted
30  take a test

Preserving Original Order Of Strings

Pursuant to your comment regarding how to preserve the original order of the lines of text seen in your input file, you can keep a second array where the strings are stored in the order they are seen using a sequential index to keep them in order. For example the o array (order array) is used below to store the unique string (fields 2-NF) and the variable n is used as a counter. A loop over the array is used to check whether the string is already contained, and if so, next is used to avoid storing the string and jump to the next record of input. In END the loop then uses a for (i = 0; i < n; i++) form to output the information from both arrays in the order the string were seen in the original file, e.g.

awk -v n=0 '{
    for (i=2; i<=NF; i++)
        str = str " " $i
    a[str] += $1
    for (i = 0; i < n; i++)
        if (o[i] == str) {
            str=""
            next;
        }
    o[n++] = str;
    str=""
}
END {
    for (i = 0; i < n; i++) print a[o[i]], o[i]
}' file

Output

37  cup of coffee
75  sign on the dotted
30  take a test

How Can I calculate the sum of a specific column using bash?

If you wanted to sum over, say, the second column, but print all columns in some pipeline:

cat data | awk '{sum+=$2 ; print $0} END{print "sum=",sum}'

If the file data looks like:

1 2 3
4 5 6
7 8 9

Then the output would be:

1 2 3
4 5 6
7 8 9
sum= 15

---

Related Topics