Is Floating Point Addition Commutative in C++

Is floating point addition commutative in C++?

It is not even required that a + b == a + b. One of the subexpressions may hold the result of the addition with more precision than the other one, for example when the use of multiple additions requires one of the subexpressions to be temporarily stored in memory, when the other subexpression can be kept in a register (with higher precision).

If a + b == a + b is not guaranteed, a + b == b + a cannot be guaranteed. If a + b does not have to return the same value each time, and the values are different, one of them necessarily will not be equal to one particular evaluation of b + a.

Is floating-point addition and multiplication associative?

Floating point addition is not necessarily associative. If you change the order in which you add things up, this can change the result.

The standard paper on the subject is What Every Computer Scientist Should Know about Floating Point Arithmetic. It gives the following example:

Another grey area concerns the interpretation of parentheses. Due to roundoff errors, the associative laws of algebra do not necessarily hold for floating-point numbers. For example, the expression (x+y)+z has a totally different answer than x+(y+z) when x = 1e30, y = -1e30 and z = 1 (it is 1 in the former case, 0 in the latter).

Is floating-point multiplication commutative?

According to Wikipedia, yes, float multiplication is commutative.

While floating-point addition and multiplication are both commutative (a + b = b + a and a×b = b×a), they are not necessarily associative. That is, (a + b) + c is not necessarily equal to a + (b + c).

Example of non associative floating point addition

There is no such example, because floating point addition, as defined by IEEE-754, is guaranteed to be commutative. The relevant verbiage:

...Each of the operations shall be performed as if it first produced an intermediate result correct to infinite precision and with unbounded range, and then coerced this intermediate result to fit in the destination's format.

That is, the addition itself is performed with infinite precision; all errors in the result come from the need to discard some of that precision. Since infinite-precision (that is, "regular") addition is commutative, that means the operation as a whole is commutative.

What you may be thinking of is associativity. Floating point addition is not associative, because the precision loss following adding the first two numbers will not generally be the same as that from adding the last two numbers. The most common example of this is known as "catastrophic cancellation": (1 + 1e100) + -1e100 = 0, and 1 + (1e100 + -1e100) = 1.

Is arithmetic commutative and associative?

Assuming no unchecked overflows occur, .NET's integer addition and multiplication (int, long, etc.) is commutative and associative, like the real numbers in math. Floating point arithmetic (float and double) is commutative, but not always precisely associative, due to the limits of precision. From Wikipedia (with an example in the article):

While floating-point addition and multiplication are both commutative (a + b = b + a and a×b = b×a), they are not necessarily associative. That is, (a + b) + c is not necessarily equal to a + (b + c).

Here's an example:

a: 0.825402526103613
b: 0.909231618470155
c: 0.654626872695343
(a*b)*c: 0.491285733573819
a*(b*c): 0.49128573357382

There are some examples where the results appear the same when turned into a string, but are different ((a*b)*c != a*(b*c) is true, and (a*b)*c - a*(b*c) returns a small value, not 0).

a: 0.613781429181705
b: 0.648859122604532
c: 0.795545351596337
(a*b)*c: 0.316832045751117
a*(b*c): 0.316832045751117
difference: 5.55111512312578E-17

Floating-point arithmetic: why would order of addition matter?

Suppose you have a decimal floating-point type with three digits of precision. Not very realistic, but it makes for a simpler example.

Suppose you have three variables, a, b and c. Suppose a is 1000, b and c are both 14.

a + b would be 1014, rounded down to 1010. (a + b) + c would then be 1024, rounded down to 1020.

b + c would be 28. a + (b + c) would then be 1028, rounded up to 1030.

Floating point arithmetic rounding to nearest

Properly implemented IEEE-754 floating-point addition is commutative (a+b equals b+a) regardless of rounding mode.

The rounding mode affects how the exact mathematical result is rounded to fit into the destination format. Since the exact mathematical results of a+b and b+a are identical, they are rounded identically.

Is multiplication always commutative in inexact floating point arithmetic?

I know nothing about the D language, but I'll happily answer the question in your title:

Is multiplication always commutative in inexact floating point arithmetic?

Up to the "payload" of NaN values, yes. IEEE-754 floating-point multiplication is commutative (and so is addition). If you don't know what the payload of a NaN is, don't worry about it.

Order of commutative mathematical operations

Based on the standard

[intro.abstract] - Note 7 (non-normative):

Operators can be regrouped according to the usual mathematical rules
only where the operators really are associative or commutative.

Mathematical rule for MDAS is from left to right (considering the associativity and precedence of operators). So it is evaluated as follows:

(((((c * d) * e) * 2) / 3) * f)

---

Related Topics