Why Do I Need Double Layer of Indirection for MACros

Why do I need double layer of indirection for macros?

The relevant part of the C spec:

6.10.3.1 Argument substitution

After the arguments for the invocation of a function-like macro have been identified,
argument substitution takes place. A parameter in the replacement list, unless preceded
by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is
replaced by the corresponding argument after all macros contained therein have been
expanded. Before being substituted, each argument’s preprocessing tokens are
completely macro replaced as if they formed the rest of the preprocessing file; no other
preprocessing tokens are available.

The key part that determines whether you want the double indirection or not is the second sentence and the exception in it -- if the parameter is involved in a # or ## operation (such as the params in mymacro and NAME2_HIDDEN), then any other macros in the argument are NOT expanded prior to doing the # or ##. If, on the other hand, there's no # or ## IMMEDIATELY in the macro body (as with NAME2), then other macros in the parameters ARE expanded.

So it comes down to what you want -- sometimes you want all macros expanded FIRST, and then do the # or ## (in which case you want the double layer indirection) and sometime you DO NOT want the macros expanded first (in which case you CAN'T HAVE double layer macros, you need to do it directly.)

Why is a level of indirection needed for this concatenation macro?

Why does the LINE_NAME macro even work if JOIN is declared after it in the file?

Macros are not functions, when you call them compiler expands them, and there compiler at the using point knows about all defined macros.

Why is it necessary to use both the JOIN and _DO_JOIN macros in order to get the correct result? Having this level of indirection in the macros seems very strange.

Because __LINE__ itself is a macro, it needs two level expanding.

Otherwise the output is not prefix1234 it will be prefix__LINE__.

it's useful to read this answer too, and this thread.

Understanding macro expansion

EDIT

3.9.6 Argument Prescan

Macro arguments are completely macro-expanded before they are
substituted into a macro body, unless they are stringified or pasted
with other tokens. After substitution, the entire macro body,
including the substituted arguments, is scanned again for macros to be
expanded. The result is that the arguments are scanned twice to expand
macro calls in them.

Macros that call other macros that stringify or concatenate. If an
argument is stringified or concatenated, the prescan does not occur.
If you want to expand a macro, then stringify or concatenate its
expansion, you can do that by causing one macro to call another macro
that does the stringification or concatenation.

For instance, if you have

#define AFTERX(x) X_ ## x
#define XAFTERX(x) AFTERX(x)
#define TABLESIZE 1024
#define BUFSIZE TABLESIZE

then AFTERX(BUFSIZE) expands to X_BUFSIZE, and XAFTERX(BUFSIZE) expands to X_1024. (Not to X_TABLESIZE. Prescan always does a complete expansion.)

CASE1

#define JOIN__(lhs, rhs) lhs##rhs => since it has a token pasting operator, it will concatenate the macro arguments those are not completely macro-expansion before substituted. --> which is a bad way to expand and on first place we don't know, what will be the arguments it will be passed to it, it won't wait for its expansion and it will simply concatenate it.

hence, when ever you call JOIN__(Foo, JOIN(A,B));, it won't allow JOIN(A,B) to expand and it will concatenate it to FOOJOIN(A,B).

CASE2
now, on the other hand,
#define JOIN_(lhs, rhs) JOIN__(lhs, rhs) => here, there is no token pasting operator, Macro arguments are completely macro-expanded before they are substituted into a macro body. hence, it will allow lhs and rhs to expanded and is called with expanded parameters JOIN__(FOO,AB), hence now, JOIN__ has a token pasting operator, it will simply concatenate it arguments FOO and AB i.e. FOOAB. which is the appropriate way to do it.

CASE3
#define JOIN(lhs, rhs) JOIN_(lhs, rhs) => same as CASE2.

Hope, it explains the reason behind multi-level expansion paradigm.

ORIGINAL
The preprocessor operator ## provides a way to concatenate actual arguments during macro expansion. If a parameter in the replacement text is adjacent to a ##, the parameter is replaced by the actual argument, the ## and surrounding white space are removed, and the result is re-scanned. For example, the macro paste concatenates its two arguments:

#define  paste(front, back)  front ## back

so paste(name, 1) creates the token  name1.

Why 2 levels of indirection in definition of macro ## operation

Forcing an additional expansion can make a difference where the initial expansion results in something that can be expanded further. A trivial example, given:

#define MACRO(x) x
#define EXPAND(x) x
#define NOEXPAND()

is:

MACRO NOEXPAND() (123)

This expands to MACRO (123). On the other hand, if you force an additional expansion, like so:

EXPAND(MACRO NOEXPAND() (123))

results in:

123

Normally, forcing additional expansions like this is unnecessary: any macro definitions that would cause this to make any difference are generally considered poor style anyway. But with certain specific inputs, it may be useful.

So for a concrete example with your MACRO_JOIN:

MACRO_JOIN(123, MACRO NOEXPAND() (456)) // expands to 123456
MACRO_JOIN1(123, MACRO NOEXPAND() (456)) // expands to 123MACRO (456)

C/C++ macro parameter containing dot (member access operator)

The token pasting operator ## requires its two operands to be valid preprocessing tokens, and yields a single preprocessing token. It is often used to concatenate two identifiers into a single identifier.

What you are trying to do here is not token pasting. Instead, you are seeking to create expressions like s.x or s.c.x where the x part is always a single token. Therefore, the ## operator should not be used. Instead, you can just do this:

#define M(obj, y) obj.y x

When you try to use the ## operator, the preprocessor tries to combine the last token in the argument y with the token x. In c., the . is a token, so the result is .x, which is not a valid token. Rather, .x is only valid as a sequence of two tokens.

How to force macro to not expand

How can we define CONCATENATE(x) macro to expand only one layer deep. ... Is this possible?

No. Here's what you have available. When macro invocation occurs, the first step is argument substitution (a.s.; 6.10.3.1); during that step, the tokens in the argument are evaluated if their corresponding parameter is mentioned in the macro's replacement list with said mention not being involved in a stringification or paste. The resulting expansion is substituted for said parameters in the replacement list. Next, stringification/pastes are applied in no particular order. Finally, rescan and further replacement (r.a.f.r; 6.10.3.4p1) occurs, during which the resulting replacement list itself is scanned; during this scan, the macro's name is "painted blue" (6.10.3.4p2; "blue paint" is not mentioned by name but is technical jargon for this), meaning if it's encountered it will not expand further.

So let's look at it from this point of view. UART_PERIPH is an identifier. Either it will be recognized as a macro in some context (i.e., will trigger macro invocation), or it will not. It doesn't matter if the context is during an a.s. or a r.a.f.r.; if this is invoked, the invocation involves r.a.f.r. (no a.s. because it's object-like). So the invocation involves taking USART2 and rescanning it. The only possible way for this to not expand USART2 is for this identifier to not be recognized as a macro, but since it's currently defined as one, the only way for that to happen is for this identifier to be painted blue. That's not possible (at least in the intended context) because USART2 would have to be expanding for this to happen, and by that time you're already injecting tokens you don't want.

---

Related Topics