Format String Unused Named Arguments

Format string unused named arguments

If you are using Python 3.2+, use can use str.format_map().

For bond, bond:

from collections import defaultdict
'{bond}, {james} {bond}'.format_map(defaultdict(str, bond='bond'))

Result:

'bond,  bond'

For bond, {james} bond:

class SafeDict(dict):
    def __missing__(self, key):
        return '{' + key + '}'

'{bond}, {james} {bond}'.format_map(SafeDict(bond='bond'))

Result:

'bond, {james} bond'

In Python 2.6/2.7

For bond, bond:

from collections import defaultdict
import string
string.Formatter().vformat('{bond}, {james} {bond}', (), defaultdict(str, bond='bond'))

Result:

'bond,  bond'

For bond, {james} bond:

from collections import defaultdict
import string

class SafeDict(dict):
    def __missing__(self, key):
        return '{' + key + '}'

string.Formatter().vformat('{bond}, {james} {bond}', (), SafeDict(bond='bond'))

Result:

'bond, {james} bond'

Why does `str.format()` ignore additional/unused arguments?

Ignoring un-used arguments makes it possible to create arbitrary format strings for arbitrary-sized dictionaries or objects.

Say you wanted to give your program the feature to let the end-user change the output. You document what fields are available, and tell users to put those fields in {...} slots in a string. The end-user then can create templating strings with any number of those fields being used, including none at all, without error.

In other words, the choice is deliberate, because there are practical reasons for allowing more arguments than are converted. Note that the C# String.Formatter implementation that inspired the Python PEP does the same, for those same reasons.

Not that the discussion on this part of the PEP is that clear cut; Guido van Rossum at some point tries to address this issue:

The PEP appears silent on what happens if there are too few or too
many positional arguments, or if there are missing or unused keywords.
Missing ones should be errors; I'm not sure about redundant (unused)
ones. On the one hand complaining about those gives us more certainty
that the format string is correct. On the other hand there are some
use cases for passing lots of keyword parameters (e.g. simple web
templating could pass a fixed set of variables using **dict). Even in
i18n (translation) apps I could see the usefulness of allowing unused
parameters

to which the PEP author responded that they were still undecided on this point.

For use-cases where you must raise an exception for unused arguments you are expected to subclass the string.Formatter() class and provide an implementation for Formatter.check_unused_args(); the default implementation does nothing. This of course doesn't help your friend's case where you used str.format(*args, **kwargs) rather than Formatter().format(str, *args, **kwargs). I believe that at some point the idea was that you could replace the formatter used by str.format() with a custom implementation, but that never came to pass.

If you use the flake8 linter, then you can add the flake8-string-format plugin to detect the obvious cases, where you passed in an explicit keyword argument that is not being used by the format string.

format strings and named arguments in Python

Named replacement fields (the {...} parts in a format string) match against keyword arguments to the .format() method, and not positional arguments.

Keyword arguments are like keys in a dictionary; order doesn't matter, as they are matched against a name.

If you wanted to match against positional arguments, use numbers:

"{0} {1}".format(10, 20)

In Python 2.7 and up, you can omit the numbers; the {} replacement fields are then auto-numbered in order of appearance in the formatting string:

"{} {}".format(10, 20) 

The formatting string can match against both positional and keyword arguments, and can use arguments multiple times:

"{1} {ham} {0} {foo} {1}".format(10, 20, foo='bar', ham='spam')

Quoting from the format string specification:

The field_name itself begins with an arg_name that is either a number or a keyword. If it’s a number, it refers to a positional argument, and if it’s a keyword, it refers to a named keyword argument.

Emphasis mine.

If you are creating a large formatting string, it is often much more readable and maintainable to use named replacement fields, so you don't have to keep counting out the arguments and figure out what argument goes where into the resulting string.

You can also use the **keywords calling syntax to apply an existing dictionary to a format, making it easy to turn a CSV file into formatted output:

import csv

fields = ('category', 'code', 'price', 'description', 'link', 'picture', 'plans')
table_row = '''\
    <tr>
      <td><img src="{picture}"></td>
      <td><a href="{link}">{description}</a> ({price:.2f})</td>
   </tr>
'''

with open(filename, 'rb') as infile:
    reader = csv.DictReader(infile, fieldnames=fields, delimiter='\t')
    for row in reader:
        row['price'] = float(row['price'])  # needed to make `.2f` formatting work
        print table_row.format(**row)

Here, picture, link, description and price are all keys in the row dictionary, and it is much easier to see what happens when I apply the row to the formatting string.

{fmt}: Will a named argument be ignored if it doesn't exist in the formatting string?

This is by design. Unused formatting arguments are essentially the same as unused arguments to any other function and are not an error. This is the case in {fmt} and Python's str.format it is modeled after as well as printf.

Unused positional argument skipped in String formatting (Swift)

I answer my own question but all credit to @Martin R that provides the relevant information in comments.

• It is not a bug, String(format:) does not support omitting positional parameters.
• It is a known behavior since ObjectiveC, see: stackoverflow.com/a/2946880/1187415
• If you only have String arguments you can use multiple String substitutions with String.replacingOccurrences(of:, with:) instead of String(format:).

More precision on the last solution. The following will work in the case that only one argument is used and also if both arguments are used in the greetingFormat String:

greetingFormat.replacingOccurrences(
    of: "%1$@", with: title)
    .replacingOccurrences(
    of: "%2$@", with: name)

Of course, with String.replacingOccurrences(of:, with:) you can choose other identifiers for the substitution than %1$@ and %2$@.

Allow unused named arguments in Rust's format!() family

If the set of colors are all known, you could "consume" them with zero-length arguments:

macro_rules! log {
    ($fmt:expr, $($arg:tt)*) => {
        println!(concat!($fmt, "{blue:.0}{red:.0}{reset:.0}"),  // <--
                 $($arg)*,
                 blue="BLUE", 
                 red="RED", 
                 reset="RESET")
    }
}

fn main() {
    log!("{red}{}{reset}", "<!>");
    // prints: RED<!>RESET
}

(Docs for concat! macro)

Note that the strings BLUE, RED, RESET will still be sent to the formatting function, so it will incur a minor overhead even nothing will be printed.

---

I think this is quite error prone, since if you forget a {reset} the rest of your console will become red. I wonder why not write something like:

macro_rules! log_red {
    ($fmt:expr, $($arg:tt)*) => {
        println!(concat!("RED", $fmt, "RESET"), $($arg)*);
    }
}
// also define `log_blue!`.

log_red!("{}", "text");

String formatting for keys with : and /

This is a known limitation of the basic parsing used in string formatting implementation. The section Simple and Compound Field Names in PEP 3101 describes 'getitem' support in str.format syntax (emphasis mine):

An example of the 'getitem' syntax:

"My name is {0[name]}".format(dict(name='Fred'))

It should be noted that the use of 'getitem' within a format string is much more limited than its conventional usage. In the above example, the string 'name' really is the literal string 'name', not a variable named 'name'. The rules for parsing an item key are very simple. If it starts with a digit, then it is treated as a number, otherwise it is used as a string.

Because keys are not quote-delimited, it is not possible to specify arbitrary dictionary keys (e.g., the strings "10" or ":-]") from within a format string.

And later under the "Implementation note":

The str.format() function will have a minimalist parser which only attempts to figure out when it is "done" with an identifier (by finding a '.' or a ']', or '}', etc.).

So this a shortcoming of str.format by design. The attentive reader may note that the OP's string formatting works in Python 3. Some edge cases were patched up in Python 3.4, yet the same issue is still present in Python 3.3 and below.

The relevant ticket was issue12014: str.format parses replacement field incorrectly. Since Python 2.7 is end-of-life now, the chances of getting those improvements from Python 3.4 backported to 2.7 are zero, so you will have to choose between two options:

1. Upgrade to Python 3
2. Refactor your code so that string formatting only uses simple names or numbers

Stricter String.format that does not ignore extra arguments?

Taking a different approach without counting or parsing for %. Not an efficient one though.

public static String safeFormat(String formatStr, Object... args) {
    List<Object> objects = Arrays.asList(args);
    // https://commons.apache.org/proper/commons-lang/javadocs/api-3.7/org/apache/commons/lang3/mutable/MutableBoolean.html
    MutableBoolean isCalled = new MutableBoolean(false);
    objects.add(new Object(){
        @Override
        public String toString() {
            isCalled.setTrue();
            return "";
        }
    });
    String result = String.format(formatStr+"%s", objects.toArray());
    if(isCalled.isFalse()) {
        throw new IllegalArgumentException("Not all arguments used by formatter");
    }

    return result;
}

---

Related Topics