Type Annotations for *Args and **Kwargs

Type annotations for *args and **kwargs

For variable positional arguments (*args) and variable keyword arguments (**kw) you only need to specify the expected value for one such argument.

From the Arbitrary argument lists and default argument values section of the Type Hints PEP:

Arbitrary argument lists can as well be type annotated, so that the definition:

def foo(*args: str, **kwds: int): ...

is acceptable and it means that, e.g., all of the following represent function calls with valid types of arguments:

foo('a', 'b', 'c')
foo(x=1, y=2)
foo('', z=0)

So you'd want to specify your method like this:

def foo(*args: int):

However, if your function can only accept either one or two integer values, you should not use *args at all, use one explicit positional argument and a second keyword argument:

def foo(first: int, second: Optional[int] = None):

Now your function is actually limited to one or two arguments, and both must be integers if specified. *args always means 0 or more, and can't be limited by type hints to a more specific range.

How do I annotate a callable with *args and **kwargs?

As I know, python's typing does not allow do that straightforwardly as stated in the docs of typing.Callable:

There is no syntax to indicate optional or keyword arguments; such function types are rarely used as callback types. Callable[..., ReturnType] (literal ellipsis) can be used to type hint a callable taking any number of arguments and returning ReturnType.

But you could use mypy extensions like this:

from typing import Callable
from mypy_extensions import Arg, VarArg, KwArg

def foo(a: str, *args: int, **kwargs: float) -> str:
    return 'Hello, {}'.format(a)
    
def bar() -> Callable[[Arg(str, 'a'), VarArg(int), KwArg(float)], str]:
    return foo

Type annotation with multiple types in **kwargs

See this bug and this bug on the issue tracker for PyCharm. This is apparently an issue with PyCharm's checker; mypy (another type checker for Python) does not complain when I execute similar code.

There's already a fix for this and, it's apparently available in build 171.2014.23. Until then, I'd suppose Any would suffice as a temporary workaround to get the checker to stop complaining.

Type annotation for Callable that takes **kwargs

tl;dr: Protocol may be the closest feature that's implemented, but it's still not sufficient for what you need. See this issue for details.

---

Full answer:

I think the closest feature to what you're asking for is Protocol, which was introduced in Python 3.8 (and backported to older Pythons via typing_extensions). It allows you to define a Protocol subclass that describes the behaviors of the type, pretty much like an "interface" or "trait" in other languages. For functions, a similar syntax is supported:

from typing import Protocol
# from typing_extensions import Protocol  # if you're using Python 3.6

class MyFunction(Protocol):
    def __call__(self, a: Any, b: Any, **kwargs) -> bool: ...

def decorator(func: MyFunction):
    ...

@decorator  # this type-checks
def my_function(a, b, **kwargs) -> bool:
    return a == b

In this case, any function that have a matching signature can match the MyFunction type.

However, this is not sufficient for your requirements. In order for the function signatures to match, the function must be able to accept an arbitrary number of keyword arguments (i.e., have a **kwargs argument). To this point, there's still no way of specifying that the function may (optionally) take any keyword arguments. This GitHub issue discusses some possible (albeit verbose or complicated) solutions under the current restrictions.

---

For now, I would suggest just using Callable[..., bool] as the type annotation for f. It is possible, though, to use Protocol to refine the return type of the wrapper:

class ReturnFunc(Protocol):
    def __call__(self, s: str, **kwargs) -> bool: ...

def comparator(f: Callable[..., bool]) -> ReturnFunc:
    ....

This gets rid of the "unexpected keyword argument" error at equal_within("5.0 4.99998", rel_tol=1e-5).

Inferring argument types from **args

This is an old story, which was discussed for a long time. Here's mypy issue related to this problem (still open since 2018), it is even mentioned in PEP-589. Some steps were taken in right direction: python 3.11 introduced Unpack and allowed star unpacking in annotations - it was designed together with variadic generics support, see PEP-646 (backported to typing_extensions, but no mypy support yet AFAIC). But it works only for *args, **kwargs construction is still waiting.

However, it is possible with additional efforts. You can create your own decorator that can convince mypy that function has expected signature (playground):

from typing import Any, Callable, TypeVar, cast

_C = TypeVar('_C', bound=Callable)

def preserve_sig(func: _C) -> Callable[[Callable], _C]:
    def wrapper(f: Callable) -> _C:
        return cast(_C, f)
    return wrapper

def f(x: int, y: str = 'foo') -> int:
    return 1

@preserve_sig(f)
def g(**kwargs: Any) -> int:
    return f(**kwargs)

g(x=1, y='bar')
g(z=0)  # E: Unexpected keyword argument "z" for "g"

You can even alter function signature, appending or prepending new arguments, with PEP-612 (playground:

from functools import wraps
from typing import Any, Callable, Concatenate, ParamSpec, TypeVar, cast

_R = TypeVar('_R')
_P = ParamSpec('_P')

def alter_sig(func: Callable[_P, _R]) -> Callable[[Callable], Callable[Concatenate[int, _P], _R]]:
    
    def wrapper(f: Callable) -> Callable[Concatenate[int, _P], _R]:
        
        @wraps(f)
        def inner(num: int, *args: _P.args, **kwargs: _P.kwargs):
            print(num)
            return f(*args, **kwargs)
            
        return inner
    
    return wrapper

def f(x: int, y: str = 'foo') -> int:
    return 1

@alter_sig(f)
def g(**kwargs: Any) -> int:
    return f(**kwargs)

g(1, x=1, y='bar')
g(1, 2, 'bar')
g(1, 2)
g(x=1, y='bar')  # E: Too few arguments for "g"
g(1, 'baz')  # E: Argument 2 to "g" has incompatible type "str"; expected "int"
g(z=0)  # E: Unexpected keyword argument "z" for "g"

How to type hint args and kwargs, if not unpacking, in most general form?

If you know your actual arguments and kwargs then you can obviously use the specifics, including TypedDict for the the arguments.

In this most-general case you you can use Iterable and Mapping[str, object].

For known functions over-typing is quite possible here, thanks to named non-keywords, variable length positional arguments, etc. You may not want to disallow def foo(a, b, c=4) being called as foo(1, b=1, c=2)

How to annotate the type of arguments forwarded to another function?

I think out of the box this is not possible. However, you could write a decorator that takes the function that contains the arguments you want to get checked for (open in your case) as an input and returns the decorated function, i.e. open_for_writing in your case. This of course only works with python 3.10 or using typing_extensions as it makes use of ParamSpec

from typing import TypeVar, ParamSpec, Callable, Optional

T = TypeVar('T')
P = ParamSpec('P')

def take_annotation_from(this: Callable[P, Optional[T]]) -> Callable[[Callable], Callable[P, Optional[T]]]:
    def decorator(real_function: Callable) -> Callable[P, Optional[T]]:
        def new_function(*args: P.args, **kwargs: P.kwargs) -> Optional[T]:
            return real_function(*args, **kwargs)

        return new_function
    return decorator

@take_annotation_from(open)
def open_for_writing(*args, **kwargs):
    kwargs['mode'] = 'w'
    return open(*args, **kwargs)

open_for_writing(some_fake_arg=123)
open_for_writing(file='')

As shown here, mypy complains now about getting an unknown argument.

Python type hints and `*args`

According to PEP-484:

Arbitrary argument lists can as well be type annotated, so that the definition:

def foo(*args: str, **kwds: int): ...

is acceptable and it means that, e.g., all of the following represent function calls with valid types of arguments:

foo('a', 'b', 'c')
foo(x=1, y=2)
foo('', z=0)

In the body of function foo, the type of variable args is deduced as Tuple[str, ...] and the type of variable kwds is Dict[str, int].

The correct way to annotate the foo function from your example is:

def foo(*args: int) -> None:
    for x in args:
        print(x)

In Python 2:

def foo(*args):
    # type: (*int) -> None
    for x in args:
        print(x)

---

Related Topics