How should I use the Optional type hint?
Optional[...]
is a shorthand notation for Union[..., None]
, telling the type checker that either an object of the specific type is required, or None
is required. ...
stands for any valid type hint, including complex compound types or a Union[]
of more types. Whenever you have a keyword argument with default value None
, you should use Optional
. (Note: If you are targeting Python 3.10 or newer, PEP 604 introduced a better syntax, see below).
So for your two examples, you have dict
and list
container types, but the default value for the a
keyword argument shows that None
is permitted too so use Optional[...]
:
from typing import Optional
def test(a: Optional[dict] = None) -> None:
#print(a) ==> {'a': 1234}
#or
#print(a) ==> None
def test(a: Optional[list] = None) -> None:
#print(a) ==> [1, 2, 3, 4, 'a', 'b']
#or
#print(a) ==> None
There is technically no difference between using Optional[]
on a Union[]
, or just adding None
to the Union[]
. So Optional[Union[str, int]]
and Union[str, int, None]
are exactly the same thing.Personally, I'd stick with always using Optional[]
when setting the type for a keyword argument that uses = None
to set a default value, this documents the reason why None
is allowed better. Moreover, it makes it easier to move the Union[...]
part into a separate type alias, or to later remove the Optional[...]
part if an argument becomes mandatory.
For example, say you have
from typing import Optional, Union
def api_function(optional_argument: Optional[Union[str, int]] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
then documentation is improved by pulling out the Union[str, int]
into a type alias:from typing import Optional, Union
# subwidget ids used to be integers, now they are strings. Support both.
SubWidgetId = Union[str, int]
def api_function(optional_argument: Optional[SubWidgetId] = None) -> None:
"""Frob the fooznar.
If optional_argument is given, it must be an id of the fooznar subwidget
to filter on. The id should be a string, or for backwards compatibility,
an integer is also accepted.
"""
The refactor to move the Union[]
into an alias was made all the much easier because Optional[...]
was used instead of Union[str, int, None]
. The None
value is not a 'subwidget id' after all, it's not part of the value, None
is meant to flag the absence of a value.Side note: Unless your code only has to support Python 3.9 or newer, you want to avoid using the standard library container types in type hinting, as you can't say anything about what types they must contain. So instead of dict
and list
, use typing.Dict
and typing.List
, respectively. And when only reading from a container type, you may just as well accept any immutable abstract container type; lists and tuples are Sequence
objects, while dict
is a Mapping
type:
from typing import Mapping, Optional, Sequence, Union
def test(a: Optional[Mapping[str, int]] = None) -> None:
"""accepts an optional map with string keys and integer values"""
# print(a) ==> {'a': 1234}
# or
# print(a) ==> None
def test(a: Optional[Sequence[Union[int, str]]] = None) -> None:
"""accepts an optional sequence of integers and strings
# print(a) ==> [1, 2, 3, 4, 'a', 'b']
# or
# print(a) ==> None
In Python 3.9 and up, the standard container types have all been updated to support using them in type hints, see PEP 585. But, while you now can use dict[str, int]
or list[Union[int, str]]
, you still may want to use the more expressive Mapping
and Sequence
annotations to indicate that a function won't be mutating the contents (they are treated as 'read only'), and that the functions would work with any object that works as a mapping or sequence, respectively.Python 3.10 introduces the |
union operator into type hinting, see PEP 604. Instead of Union[str, int]
you can write str | int
. In line with other type-hinted languages, the preferred (and more concise) way to denote an optional argument in Python 3.10 and up, is now Type | None
, e.g. str | None
or list | None
.
Python 3.10+: Optional[Type] or Type | None
PEP 604 covers these topics in the specification section.
The existingtyping.Union
and|
syntax should be equivalent.int | str == typing.Union[int, str]
The order of the items in the Union should not matter for equality.As @jonrsharpe comments,Optional values should be equivalent to the new union syntax(int | str) == (str | int)
(int | str | float) == typing.Union[str, float, int]None | t == typing.Optional[t]
Union
and Optional
are not deprecated, so the Union
and |
syntax are acceptable.Łukasz Langa, a Python core developer, replied on a YouTube live related to the Python 3.10 release that
Type | None
is preferred over Optional[Type]
for Python 3.10+. How to use typing hints with an optional first parameter
Let's see how slice
is hinted:
class slice(object):
start: Any
step: Any
stop: Any
@overload
def __init__(self, stop: Any) -> None: ...
@overload
def __init__(self, start: Any, stop: Any, step: Any = ...) -> None: ...
__hash__: None # type: ignore
def indices(self, len: SupportsIndex) -> Tuple[int, int, int]: ...
and range
class range(Sequence[int]):
start: int
stop: int
step: int
@overload
def __init__(self, stop: SupportsIndex) -> None: ...
@overload
def __init__(self, start: SupportsIndex, stop: SupportsIndex, step: SupportsIndex = ...) -> None: ...
[...]
Basically, you hint the two overloaded versions separately, one with the optional first parameter and one without. (So basically, your Attempt #2.)from typing import overload, List
@overload
def fun(start: int, stop: int, divisors: List[int]):
...
@overload
def fun(stop: int, divisors: List[int]):
...
def fun(start, stop, divisors=None):
if divisors is None:
divisors = stop
stop = start
start = 0
...
fun(1, 2, [1,2,3]) # OK
fun(2, [1,2,3]) # OK
If you like, you can also make both variants accept positional arguments only:# e.g.
def fun(start, stop, divisors=None, /):
...
How to type hint a function's optional return parameter?
Since Python 3.10 and PEP 604 you now can use |
instead of Union
.
The return type would be float | Tuple[float, float]
The right type hint would be:
from typing import Tuple, Union
def myfunc(x: float, return_y: bool = False) -> Union[float, Tuple[float, float]]:
z = 1.5
if return_y:
y = 2.0
return z, y
return z
However, it is usually not a good practice to have these kinds of return. Either return something like Tuple[float, Optional[float]]
or write multiple functions, it will be much easier to handle later on.More about return statement consistency:
- PEP8 - Programming Recommendations
Be consistent in return statements. Either all return statements in a function should return an expression, or none of them should. If any return statement returns an expression, any return statements where no value is returned should explicitly state this as return None, and an explicit return statement should be present at the end of the function (if reachable).
- Why should functions return values of a consistent type?
Optional type hint in Google Style Guide
Just because it's Optional
doesn't mean that it needs a default argument, or that the argument is "optional".
Optional[Text]
means "it can be a Text
object, or it can be None
". The None
value need not be a specified default though; it can be user supplied. You may, for whatever reason, want the user to pass that argument, even if it's just None
.
Part of the confusion might be the use of the term "optional" here. "Optional" in this context doesn't mean that the argument is optional. It means that it's an option type.
How to type hint with an optional import?
Try sticking your import inside of an if typing.TYPE_CHECKING
statement at the top of your file. This variable is always false at runtime but is treated as always true for the purposes of type hinting.
For example:
# Lets us avoid needing to use forward references everywhere
# for Python 3.7+
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import pandas as pd
def my_func() -> pd.DataFrame:
import pandas as pd
return pd.DataFrame()
You can also do if False:
, but I think that makes it a little harder for somebody to tell what's going on.One caveat is that this does mean that while pandas will be an optional dependency at runtime, it'll still be a mandatory one for the purposes of type checking.
Another option you can explore using is mypy's --always-true
and --always-false
flags. This would give you finer-grained control over which parts of your code are typechecked. For example, you could do something like this:
try:
import pandas as pd
PANDAS_EXISTS = True
except ImportError:
PANDAS_EXISTS = False
if PANDAS_EXISTS:
def my_func() -> pd.DataFrame:
return pd.DataFrame()
...then do mypy --always-true=PANDAS_EXISTS your_code.py
to type check it assuming pandas is imported and mypy --always-false=PANDAS_EXISTS your_code.py
to type check assuming it's missing.This could help you catch cases where you accidentally use a function that requires pandas from a function that isn't supposed to need it -- though the caveats are that (a) this is a mypy-only solution and (b) having functions that only sometimes exist in your library might be confusing for the end-user.
Optional Union in type hint
You may think of the typing
library as a specification on how to declare certain types. If something is not defined in that specification then it's always better assume it to be an undefined behavior.
However in the particular case of python and typing we have a kind-of-reference static type checker which is mypy. So in order to get an answer for your question, or just programmatically check types, we may use it and see if it shows any warnings.
Here's an example:
$ cat check_optional.py
import typing
def fn(x: typing.Optional[int, str]):
pass
$ mypy check_optional.py
check_optional.py:3: error: Optional[...] must have exactly one type argument
So no, Optional[T, U]
is not possible in terms of mypy even if there's no trouble declaring it within the typing
library.Besides from "functional programming" perspective both Optional
and Union
are two distinct but well-known and well defined monads. A combination of two monads (Union[T, U, None]
) is another monad, which however behaves differently than Optional
and thus should not be named so. In other words, Union[T, U, None]
is isomorphic to (=same as) Optional[Union[T, U]]
, but not to a general Optional[X]
.
Related Topics
Find the Date for the First Monday After a Given Date
Elif' in List Comprehension Conditionals
How to Save and Restore Multiple Variables in Python
How to Include Image Files in Django Templates
Regex for Existence of Some Words Whose Order Doesn't Matter
Extracting Days from a Numpy.Timedelta64 Value
Improve Current Implementation of a Setinterval
Pyspark Dataframes - Way to Enumerate Without Converting to Pandas
How to Limit the Maximum Value of a Numeric Field in a Django Model
Overriding the Save Method in Django Modelform
Why Is True Returned When Checking If an Empty String Is in Another
Opencv Python: Draw Minarearect ( Rotatedrect Not Implemented)
How to Print a Dictionary Line by Line in Python
Join Two Lists of Dictionaries on a Single Key