Python Argparse - Add Argument to Multiple Subparsers

Python argparse - Add argument to multiple subparsers

This can be achieved by defining a parent parser containing the common option(s):

import argparse

parent_parser = argparse.ArgumentParser(description="The parent parser")
parent_parser.add_argument("-p", type=int, required=True,
                           help="set db parameter")
subparsers = parent_parser.add_subparsers(title="actions")
parser_create = subparsers.add_parser("create", parents=[parent_parser],
                                      add_help=False,
                                      description="The create parser",
                                      help="create the orbix environment")
parser_create.add_argument("--name", help="name of the environment")
parser_update = subparsers.add_parser("update", parents=[parent_parser],
                                      add_help=False,
                                      description="The update parser",
                                      help="update the orbix environment")

This produces help messages of the format:

parent_parser.print_help()

Output:

usage: main.py [-h] -p P {create,update} ...
The parent parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

parser_create.print_help()

Output:

usage: main.py create [-h] -p P [--name NAME] {create,update} ...
The create parser
optional arguments:
  -h, --help       show this help message and exit
  -p P             set db parameter
  --name NAME      name of the environment
actions:
  {create,update}
    create         create the orbix environment
    update         update the orbix environment

However, if you run your program, you will not encounter an error if you do not specify an action (i.e. create or update). If you desire this behavior, modify your code as follows.

<...>
subparsers = parent_parser.add_subparsers(title="actions")
subparsers.required = True
subparsers.dest = 'command'
<...>

This fix was brought up in this SO question which refers to an issue tracking a pull request.

update by @hpaulj

Due to changes in handling subparsers since 2011, it is a bad idea to use the main parser as a parent. More generally, don't try to define the same argument (same dest) in both main and sub parsers. The subparser values will overwrite anything set by the main (even the subparser default does this). Create separate parser(s) to use as parents. And as shown in the documentation, parents should use add_help=False.

Argparse with multiple subparsers in one command

With current code you can create only host --add 192.168.1.1 but it is much simpler code.

import argparse

parser = argparse.ArgumentParser()

subparsers = parser.add_subparsers(dest='parser')

host_cmd = subparsers.add_parser('host')

host_cmd.add_argument('--add')
host_cmd.add_argument('--remove')

args = parser.parse_args()
print(args)

if args.parser == 'host':
    if args.add is not None:
        print('add host:', args.add)
    if args.remove is not None:
        print('remove host:', args.remove)

You need subparser in subparser - host add 192.168.1.1

import argparse

parser = argparse.ArgumentParser()

subparsers = parser.add_subparsers(dest='parser')

host_cmd = subparsers.add_parser('host')

host_subparsers = host_cmd.add_subparsers(dest='parser_host')

host_add_cmd = host_subparsers.add_parser('add')
host_add_cmd.add_argument('ip')

host_remove_cmd = host_subparsers.add_parser('remove')
host_remove_cmd.add_argument('ip')

args = parser.parse_args()
print(args)

if args.parser == 'host':
    if args.parser_host == 'add':
        print('add host:', args.ip)
    elif args.parser_host == 'remove':
        print('remove host:', args.ip)

EDIT: example for host add port 80 but there is conflict with host add 192.168.1.1 so I removed it

import argparse

parser = argparse.ArgumentParser()

subparsers = parser.add_subparsers(dest='parser')

host_cmd = subparsers.add_parser('host')

host_subparsers = host_cmd.add_subparsers(dest='parser_host')

host_add_cmd = host_subparsers.add_parser('add')
#host_add_cmd.add_argument('ip')

add_subparsers = host_add_cmd.add_subparsers(dest='parser_add')

host_add_port_cmd = add_subparsers.add_parser('port')
host_add_port_cmd.add_argument('add_port')

host_remove_cmd = host_subparsers.add_parser('remove')
host_remove_cmd.add_argument('ip')

args = parser.parse_args()
print(args)

if args.parser == 'host':
    if args.parser_host == 'add':
        if args.parser_add == 'port':
            print('add port', args.add_port)
    elif args.parser_host == 'remove':
        print('remove', args.ip)

argparse with multiple subparsers and positionals

Argparse is not suited for this kind of thing. add_subparsers assumes that exactly 1 of the sub-commands will be used, so it throws an error if you try to set both opencv and boost. And other than that, argparse has no concept of arguments being associated with other arguments.

Option 1

If you don't mind using keyword options instead of positional options, you can use the solution from this answer:

argv = '-l opencv -b stable -v 3.4.1 -l boost -b development -v 1.67.0'.split()

parser = argparse.ArgumentParser()
parent = parser.add_argument('-l', '--lib', choices=['opencv', 'boost'], action=ParentAction)

parser.add_argument('-b', '--build', action=ChildAction, parent=parent)
parser.add_argument('-v', '--version', action=ChildAction, parent=parent)

args = parser.parse_args(argv)
print(args)
# output:
# Namespace(lib=OrderedDict([('opencv', Namespace(build='stable',
#                                                 version='3.4.1')),
#                            ('boost', Namespace(build='development',
#                                                version='1.67.0'))]))

Option 2

Use the nargs argument to make it a mix of positional and named arguments:

argv = '-l opencv stable 3.4.1 -l boost development 1.67.0'.split()

parser = argparse.ArgumentParser()
parent = parser.add_argument('-l', '--lib', nargs=3, action='append')

args = parser.parse_args(argv)
print(args)
# output:
# Namespace(lib=[['opencv', 'stable', '3.4.1'],
#                ['boost', 'development', '1.67.0']])

Option 3

Parse the arguments manually:

argv = 'opencv stable 3.4.1 boost development 1.67.0'.split()

args = {}
argv_itr = iter(argv)
for lib in argv_itr:
    args[lib] = {'build': next(argv_itr),
                 'version': next(argv_itr)}

print(args)
# output:
# {'opencv': {'build': 'stable',
#             'version': '3.4.1'},
#  'boost': {'build': 'development',
#            'version': '1.67.0'}}

Python Argparse - Add multiple arguments to a subparser

After:

config=parser.parse_args()

add a print(config).

You'll see it's a namespace object with listed attributes. One is config.func, which is the func you call.

In:

config.func(config)

You call the func, but you pass it the whole config object.

Both functions should accept one argument, this config object.

One can ignore it, since it won't have any attributes that it needs. The other can fetch the two attributes it needs.

Change these two lines so it is easier to fetch the attributes:

parser_open.add_argument("port", help="port to open serial terminal")
parser_open.add_argument("baud", help="frequency of serial communication")

in the function

def open(config):
    port = config.port
    baud = config.baud
    ....

In effect I am telling you same thing that the # sub-command functions example does in the argparse docs. (e.g. the foo and bar functions).

How to add common arguments to argparse subcommands?

Make a separate parent parser and pass it to subparsers

import argparse                                                                  

parent_parser = argparse.ArgumentParser(add_help=False)                                 
parent_parser.add_argument('-H', '--host', default='192.168.122.1')                     
parent_parser.add_argument('-P', '--port', default='12345')                             

parser = argparse.ArgumentParser(add_help=False) 
subparsers = parser.add_subparsers()                                             

# subcommand a                                                                   
parser_a = subparsers.add_parser('a', parents = [parent_parser])                          
parser_a.add_argument('-D', '--daemon', action='store_true')                     

parser_a.add_argument('-L', '--log', default='/tmp/test.log')                    

# subcommand b                                                                   
parser_b = subparsers.add_parser('b', parents = [parent_parser])                          
parser_b.add_argument('-D', '--daemon', action='store_true')                     

# subcommand c                                                                   
parser_c = subparsers.add_parser('c', parents = [parent_parser])                          
args = parser.parse_args()                                                       

print args

This gives desired result

$ python arg.py a
Namespace(daemon=False, host='192.168.122.1', log='/tmp/test.log', port='12345')
$ python arg.py b -H 127.0.0.1 -P 11111
Namespace(daemon=False, host='127.0.0.1', port='11111')
$ python arg.py c
Namespace(host='192.168.122.1', port='12345')

How can I add more subparsers to an argpare parser?

What kind of behavior are you expecting from the added subparsers?

sp = parser.add_subparsers(...)

add_subparsers creates a positional argument of _SubParsersAction class, and makes a note of it in a parser._subparsers attribute. It raises that error if that attribute has already been set. sp is that Action object.

Logically it doesn't make sense to have more than one 'subparsers' action. When the main parser encounters of a subparser command, it delegates the parsing to that subparser. Apart from some cleanup error checking the main parser does not resume parsing. So it couldn't handle a second subparser command.

(the terminology for what add_subparsers creates and what add_parser does can be a confusing. One is a positional argument/Action, the other a parser (instance of ArgumentParser).

It should be possible to add new parsers to an existing subparsers Action (I'd have to experiment to find a clean way of doing this). It also possible to add nest subparsers, that is, define add_subparsers for a subparser.

That parents approach bypasses thismultiple subparser arguments test, probably because it doesn't set the parser._subparsers attribute the first time (when copying Actions from the parent). So that in effect creates two subparser arguments. That's what the help shows. But my guess is that the parsing will not be meaningful. It will still expect 'original_subparser' and not look for 'extra' or 'extra2'.

The parents mechanism is not robust. There are straight forward uses of it, but it's easily broken.

In [2]: parser = argparse.ArgumentParser(prog='main')
In [3]: a1 = parser.add_argument('--foo')
In [4]: parser._subparsers    # initial value None

In [5]: sp = parser.add_subparsers(dest='cmd')
In [6]: parser._subparsers    # new non-None value
Out[6]: <argparse._ArgumentGroup at 0xaf780f0c>

The sp object, an Action subclass:

In [8]: sp
Out[8]: _SubParsersAction(option_strings=[], dest='cmd', nargs='A...', const=None, default=None, type=None, choices=OrderedDict(), help=None, metavar=None)
In [9]: sp1 = sp.add_parser('cmd1')

sp is an Action (positional argument); sp1 is a parser.

In [10]: parser.print_help()
usage: main [-h] [--foo FOO] {cmd1} ...

positional arguments:
  {cmd1}

optional arguments:
  -h, --help  show this help message and exit
  --foo FOO

The _actions attribute is a list of actions. Note the help, and foo

In [12]: parser._subparsers._actions
Out[12]: 
[_HelpAction(option_strings=['-h', '--help'], dest='help',...),
 _StoreAction(option_strings=['--foo'], dest='foo',....),
 _SubParsersAction(option_strings=[], dest='cmd', ...]

So the last item in this list is the sp object

In [13]: parser._subparsers._actions[-1]
Out[13]: _SubParsersAction(option_strings=[], dest='cmd', nargs='A...', const=None, default=None, type=None, choices=OrderedDict([('cmd1', ArgumentParser(prog='main cmd1', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True))]), help=None, metavar=None)

So we can added a new subparser with

In [14]: parser._subparsers._actions[-1].add_parser('extra')
...
In [15]: parser.print_help()
usage: main [-h] [--foo FOO] {cmd1,extra} ...

positional arguments:
  {cmd1,extra}

optional arguments:
  -h, --help    show this help message and exit
  --foo FOO

So if you want just add extra and extra2 to original_subparser, this should work.

So as long as we don't try further parser.add_argument(...) grabbing the last _actions item should work. If it isn't the last, we'd have to do a more sophisticate search.

Judging from my rep change, someone found this earlier exploration of the _subparsers attribute:

How to obtain argparse subparsers from a parent parser (to inspect defaults)

Python Argparse - Add Argument to Multiple Subparsers