Question

How can I parse variable length argument lists delimited by special predefined syntax. An example:

   ./script --arg1 --cmdname otherscript --a1 --a2 --cmdname-- --arg3

After parsing with argparse script should have three arguments: arg1, cmdname, arg3. The argument cmdname should consist of a list of three values otherscript, a1, a2.

Having such a recipe would be useful to be able to pass on everything in cmdname into a subprocess.popen(cmdname, ...) call.

I was thinking about subparsers. But I believe a subparser cannot be stopped, and really is mutually exclusive with other subparsers. Any other easy, already provided way? Is subclassing the Action the way to do it?

Answer 1

As you indicated in your post, subclassing Action is probably the way to do this — Although that gets pretty tricky if the arguments to otherscript aren’t known by argparse. You might be able to get around this with parse_known_args, but you might not. Honestly, I really think the easiest way is to preprocess sys.argv yourself.

import shlex
s = shlex.split("./script --arg1 --cmdname otherscript --a1 --a2 --cmdname-- --arg3")
def preprocess(lst):
    """
    process an iterable into 2 lists.
    The second list contains the portion bracketed by '--cmdname' and '--cmdname--'
    whereas the first portion contains the rest of it.
    """
    argv1,argv2 = [],[]
    current = argv1
    for i in lst:
        if i == '--cmdname':
           current = argv2
        elif i == '--cmdname--':
           current = argv1
        else:
           current.append(i)
    return argv1,argv2

l1,l2 = preprocess(s)
print l1
print l2

And an alternative implementation of preprocess which works for sliceable objects that have a .index method — sys.argv would work just fine:

def preprocess(lst):
    """
    process an iterable into 2 lists.
    The second list contains the portion bracketed by '--cmdname' and '--cmdname--'
    whereas the first portion contains the rest of it.
    """
    try:
        i1 = lst.index('--cmdname')
        i2 = lst.index('--cmdname--')
        argv1 = lst[i1+1:i2]
        argv2 = lst[:i1]+lst[i2+1:]
    except ValueError:
        argv1 = lst
        argv2 = []

    return argv1,argv2

---

Another option (pointed out in an excellent comment by @unutbu) is to change the commandline syntax to something a little more standard which simplifies the problem greatly:

./script --arg1 --cmd "otherscript --a1 --a2" --arg3

Then you can parse cmd as you normally would using argparse (specify type=shlex.split for this argument to convert from a string to a list of arguments).