Provide convenience API for global options (was: Feature request: inheriting mixins in subcommands)

[garretwilson]

I made a base class BaseCliApplication that all my CLI applications will extend. The base class does all the special picocli setup needed and other things for CLI applications. The class adds a --debug option that turns on debug-level logging:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
  …
}

Unfortunately in my Foo CLI application subclass, I introduce a subcommand bar using a method:

@Command(description = "Do the bar thing.")
public void bar(…) {
  …
}

Picocli doesn't seem to pick up the --debug switch in the subcommand. If I attempt to use the command foo bar, it tells me:

Unknown option: --debug
Usage: foo bar
Do the bar thing.

Is there a @Command parameter that I can use to say "this switch applies to all subcommands"? Or is there a way to specify for a subcommand that the --debug switch should work for it as well?

I looked for documentation for mixins, but I only found information on how to include them if they are already defined. I'm not sure how to define a mixin. How can I say that the setDebug() method is a mixin? (Frankly I'd rather use the first approach: some way to say "this switch applies to all subcommands".)

Thanks in advance.

[remkop]

Picocli provides a @ParentCommand for situations like this, where subcommands need to access "global" options defined on their parent command.

The user manual has an example.

[garretwilson]

Thanks, but I'm not sure how I would use it. In my Foo application, if I call the foo --debug parent command, Picocli automatically calls Foo.setDebug() on the application instance.

Where would I put the @ParentCommand annotation so that Picocli continues to call Foo.setDebug even for foo bar --debug? There is only one example I can find, and it doesn't seem to apply to this situation.

Are you saying I could put this in the annotation for Foo.bar() (the subcommand)?

@Command(description = "Do the bar thing.")
@ParentCommand(Foo.class)
public void bar(…) {
  …
}

Will that work? (I doubt it.) Even if it does, I don't see the point, because it would seem that foo() is already inherently a subcommand of Foo, because it is, after all, Foo.bar(), and both Foo and bar() have @Command annotations. That's how we get foo bar to work, after all.

Perhaps I'm just not understanding what you're suggesting.

[remkop]

The picocli-examples module has this example: https://github.com/remkop/picocli/blob/master/picocli-examples/src/main/java/picocli/examples/subcommands/ParentCommandDemo.java

[remkop]

You could define a --debug option on every subcommand, but I the problem is how to detect that the debug option was specified on any of the parent or grandparent etc commands in the hierarchy on the command line.

For example, if the user invoked:

top-level-command --debug subcommand sub-subcommand --other-option

The SubSubcommand may have its own definition of the --debug option, but this is not set. What was set was the --debug option on the TopLevelCommand. So the business logic of SubSubCommand needs to ask its parent (and potentially its parent's parent) whether the --debug option was set.

This can be implemented something like this:

@Command(name = "top-level-command", subcommands = SubCommand.class)
class TopLevelCommand {
  @Option(name = "--debug") boolean debug;
}

@Command(name = "subcommand", subcommands = SubSubCommand.class)
class SubCommand {
  @Option(name = "--debug") boolean debug;

  @ParentCommand
  TopLevelCommand topLevelCommand;
}

@Command(name = "sub-subcommand")
class SubSubCommand {
  @Option(name = "--debug") boolean debug;

  @ParentCommand
  SubCommand myParent;

  public void run() {
     if (this.debug || myParent.debug || myParent.topLevelCommand.debug) {
        System.out.println("I'm debugging...");
     }
  }
}

[garretwilson]

Yeah, I'm still not getting it. I have the feeling you might not have read my example very closely. The example that was given uses a static subclass for a command. But in my example the subcommand is a method. Where do I put an annotation to an instance variable inside a method? Methods can't have instance variables. They just have variables on the stack.

Let me show all the pieces together:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar(…) {
    …
  }

So where would I use @ParentCommand?

Are you saying that I must change my subcommand to use a static internal class? This is starting to look messy:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(name = "bar", description = "Do the bar thing.")
  static class Bar implements Runnable {

    public void run() {
      //I have to transfer the logic from the bar() method to here
    }
  }

And that was just to refactor. Now I have to go back and add the @ParentCommand:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(name = "bar", description = "Do the bar thing.")
  static class Bar implements Runnable {

    @ParentCommand
    private Foo parent;

    public void run() {
      //I have to transfer the logic from the bar() method to here

      //plus now I have to do this??
      if(parent.isDebug()) {
        //wait, what do I do here? if parent.setDebug() had been called, I wouldn't need any of this
      }
    }
  }

I guess I'm still lost.

[garretwilson]

The SubSubcommand may have its own definition of the --debug option, but this is not set. What was set was the --debug option on the TopLevelCommand.

No, but that's the point—the --debug option was not set for the top-level command. If Picocli had called Foo.setDebug(), then I would be happy, because this is a global setting. The subcommand bar doesn't need to access any "parent command" to find out if Foo.setDebug() was called. The problem is that Foo.setDebug() isn't even being called. Even worse, Picocli specifically prevents the --debug switch in the subcommands.

If I add the --debug option to the Foo.bar() subcommand, will Picocli automatically call the parent command Foo.setDebug() method? That would be one workaround, although it would be a lot of work to remember to add @Option(names = {"--debug", "-d"} to every single subcommand for every single application I ever create.

The whole point of creating BaseCliApplication is so that all applications that extend BaseCliApplication would automatically get a --debug switch, and BaseCliApplication.setDebug() would automatically get called if that switch was present. That's what I need.

[remkop]

Ah ok, yes, I missed that the @Command annotation was on a method. This does not support the @ParentCommand annotation. Apologies for not reading more closely.

Methods do not inherit from BaseCliApplication though, as you point out.

How about using a mixin?

class DebugMixin {
    @Option(names = "--debug")
    protected void setDebug(final boolean debug) {
       // one idea is to update static state here, so that the DebugMixin instance does not matter
        …
    }
}

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Mixin
  DebugMixin debugMixin;

  @Command(description = "Do the bar thing.")
  public void bar(@Mixin DebugMixin debugMixin, @Option other…) {
    …
  }

Would that work?

[garretwilson]

OK, so I have good news and bad news.

If I manually add @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug to every single subcommand, then Picocli will automatically call the parent command Foo.setDebug(). Like this:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar(@Option(names = {"--debug", "-d"} boolean debug) {
    …
  }

The bad news is that this defeats half the purpose of having a base application class. Now I'll have to remember to include @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug with every single subcommand I ever create in the future for any application I create. I had wanted this to "just work" automatically.

So this will get me by in the short term, but we really need an option that says inherited=true or something similar for any option, so I could just do this:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.", inherited=true)
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar() {
    …
  }

That way, any subcommand of any subclass of BaseCliApplication would automatically have its Subclass.setDebug() method called if --debug were present.

Would it be agreeable to add such an inherited flag? (Or something similar? The name is just off the top of my head.)

(BTW I see our replies are sort of going past each other as we both respond. 😄 ) To respond to your @DebugMixin (thanks for the example), that helps but not much. Really it's just a variation of what I say in this reply. In fact I can do the same thing using @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug instead of @Mixin DebugMixin debugMixin, and now that I look at it, that's almost uglier than just spelling out the option. Plus I have to create a completely new class just to pull it off.

And the weird thing is, if I add this @DebugMixin, wouldn't it result in Picocli still calling Foo.setDebug() as well? So really @DebugMixin would be a fake mixin, a lot of work just to get Picocli to call Foo.setDebug() , which is all I really wanted to begin with.

So for the meantime I'll just add @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug to every single subcommand, and hope that you'll let me add some sort of inherited flag in the future.

[remkop]

I'm not sure what you mean by

If I manually add @Option(names = {"--debug", "-d"} boolean debug to every single subcommand, then Picocli will automatically call the parent command Foo.setDebug().

Picocli should only call parent command Foo.setDebug() if the user specified foo --debug bar. If the user specified foo bar --debug then Foo.setDebug() should not be invoked (but the bar subcommand will get its --debug option set to true). For foo to be invoked for an option on its bar subcommand would be a bug. I just tried, but could not reproduce this.

Are you sure this is what is happening? Can you share code that reproduces this behaviour?

---

Generally, Picocli offers two mechanisms for reuse: inheritance and mixins. Command methods cannot inherit, so that leaves mixins. For a single option, it may be too much overhead to define a mixin class, I guess that depends on the implementation of the setDebug method.

Mixins allow you to invoke logic when an @Option is set (the setDebug method in your example), while an @Option on a command method is just a parameter, and you would need to have logic inside the command method to do something with the parameter value.

If you do decide to go with a mixin, I like to make mixins reusable components, so it would probably make more sense to move the logic from Foo.setDebug to the DebugMixin implementation, to make it more independent.

[garretwilson]

Picocli should only call parent command Foo.setDebug() if the user specified foo --debug bar. If the user specified foo bar --debug then Foo.setDebug() should not be invoked (but the bar subcommand will get its --debug option set to true). For foo to be invoked for an option on its bar subcommand would be a bug. I just tried, but could not reproduce this.

Oops, you're right. The reason it worked for me is that I had added a hack workaround to get debug turned on by calling setDebug(true) in the subclass constructor. so the only thing I accomplished with the extra @Option was to get Picocli to allow me to send a --debug switch with the bar command. The subclass was what was calling Foo.setDebug().

😞

Generally, Picocli offers two mechanisms for reuse: inheritance and mixins. Command methods cannot inherit, so that leaves mixins.

But even if I refactor the code to switch to internal classes (see above), that doesn't help anything. So this "inheritance" doesn't help my situation, regardless of whether it's for a subcommand or not.

Mixins are bulky and, and require that I remember in every single application and every single command to add this mixin manually.

We need a simply way to say, "this option applies to all subcommands". It's a simple flag. Why do we need to make things complicated? I think this mixin idea is wonderful for more complicated needs that require more flexibility and more access. But this is not a complicated need. It is a simple need.

[remkop]

So, given an option defined in a command (which is bound to a field or method in that command) you’re asking to introduce a mechanism for adding this option to subcommands, while keeping these subcommand options bound to the parent command field or method.

I need to think about this. At first glance it seems very application-specific and I have doubts that it is a general enough use case to warrant support in the library. But I could be wrong. I frequently am.

For now I would recommend using a Mixin to allow invoking the setDebug method in any subcommand just by defining an annotated field or method parameter.

[garretwilson]

I need to think about this.

Definitely, and so do I! My proposal was meant to be the impetus for more thinking and coming up with the best design. The initial solution was off the top of my head; we may still think up something better, or realize why this is a bad idea.

On the surface of it, though, it would seem something generally useful.

I'll be thinking more about it too, thanks.

[remkop]

I don't see how this would work for options that are bound to method parameters (when the option is defined on an annotated @Command method). For example:

public class Xxx {

  @Command(subcommands = Baz.class)
  public void bar(@Option(names = "--debug", inherited=true) boolean debug) {
    …
  }

  @Command(name = "baz")
  static class Baz implements Runnable {
    public void run() {
      // how can baz find out whether `bar baz --debug` was specified?
    }
  }
}

[garretwilson]

First let me ask you this (because I haven't used this configuration): in the example you gave, if I issue the command bar baz, is the Xxx.bar() method called, or only Xxx.Baz.run()?

[garretwilson]

how can baz find out whether bar baz --debug was specified?

Your library allows a lot of combinations of things that can work together. We can't use all the combinations together at one time, because they don't always make sense. You already pointed out, for example, that @ParentCommand doesn't work with subcommands that are methods.

So the simple answer to your question seems to be that you can say that the "inherited" flag only works for annotations on accessor methods (e.g. setDebug()) that are guaranteed to present at the time the subcommand is invoked. Otherwise, "inherited" doesn't make sense, as there is "nothing up there" (at the parent command level) to even know about the flag.

That seems like a reasonable answer and not too much of a kludge. Nobody expects all the combinations of features to work with all the available ways there is to put the application together.

[remkop]

First let me ask you this (because I haven't used this configuration): in the example you gave, if I issue the command bar baz, is the Xxx.bar() method called, or only Xxx.Baz.run()?

By default, only Xxx.Baz.run()

[remkop]

I try to avoid providing features that only sometimes work.

One could argue that @ParentCommand is an example of a feature that does not always work, but that one is more like a missing feature that could be added if necessary, and anyway, if that was a mistake, would it be a good idea to repeat it...? :-)

I would only consider introducing a potentially confusing feature like this if it brings considerable benefits when it is applicable. Given that the same (and more) can be achieved with mixins, the only argument for this new mechanism is that it gives more terse syntax than mixins.

To be honest, I am liking this less the more I think about it.

[garretwilson]

But we've only thought about it for an hour or so! haha I don't even know how much I like it either. I think about things for weeks or longer.

So far I'm liking the idea of a mixin that can be declared "inherited". It is self-contained. If a subcommand doesn't need the information passed, then it doesn't need to redeclare the mixin. The mixin would still "do its job" down the hierarchy as long as it was declared "inherited".

I could do @Mixin(inherited=true) DebugMixin debugMixin and all the subcommands would get the mixin. I suppose I could do the same with the version mixin.

It seems that what I'm talking about are options for cross-cutting concerns—things that apply across functionalities. These include:

• logging
• internationalization
• configuration

So I would want to turn on debugging for any command. I would want to set the user interface language for any command. I might want to specify the configuration for any command.

But there is still more thinking to do.

[garretwilson]

To put this into perspective, for the past several years I've created a series of small, loosely-coupled libraries for cross-cutting concerns. (They are refactorings of functionality I've built into applications for over a decade.) These include:

• Rincl (internationalization)
• Clogr (logging)
• Confound (configuration)

Take logging for instance. I have a base application class as I mentioned, BaseCliApplication. All my CLI applications, regardless of commands or subcommands or sub-sub-commands, need a way to initialize the Clogr SLF4J logging so that all the application and all its dependencies auto-magically get their log files put into some file. For example I might want to say --log-file mylogs.log --log-rollover 24h or some sort. Imagine if I had to remember, not only for every single application, but every available subcommand, to include a LoggingMixin!! 😮 But if I could declare LoggingMixin to be inherited at the BaseCliApplication level, that may be all that's needed.

I'm still not sure that's the perfect path forward, but hopefully you understand better that this isn't some esoteric, one-off functionality that's needed for some niche application.

[remkop]

The thing is, all @Option and @Parameters annotations are bound to something. This something can be a field, a method, or a method parameter. This binding captures the value that was specified for that option or positional parameter.

If you look at picocli's mechanisms for reuse, it is clear how the binding works for both, and how the command can retrieve the specified values:

• class inheritance: the subclass inherits the fields and methods from its superclass. Command obtains values by inspecting the inherited fields.
• mixins: each mixin class defines its own annotated fields and methods, and the command obtains the values by inspecting the mixin's fields.

Any new mechanism reuse would need to clearly define how the bindings for the reused options and positional parameters would work. With java class inheritance and mixins, it is very easy to understand for users how this works. People can just use the annotations without thinking about how things work under the hood. I doubt we will be able to achieve a similar simplicity with an additional reuse mechanism.

I really hesitate to introduce a 3rd mechanism for reuse. Bear in mind, if you don't like using mixins, perhaps instead of using @Command-annotated methods, you may be better off using @Command-annoted classes, so you can use inheritance as a reuse mechanism.

[garretwilson]

Bear in mind, if you don't like using mixins, perhaps instead of using @Command-annotated methods, you may be better off using @Command-annoted classes, so you can use inheritance as a reuse mechanism.

Could you tell me more about how to do this? My applications already inherit from BaseCliApplication. So you're saying I could somehow add a @Command to BaseCliApplication just to add an option, without knowing what the name of the "root" command will be?

I really hesitate to introduce a 3rd mechanism for reuse.

Well, if Picocli already has a mechanism for reuse, then I'll use that! So here is what I want to know: what can I add to BaseCliApplication so that all subclasses and all their subcommands automatically get a working --debug switch? It could be a mixin, a method, an annotation, a huge subclass—I don't care, as long as I hide it from the base classes and base classes don't have to do any more work in their part to get this functionality. Just let me know which mechanism lets me do that.

[remkop]

Basically what you were doing originally, but using classes for commands instead of @Command-annotated methods.

If BaseCliApplication defines the debug option like this:

@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
  // I don't know what this does but I assume it modifies static state
  // so that it works the same regardless of whether it is set
  // from a subcommand or from a top-level command
}

And all your commands inherit (directly or indirectly) from BaseCliApplication, would that not work? (Note, that means no @Command-annotated methods because these do not inherit from BaseCliApplication.)

For example:

@Command(name = "foo", subcommands = Bar.class)
public class Foo extends BaseCliApplication implements Runnable {
  @Option(name = "--other")
  int other;

  // ...
}

@Command(name = "bar", subcommands = Baz.class)
public class Bar extends BaseCliApplication implements Callable<Void> {
  @Option(name = "--other-option")
  String otherOption;

  public Void call() throws Exception {
    // ...
  }

  @Command(name = "baz")
  static class Baz extends BaseCliApplication implements Callable<Void> {
    @Option(name = "--yet-other-option")
    String yetAnother;

    public Void call() throws Exception {
      // ...
    }
  }
}

[garretwilson]

But bar is not an application! There is only one application, the foo application. It inherits from BaseCliApplication. The base application class comes with lots of extra things related to making a CLI application. That's why its'a base application class.

bar is only a subcommand of foo. It has no reason to inherit the entire application infrastructure.

Maybe I could refactor the debug() option out into a separate ApplicationCommand class, separate from the BaseCliApplication hierarchy, and then have both BaseCliApplication and any subcommand internal class inherit from this ApplicationCommand?

Even if that works (I'll have to try it), still that pollutes the structure of the application a lot. It was so simple, and I was so happy, to be able to add a subcommand just by adding a new method bar() and annotating it. Now I would have to add a new class, implement a call method, etc. And now there's no easy way to pass parameters to the bar() method. Now I have to make instance variables and/or setter methods and then somehow retrieve them from within Bar.call(). So much boilerplate now, for what used to be simple, just so I can get --debug to work across all commands!!

[garretwilson]

@Command(name = "bar", subcommands = Baz.class)
public class Bar extends BaseCliApplication implements Callable<Void> {
  @Option(name = "--other-option")
  String otherOption;

  public Void call() throws Exception {
    // ...
  }

But look how complicate this has become! Before I was able to do this:

  @Command(name = "bar")
  void bar(@Option(name = "--other-option") String option) {
      // ...
    }
  }

[remkop]

I was just trying to answer your question, sorry to upset you.

We've gone over the options that are available to you now for defining a --debug option in multiple commands. None of them are perfect, they all have trade-offs. You are best positioned to select the most palatable trade-off, given your understanding of the requirements, existing components like BaseCliApplication, and what code style you are aiming for.

[garretwilson]

I was just trying to answer your question, sorry to upset you.

I'm not upset. I was just listing the downsides of the current inheritance approaches, because you seemed to imply the existing functionality was sufficient.

None of them are perfect, they all have trade-offs.

And that's why I was proposing a solution that would meet these needs. But you don't want to allow any additional functional here. And it is your library, of course.

But since it's such a big need, and it applies to so many cross-cutting functionality configurations, I'll likely just keep limping along with Picocli until I get some time and then just write my own system (or maybe just expand on the one I wrote 10 or 15 years ago).

Thanks for the discussion.

[remkop]

@garretwilson Reopening as we may be able to accomplish this without introducing a completely new mechanism for reuse.

Let's look further at your idea to add an attribute to the mixin annotation that tells picocli to apply the mixin to subcommands. So it could be used something like this:

@Command 
class BaseCommand {

    @Mixin(inheritedBySubcommands= true)
    DebugMixin debugMixin;
}

This raises some new questions:

• How would the business logic in subcommands be able to inspect the inherited options and positional parameters?
• Do we need a mechanism for more fine-grained control over which subcommands inherit or don’t inherit such Mixins? I’m sure there’ll be use cases for exceptions, like inherit only in the subcommands that are direct subcommands, or exclude specific subcommands.

This last point may be addressed with an explicit exclusion list:

@Mixin(inheritedBySubcommands = true, 
    exclude = { AaaCommand.class, BbbCommand.class, CccCommand.class}

The first point is trickier.

[garretwilson]

I'm cleaning up my inbox and wanted to circle back on this ticket. I hope you are are doing well, and that you are keeping safe and healthy in this world health crisis.

Interesting feedback from a colleague: an enum instead of a boolean may be more flexible (extensible in the future).

Perhaps, but in your example I question the terms "local" and "global". If I set scope = Scope.GLOBAL, will this command be visible in an unrelated command subtree? I think the answer is no. If so, then it's not "global"; it's merely "inherited".

Hopefully within the next month or two I'll be back to the point on my project where I can upgrade to the latest picocli version, so I news on this from time to time is great.

[remkop]

In the latest version (now in master) it’s actually called INHERITED.

[remkop]

Quick update: I am hoping to do a release that includes this feature soon. Relevant snippet from the release notes follows below:

---

Inherited Options

This release adds support for "inherited" options. Options defined with scope = ScopeType.INHERIT are shared with all subcommands (and sub-subcommands, to any level of depth). Applications can define an inherited option on the top-level command, in one place, to allow end users to specify this option anywhere: not only on the top-level command, but also on any of the subcommands and nested sub-subcommands.

Below is an example where an inherited option is used to configure logging.

@Command(name = "app", subcommands = Sub.class)
class App implements Runnable {
    private static Logger logger = LogManager.getLogger(App.class);

    @Option(names = "-x", scope = ScopeType.LOCAL) // option is not shared: this is the default
    int x;
    
    @Option(names = "-v", scope = ScopeType.INHERIT) // option is shared with subcommands, sub-subcommands, etc
    public void setVerbose(boolean verbose) {
        // Configure log4j.
        // This is a simplistic example: you probably only want to modify the ConsoleAppender level.
        Configurator.setRootLevel(verbose ? Level.DEBUG : Level.INFO);
    }
    
    public void run() {
        logger.debug("-x={}", x);
    }
}

@Command(name = "sub")
class Sub implements Runnable {
    private static Logger logger = LogManager.getLogger(Sub.class);

    @Option(names = "-y")
    int y;
    
    public void run() {
        logger.debug("-y={}", y);
    }
}

Users can specify the -v option on either the top-level command or on the subcommand, and it will have the same effect.

# the -v option can be specified on the top-level command
app -x=3 -v sub -y=4

Specifying the -v option on the subcommand will have the same effect. For example:

# specifying the -v option on the subcommand also changes the log level
app -x=3 sub -y=4 -v

[garretwilson]

Good morning! I guess this isn't released yet?

I just finished off a some big features in my main project, so for a break I figured I would come bring in some of the exciting new picocli feature you've been talking about. So this morning I brought up my project and did a search, but Maven Central is still showing v4.2.0 so I guess the version you mentioned is still in the oven. Oh, well. I guess I'll work on something else this morning.

I'm looking forward to the new version, though. Have a good week.

[remkop]

Getting close but not yet.
See the 4.3 roadmap for the remaining items.
Some follow up items came out of the “inherited options” feature, like when to apply default and initial values, and an extra feature where inherited options are hidden (don’t show up in the usage help message) in subcommands.

I also took on some extra work helping out in the https://github.com/petermr/openVirus project which reduced the time I could spend on picocli.

[garretwilson]

Thanks for the update. There's no emergency need for it at the moment. I'll circle back in a week or two. Good luck.

[remkop]

Picocli 4.3 has been released.

Enjoy!

[garretwilson]

As luck would have it I hit a huge milestone a couple of days ago in one of my projects, so today I actually took a day off from my employer, intending to spend the morning in tranquility integrating the new changes. (Right away I ran into a new Eclipse regression bug so I should have known it wouldn't be that easy.)

Here is my original base Picocli application debug setting method, as you've no doubt seen many times:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
	this.debug = debug;
	updateLogLevel();
}

So with Picocli 4.3 I thought all I needed to do was to add an "inherit" scope:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.", scope = ScopeType.INHERIT)
protected void setDebug(final boolean debug) {
	this.debug = debug;
	updateLogLevel();
}

In CLI subclass I had this:

@Command(description = "Does foo and bar.", subcommands = {HelpCommand.class})
public void foobar(
		@Parameters(paramLabel = "<project>", description = "The base directory of the project being served.%nDefaults to the working directory, currently @|bold ${DEFAULT-VALUE}|@.", defaultValue = "${sys:user.dir}", arity = "0..1") @Nullable Path argProjectDirectory,
		…
		@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") final boolean debug) throws IOException, LifecycleException {

	setDebug(debug); //TODO inherit from base class; see https://github.com/remkop/picocli/issues/649

So I removed the debug parameter and the setDebug(), which I don't need anymore:

@Command(description = "Does foo and bar.", subcommands = {HelpCommand.class})
public void foobar(
		@Parameters(paramLabel = "<project>", description = "The base directory of the project being served.%nDefaults to the working directory, currently @|bold ${DEFAULT-VALUE}|@.", defaultValue = "${sys:user.dir}", arity = "0..1") @Nullable Path argProjectDirectory,
		…
		) throws IOException, LifecycleException {

Two little changes, and I should be good to go. But no, I get:

[ERROR] wrong number of arguments

I don't know where it's coming from. I don't know which arguments.

But interestingly if I actually add the --debug flag to the command line, it apparently does turn on debug mode so that I get a full stack trace (because my debug mode changes the log level). I get this:

java.lang.IllegalArgumentException: wrong number of arguments
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1872)
	at picocli.CommandLine.access$1100(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2243)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2237)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2201)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2068)
	at picocli.CommandLine.execute(CommandLine.java:1978)
	at com.globalmentor.application.BaseCliApplication.execute(BaseCliApplication.java:186)
	at com.globalmentor.application.AbstractApplication.start(AbstractApplication.java:118)
	at com.globalmentor.application.Application.start(Application.java:132)
	at io.guise.cli.GuiseCli.main(GuiseCli.java:88)

The stack trace makes me suspect that your code is calling the subcommand method and, since one of the options was inherited, the code thinks that it should be an argument to the subcommand method (here foobar()) and tries to pass it along, but Java complains because that extra parameter doesn't match the subcommand method signature.

But surely you tested this, so I don't know how this could happen.

In any case, I'm going to suspend work on this for the moment and try to get some tranquility this morning. Maybe I'll try to integrate the new injected Picocli CommandSpec you added in #629.

[remkop]

Away from PC now, will look into this soon.
Would you mind raising a separate ticket?

[remkop]

I raised #1042

This is a bug. Thank you for raising this!

[garretwilson]

@remkop I've integrated this into my application, and so far it is everything I dreamed it would be!

Admittedly from the outside it doesn't look like a lot technically: it was just a flag to say "inherit this option to subcommands". But I hope you realize how much nicer it makes my application by simply adding that simple scope = ScopeType.INHERIT and have that functionality be turned on for all my applications (by putting it in the base class).

This is really useful; thank you so much.

[remkop]

Thanks for that!
Positive feedback from happy users is what keeps me going. I’m sure it’s the same for other open source maintainers (so drop them a kind word when you get a chance 😉).

What would be really great is if you could do a blog post or article somewhere on how picocli has been useful for you. You’d be surprised how few people know that picocli even exists. Would be good if we could change that.

[garretwilson]

What would be really great is if you could do a blog post or article somewhere on how picocli has been useful for you.

I would absolutely love to!

I haven't been producing blog entries for a while because I'm been working feverishly to migrate my entire personal site to a new Cloud-distributed platform.

• But to do that I needed to write the static site generator (which I've been working on for a year and a half). The CLI of this uses Picocli.
• And to do that I needed to upgrade my metadata framework (which I've been working on for over a decade).

But the new site was just launched, and soon the static site generator and metadata framework will be ready for the next version release. Then I would be enthusiastic to spread the word!

If you don't hear anything further from me on this in about a month, by all means don't hesitate to ping me on this.

[palade]

@remkop I've integrated this into my application, and so far it is everything I dreamed it would be!

Admittedly from the outside it doesn't look like a lot technically: it was just a flag to say "inherit this option to subcommands". But I hope you realize how much nicer it makes my application by simply adding that simple scope = ScopeType.INHERIT and have that functionality be turned on for all my applications (by putting it in the base class).

This is really useful; thank you so much.

@remkop @garretwilson Do you have an example of where you used this? As I get CommandLine$DuplicateOptionAnnotationsException when I set scope = ScopeType.INHERIT in the base class?

If I have something like:
Option1 Option2 SubCommand Option3 -> Option1 and Option2 are ignored if I provide the list of arguments like this. However, if they are declared like:
SubCommand Option1 Option2 Option3 there are not.

The class of SubCommand extends the class of the Command (base class). How do I make Option1 and Option2 available when the order of arguments is specified as above

[remkop]

Why are we getting a DuplicateOption exception?

@palade I am guessing that the DuplicateOptionAnnotationsException occurs when an option is defined with scope = INHERIT in class A, and the application then defines another class B that is both a subclass of A as well as a subcommand of A.

When the application uses 2 reuse mechanisms (see sidebar below), the option is added to the subcommand twice... I will look at improving the error message to clarify this.

---

Sidebar:

Picocli now offers 3 methods of reuse:

• java class inheritance: options declared on a superclass are added to all subclasses
• mixins: declare a @Mixin annotated field to import all options declared in the mixin class
• inherited options: options with scope = INHERIT are added to all subcommands (and sub-subcommands)

---

Are options ignored sometimes?

If I have something like:
Option1 Option2 SubCommand Option3 -> Option1 and Option2 are ignored if I provide the list of arguments like this. However, if they are declared like:
SubCommand Option1 Option2 Option3 there are not.

The class of SubCommand extends the class of the Command (base class). How do I make Option1 and Option2 available when the order of arguments is specified as above

To summarize my understanding: this part of the question is not about using scope = INHERIT, but is about reusing options with Java class inheritance (please correct me if I'm wrong).
Option1 and Option2 are defined in a class, let's call it BaseClass. Then, both the top-level command and the SubCommand extend BaseClass (or perhaps the top-level command is BaseClass).

As a result, now the top-level command has Option1 and Option2 and SubCommand has Option1 and Option2. These are separate instances. So, when the user invokes Option1 Option2 SubCommand Option3, the top-level command's Option1 and Option2 are set, and the Subcommand's Option3 is set.

When you say "Option1 and Option2 are ignored", I am guessing you mean that the SubCommand's copy of Option1 and Option2 are not set. (If you check the top-level command, you will see that the top-level command's Option1 and Option2 are set.)

How to solve this?

So, now we understand what is happening, what shall we do to fix it?

Idea 1: make Option1 and Option2 "inherited options"

Two ways to do this:

• change SubCommand so it no longer extends BaseClass where Option1 and Option2 are defined
• change BaseClass so it no longer defines Option1 and Option2. Separate BaseClass from the top-level command. Something like this;

separate BaseClass from top-level cmd; move Option1/2 out of BaseClass into top-level cmd

                    +-----------+
                    | BaseClass |
                    +-----------+
                          |
                          |
                   +---------------+
                   |               |
                   V               V
+------------------------+    +-------------+
|        TopLevel        |    | SubCommand  |
|         Command        |    +-------------+
+------------------------+    
+ Option1 scope=INHERIT  +    
+ Option2 scope=INHERIT  +
+------------------------+    

This is especially useful if Option1 and Option2 are "standalone" options (like for configuring logging) that subcommands don't need to reference. If a subcommand do need to reference inherited options, it can use the @ParentCommand annotation.

@Command(mixinStandardHelpOptions = true, versionProvider = XX.class)
class BaseCommand {
}

@Command(name = "top", subcommands = SubCommand.class)
class TopCommand extends BaseCommand implements Runnable {
    @Option(name = "Option1", scope = INHERIT) boolean option1;
    @Option(name = "Option2", scope = INHERIT) boolean option2;

    public void run() {
        // business logic of top-level cmd...
    }
}

@Command(name = "sub")
class SubCommand extends BaseCommand implements Runnable {
    @ParentCommand TopCommand parent;

    public void run() {
        if (parent.option1) { doOption1Stuff(); }
        if (parent.option2) { doOption2Stuff(); }
        if (this.option3) { doOption3Stuff(); }
    }
}

Idea 2: Use the @ParentCommand annotation

Instead of using scope=INHERIT, an alternative is to keep the current mechanism of reusing options via java subclassing. In the business logic of Subcommand the application can check both its own copy of Option1 and Option2, as well as the Option1 and Option2 on the parent command. Something like this:

class BaseCommand {
    @Option(name = "Option1") boolean option1;
    @Option(name = "Option2") boolean option2;
}

@Command(name = "top", subcommands = SubCommand.class)
class TopCommand extends BaseCommand implements Runnable {
    public void run() {
        // business logic of top-level cmd...
    }
}

@Command(name = "sub")
class SubCommand extends BaseCommand implements Runnable {
    @ParentCommand TopCommand parent;

    public void run() {
        if (this.option1 || parent.option1) { doOption1Stuff(); }
        if (this.option2 || parent.option2) { doOption2Stuff(); }
        if (this.option3) { doOption3Stuff(); }
    }
}

Idea 3: Refactor Option1 and 2 into a mixin

Mixins allow composition; I prefer mixins over subclassing.

Mixins can also refer to the parent command; or they can be stand-alone.
Please see the manual sections on mixins for more details and several examples.

---

If you want to discuss further, please don't hesitate to create a separate ticket! :-)

[garretwilson]

@palade , I think I remember getting this error at first. My main command is the class Foo itself, and the option ("opt") is a method of that class, like setOpt(). The subcommand is a different method like bar(), and it had a parameter matching the command already marked as inherited; that is, bar(boolean opt) or something. So Picocli gave me this error because the option was duplicated.

That wasn't what I was expected. I had hoped that by using bar(boolean opt) on the subcommand it would override the inherited version of the option setOpt() (even though I might want to call setOpt() from within bar(opt)). But I actually didn't need to override the inherited option, so I didn't mention it and went about my work because I had lots more to do.

Here is the ticket I used to perform the conversion. It should (in the "Development" section) link to a couple of commits and a pull request that should show you clearly the changes I made:

https://globalmentor.atlassian.net/browse/JAVA-195

Oh, actually that is just changing the debug command to an inherited command in the base class. In the subclass CLI application, I had to remove the debug option as a parameter. That was done in this ticket:

https://globalmentor.atlassian.net/browse/GUISE-114

I think these are the important commits:

• https://bitbucket.org/globalmentor/globalmentor-application/commits/66f43dfe5ba47aa3e8a0bfcde3914790038c5656
• https://bitbucket.org/globalmentor/guise/commits/ec6028ff322a1fdadd80b724c746d2bab78b6234

Hope this helps.

[palade]

Thank you both for your replies.

The Option1 and Option2 in my case are still ignored. These are logging info options which could be placed anywhere in the command-line arguments.

Please note that the parse handler is run with the RunLast option. The sub-commands are added programmatically to the top command, and to each is passed the entire array of args. What I don't understand is why the position of these options are important and how to include them in the sub-commands.

[palade]

I see this post on SO that RunLast will invoke only the last command. Still wondering if there is a way to pass the Option1 and Option2 to the sub-command. Otherwise if it is difficult or not possible, would be a way to adjust the USAGE, because at the moment shows like:

topcommand [Option1] [Option2] [COMMANDS]

would be possible to adjust the USAGE to show the exclusive option:

topcommand ( [OPTION1] [OPTION2] | [COMMANDS])

[remkop]

@palade Can you create a new ticket, and in the description show a way for me to reproduce the problem? Either example code or a link to a public repository would work.

[palade]

@remkop Will attempt to modify the usage message as making the above changes would require a larger codebase change. Was wondering if there some examples where I could programatically create the following customSynopsys:

Usage topCommand Option1 Option2
       or topCommand Subcommand 

There are some examples in the manuals, but would like to create this programmtically. I had a look at the help test but it seems that the examples that I found were harding the usage message. Would be possible to create this programmatically (i.e., instead of using the topCommand use the actual name of the top - Command?

[remkop]

@palade I’d be happy to help you accomplish your use case.

I believe it would be useful to create a separate ticket: this ticket ticket is closed, already has 70+ comments, and your use case may result in changes to picocli, which would be easier to track in a separate ticket.

So please indulge me and raise a separate ticket.

Now, I’ve been making guesses about what your code looks like, and I would like to stop guessing. :-) When you mention “my Option1 and Option2 are still being ignored” I have no idea what you tried or what could be the cause.

Please help me help you, and take the time to explain

• Steps to reproduce the issue (there’s no substitute for example code here)
• What you’re actually seeing
• What you expected to see/would like to accomplish

So far I partially understand points 2 and 3 only.

[remkop]

One more thing: there’s no need to create a custom synopsis <cmd> ([Option1] [Option2] | Subcommand). Surely that’s not the solution you really want?

We can make it work so that users can specify these options either before or after the subcommand.

But please create a separate ticket first with your code, so we can discuss more concretely.