visibility of private use in submodules using parent

[mppf]

For a regular variable, making the variable private will allow submodules to see it (even if the parent module must be imported to be available). But making a variable available with a private use will not allow submodules to have access to the same variable even if they use the parent module (after PR #15312).

This comes down to a question of how to treat private in private use/private import (which is the default). There are two approaches:

1. private use is like private var x in that submodules can access it but other modules cannot
2. private use never makes the used symbols available outside of the module and it is the same for submodules.

For example:

module parent {
  module child {
    proc secretFunction(a: int) {
      return a*3;
    }
  }
  private var parentPrivateVar = 1;
  private proc parentPrivateFn() { return 2; }
  private use child;

  module sibling {
    use parent; // does this bring in secretFunction or not?
    proc main() {
      writeln(parentPrivateVar); // ok
      writeln(parentPrivateFn()); // ok
      writeln(secretFunction(11)); // error? or ok?
    }
  }
}

The above program compiles if it is public use child. Is the author of the parent module required to make a symbol from a submodule public in order to make it available to submodules?

Associated Future Test(s):

test/visibility/private/moduleSymbols/accessPrivateUsedSibling2.chpl #15312
test/visibility/private/moduleSymbols/accessPrivateUsedSibling2a.chpl #15312

[bradcray]

I prefer not giving sibling access to symbols from child that parent received access to via a private use or private import (option 2) for the following reasons:

• Children still have the ability to access their sibling modules' symbols via their own use or import statements. By contrast, if we didn't permit a child to see its parent's private symbols, there's no recourse short of making them public to everyone. See my rewrite of your program below for an example.
• It permits the parent module to access descendent symbols that it needs for its implementation without necessarily also exposing them to all its children that use/import it. The result is that code can be written in a bit more of an airtight manner ("why leak symbols to a module that it doesn't want or need?").
• It treats the use/import symmetrically for all uses of parent whether they are external to the module or internal ("fewer special cases in the compiler code base")
• Relaxing this rule over time seems less likely to break code than introducing it later would.

module parent {
  module child {
    proc secretFunction(a: int) {
      return a*3;
    }
  }
  private var parentPrivateVar = 1;
  private proc parentPrivateFn() { return 2; }
  private use child;

  module sibling {
    use parent; // does this bring in secretFunction or not?
    proc main() {
      import super.child.secretFunction;  // added this: I need to access some of my sibling's symbols
      writeln(parentPrivateVar); // ok
      writeln(parentPrivateFn()); // ok
      writeln(secretFunction(11)); // error? or ok?
    }
  }
}

[lydia-duncan]

I prefer approach 1. I think the organization of submodules is more likely to reflect similarities between the symbols within the submodule, rather than necessarily that they should be harder barriers. Consider the following scenario:

module Parent {
  private module HelperFuncs {
    proc x(): int { ... }
  }
  private use HelperFuncs;
  public module ImportantSub {
    proc usefulFunc() {
       var res = x();
       ...
    }
  }

  module ImportantSub2 {
      proc usefulFunc() {
         var res = x();
         ...
      }
  }
}

In this scenario, HelperFuncs is a nice place to put all the helper functions required by the parent and its submodules. In this case, you'd want a private use of the helper at the parent's scope so you don't have to write a use or import in every single submodule.

[bradcray]

I think the organization of submodules is more likely to reflect similarities between the symbols within the submodule

I'm not sure I agree with this perspective. For example, I think it's been proposed that all standard modules should potentially be submodules of a module named standard or std, but that wouldn't imply that there's a similarity between Map and Regexp just because they share a parent module.

My concern with your example-based argument is that it focuses more on convenience than on expressiveness / safety. If private use means the same thing for child vs. external modules (case 2) then it becomes possible for a parent to use/import symbols for its own sake without that action imposing symbols on their children in any way. This feels like a powerful pattern while also potentially reducing the number of unnecessary scopes that have to be checked when resolving a child module's symbols. [It also means that each module only re-exports a single, consistent interface to any other module.] If private use instead means that children can refer to any symbols that the parent can then there is no way for a parent to express this pattern ("I want to watch Game of Thrones, but I don't want that to mean that my children should necessarily be able to without taking the necessary actions themselves.")

The inconvenience in your example of having each child module say use super.HelperFunc(); doesn't seem that much worse than the fact that they'd each have to each say use Parent; to get to the symbol in the indirect / case 1 world anyway. If the inconvenience proved too great for the module author, they could choose to declare private proc x() within Parent rather than placing it in a submodule. I'd argue that a major reason for putting it into a [private] submodule would be to place it in its own namespace; if the intention isn't to have anyone use/import that namespace to get at its symbols, maybe they should've remained at the top level?

[lydia-duncan]

But distinguishing between "I want this to be visible to outer modules" and "I only want this to be visible to my submodules" seems useful. Does that imply that we should look at ways to distinguish between these strategies (e.g. protected use)?

[bradcray]

I'm arguing that private module-scope code is an appropriate way to say "I only want this to be visible to my submodules" and that there's no additional semantic benefit to pushing such code into a submodule if nobody has to actually name that submodule to get at it.

[lydia-duncan]

There's the mental benefit of having it neatly filed away in a module for general helper functions as opposed to scattered through out the parent module, or arbitrarily sorted to the beginning or end of it. It all comes down to code organization preferences.

I probably should have said that I have occasionally used submodules as a way to sort common helper routines for a library

[bradcray]

To physically separate a set of routines without putting them into their own namespace, maybe what you want is the ability to include other files in a way that's independent of submodules?