proposal: Go 2: allow packages to implement interfaces using .(package) syntax

[jakoblorz]

Proposal

Given the following interface and function:

type Foo interface {
    Bar(string) (string, error)
}

func Call(f Foo) {
    bar, err := f.Bar("foo")
}

A package itself would implement / satisfy the interface Foo if it exported a Bar(string) (string, error) function:

package buz;

func Bar(s string) (string, error) {
    return s, nil
}

I propose a backwards compatible syntax extension similar to the .(type) syntax:

package main

import ".../buz"

type Foo interface {
    Bar(string) (string, error)
}

func Call(f Foo) {
    bar, err := f.Bar("foo")
}

func main() {
    Call(buz.(package))
    // package is an already existing keyword in the go language, similar to type. Because of that,
    // no struct or interface will be named "package", which allows this otherwise ambiguous construct
}

I believe that this extension works well with the composition paradigm as a package is a composition of files. Optimally, the exported functions are pure of some form, so that the behavior of a given package-level implementation can easily be determined.

Example

Access to functions of a default instance without having actually access to that instance:

type Registerer interface {
    Register(interface{}) error
}

func main() {
    var srv Registerer
    if (os.Getenv("default") != "") {
        srv = rpc.(package)
    } else {
        srv = rpc.NewServer()
    }
}

I find the proposed change intuitive and suitable for my needs. I hope it aligns with the direction golang is taking and that more people find it suiting their needs 😅. Thank you for your time.

I love golang and thank all previous contributors for their amazing work.

[DeedleFake]

It's an interesting idea, but I'm not sure what the point is. Why not just export a global instance of a type, possibly satisfying an interface, such as http.DefaultClient?

[jakoblorz]

Sure, in the simpler cases (net/http, net/rpc), this change does not yield a huge improvement, just some kind of workaround. Exporting the default, global instance might be the better option (I think I will edit the Example section)

What this proposal is meant to be is to improve generality

If a type exists only to implement an interface and will never have exported methods beyond that interface, there is no need to export the type itself. Exporting just the interface makes it clear the value has no interesting behavior beyond what is described in the interface. It also avoids the need to repeat the documentation on every instance of a common method.

Commenting on that: But why directly export a specific implementation? http.ServeMux is even a specific case, not an interface!
Of course, the exported global/default instance could be of an interface type, yet naming the default instance is restricted only through convention. It differs in each package.
I believe implementing the proposal would improve generality over all.

Let take the example from Effective Go that would follow the quote above.

The crypto/cipher interfaces look like this:

type Block interface {
    BlockSize() int
    Encrypt(src, dst []byte)
    Decrypt(src, dst []byte)
}

type Stream interface {
    XORKeyStream(dst, src []byte)
}

Any Algorithm that satisfies Block can be used in

func NewCTR(block Block, iv []byte) Stream

With the proposal implemented, all algorithms could provide ready-to-use, default instances with sane default values, where accessing the default instance is the same in all cases:

var block Block
switch alg {
case "aes":
    block = aes.(package)
case "des":
    block = des.(package)
}

var algorithm cipher.Block
candidates := []interface{}{rsa.(package), aes.(package), des.(package)}
for _, cand := range candidates {
    if alg, ok := cand.(cipher.Block); ok {
        algorithm = alg
        break
    }
}

[deanveloper]

Consider the following statement in the current generics proposal:

One consequence of the dual-implementation constraint is that we have not included support for type parameters in method declarations.
...
The problem here is that a value of type Set(int) would require an infinite number of Apply methods to be available at runtime, one for every possible type U, all discoverable by reflection and type assertions.

What you have essentially done is turn package-level functions into methods on the package; the (current) generics draft would be unable to work, assuming that pack.(package) is an expression that returns a useful value, which if it isn't, then I really don't like this proposal...

So I have a few questions:

1. Is pack.(package).PackageLevelFunction() (no matter how useless) valid?
2. What about pack.(package).PackageLevelValue?
   • If that isn't valid, then 1 should not be valid either
3. What would fmt.Println("%T", pack.(package)) print?
4. Would then I be able to use reflect to discover all functions/values that a package defines?
5. Would const values be discoverable?
6. (much different) How do you think this would impact the http.Client, rand.Rand, (etc) practice? It's generally a good practice, and I feel like this proposal would discourage package maintainers from using it

[jakoblorz]

Clarification

Every package defines an interface by default and automatically, which contains the signatures of all exported package-level functions.
pack.(package) returns this interface. This is not supposed to replace the current direct package access.

To your questions:

1. Is pack.(package).PackageLevelFunction() (no matter how useless) valid?

Yes, following the logic above, pack.(package).PackageLevelFunction() would be valid.

2. What about pack.(package).PackageLevelValue?
   • If that isn't valid, then 1 should not be valid either

Because an interface is returned, pack.(package).PackageLevelValue would not be valid. Could you please clarify why pack.(package).PackageLevelFunction() should then be invalid? I mean, if you want to access package-level values, you still can in the same way as before. Sorry if that was not clear, pack.(package) is not meant to replace the current way of accessing elements on the package-level.

3. What would fmt.Println("%T", pack.(package)) print?

Here I'm unsure, quiet honestly. It could be simply evaluated to "pack".

4. Would then I be able to use reflect to discover all functions/values that a package defines?

Optimally, you would only discover exported functions/values. That includes const. But I think the normal behavior of the reflect package is to be expected.

6. (much different) How do you think this would impact the http.Client, rand.Rand, (etc) practice? It's generally a good practice, and I feel like this proposal would discourage package maintainers from using it

Again, http.Client, rand.Rand, (etc) should keep working just like they do now. This proposal would not pose a breaking change as far as I know. Furthermore, I am unsure what your question is about - do you mean specifically the initialization of a default instance as an exported, for example in http.Client, or did you mean the package design as a whole?

rand.Rand then is the perfect example for my idea. Here you can see, that there is already a private, global rand.Rand instance (even called globalRand). The package-level functions themselves use it. Calling rand.(package) would return an interface containing the package-level functions and hereby creating an interface that any rand.Rand instance coming from rand.New() would satisfy.

Generally speaking, this change might encourage package maintainers to follow the Generality Directive more closely.

Thank you @deanveloper for your questions, they made me think about the scope more thoroughly :simple_smile:
I still believe it is a viable idea.

Regarding the generics proposal (omg what?) - I believe you mean this one: Sorry, but I am unfamiliar with it. If I find the time, I will take a look at it.

[deanveloper]

pack.(package) returns this interface.

I didn't quite understand that pack.(package) returned an interface, and that changes my perspective a bit. I have a few more questions now, although I'll have them at the bottom since I have a few more comments on this post

Because an interface is returned, pack.(package).PackageLevelValue would not be valid. Could you please clarify why pack.(package).PackageLevelFunction() should then be invalid?

Because functions can be seen as values as well, my bad as I didn't understand that it was an interface that was returned again.

Optimally, you would only discover exported functions/values. That includes const. But I think the normal behavior of the reflect package is to be expected.

Well if its an interface, again I have some more questions... also my question becomes irrelevant 😅

Again, http.Client, rand.Rand, (etc) should keep working just like they do now. This proposal would not pose a breaking change as far as I know. Furthermore, I am unsure what your question is about - do you mean specifically the initialization of a default instance as an exported, for example in http.Client, or did you mean the package design as a whole?

I mean the practice of providing a rand.Rand or log.Logger or whatever other thing you have. I've had to use a few badly-designed packages which only allow one "instance". Not sure how to explain it, but imagine using log but not having log.Logger, so you could only ever have one logger. One of the benefits of providing a log.Logger, other than allowing you to have more than one logger, is that you can pass around a log.Logger as a value, which you can't do with a package.

What I was wondering with my original question was if this feature would add another reason not to provide the equivalent of a log.Logger. For instance, I tell my friend that they should provide a pack.Packer interface and a pack.NewPack() pack.Packer function. Allowing the package to be passed around as a value is one less reason to make those constructs, which are generally good things to have, so that you can have multiple instances of the pack.Packer functionality.

I feel like this is a very unlikely situation that I'm questioning the validity of as I'm writing it, but I'm leaving it included anyway, haha

Generally speaking, this change might encourage package maintainers to follow the Generality Directive more closely.

My previous comment was more trying to say the opposite, that it may encourage people not to do things like making a private, global rand.Rand instance

Also, the (better) link for the Go2 draft is here, which contains a link to the documents relating to the generics draft (please remember that these are only drafts, and are not final in any way, and are not official proposals)

---

Anyway, onto my new questions

1. Would &pack.(package) make sense?
2. What would be the underlying type of the interface be?
3. Essentially the same question as the last one, but what would reflect.TypeOf(pack.(package)) return?
4. What about reflect.ValueOf(pack.(package))?
5. Would this make all functions actually just methods on pack.(package)?
6. If so, do we really have a reason to call them functions anymore, other than that we prefix them with func?

With these questions, what I'm getting at is that interfaces are a bit more than just a "set of methods", they have real types and values underneath them. Perhaps we could have a package pack type for each package, although it seems weird to have a type that you can only create by declaring a package.

I know I'm criticizing this pretty hard, I don't mean for it to be in any way. I'm usually pretty critical of "synthetics" (types/values that are just magically created for specific purposes). I personally think that instead, you should just create your own interface, match your package to the interface, and provide a package-level variable if you want to pass it around as a value, but that's just my opinion

[ianlancetaylor]

I'm concerned that this proposal would mean that the linker could not remove any exported functions from the generated binary, because it would not be able to be certain that the function was not referenced via an interface obtained via .(package). That would be bad.

[bcmills]

I'm concerned that this proposal would mean that the linker […] would not be able to be certain that the function was not referenced via an interface obtained via .(package).

That's already true for interface methods in general (see #25081), and unused interface methods can also pull in arbitrarily large graphs of package-level functions.

It's certainly possible that .(package) could exacerbate that problem, though.

[bcmills]

Another option might be to construct a minimal interface rather than a maximal one, including only the functions needed to satisfy the interface rather than all functions. (Failing to make some functions available via type-assertion seems better than pinning all of the code reachable from package-level functions.)

For example, we could use the make function (which already uses its first parameter to name the target type) rather than the package keyword:

type Registerer interface {
    Register(interface{}) error
}

func main() {
    var srv Registerer
    if (os.Getenv("default") != "") {
        srv = make(Registerer, rpc)
    } else {
        srv = rpc.NewServer()
    }
}

var block Block
switch alg {
case "aes":
    block = make(Block, aes)
case "des":
    block = make(Block, des)
}

If we wanted to extend that pattern further, we could employ the same syntax to convert functions to interfaces (#21670) or to promote the fields of struct types to methods. Combined with #21496, it would also provide a clean syntax for interface literals (#25860). Using @neild's example from #21670 (comment):

type io interface {
	Read(p []byte) (n int, err error)
	ReadAt(p []byte, off int64) (n int, err error)
	WriteTo(w io.Writer) (n int64, err error)
}

func ReaderWithContext(ctx context.Context, r Reader) io.Reader {
	[…]
	wrap := make(io, {
		Read: ctxr.Read,
		ReadAt: nil,
		WriteTo: writeToFn,
	})
	return wrap
}

[metakeule]

We could change the plugin mechanism to make use of such package interfaces, i.e.
a plugin would need to fullfill an interface when being imported.

That would make the contract between the main binary and the plugins more clear and visible and
plugins could easily be moved back to the main (or vice versa) without chaning much code.

[hzmsrv]

seem you all just need dynamic lib like DLL, SO, not platform limit.
and it only supports Golang. written by go loading by Golang, could be encrypted.published.

most like
pascal dcu file
python pyc file
Win dll file
linux so file

[ianlancetaylor]

The mechanism to create objects that satisfy an interface is defining new types with methods. Packages have a different function: they provide encapsulation and modularity. There is no mechanism to create multiple instances of a single package. In Go, packages are not values. We shouldn't start treating them as though they are.