proposal: Go 2: Destructuring operator to assign fields in different types of the structs

[rodcorsi]

It is normal to have a situation where we need to assign a struct using fields from another struct, an example is when the request or response type is different from the storage type. In these situations we can simply write:

data := DataModel {
  ID:    requestData.ID,
  Value: requestData.Value,
  ...
}

The problems start when we have a struct with many fields or new fields is added and we forgot to handle it.
We could use reflection to copy fields with the same name and type from a struct to another, but is slow e the code little complex

I'm trying to propose a language change to safely copy fields from a struct to another.
The idea is to use ... operator to copy the fields with the same name and type to another struct.

Example:

type RequestData struct {
  ID    string
  Value string
}

type DataModel struct {
  ID    string
  Value string
  Other bool
  AnyID int
}

Uses the RequestData fields to create a new instance of DataModel.
Important to observe that all fields need to be handled when using the ... operator.
In this case, the Other and AnyID need to be set.

requestData := RequestData{ID: "1", Value: "foo" }

data := DataModel{ Other: false, AnyID: 5, requestData... }
fmt.Printf("%#v\n", data) // DataModel{ID:"1", Value:"foo", Other:false, AnyID:5}

// it won't compile
// data := DataModel{ requestData... }

This example won't compile because DataModel has different fields from RequestData.
The fields Other and AnyID don't exist in RequestData.

requestData2 := RequestData{ data... } // it won't compile

It creates a new instance of DataModel but changes the field Value

data2 := DataModel{Value: "bar", data...}
fmt.Printf("%#v\n", data2) // DataModel{ID:"1", Value:"bar", Other:false, AnyID:5}

Embedded structs
When you use ... operator, the embedded struct fields will be handled as a normal field.

type A struct { i int }
type B struct { A }
type C struct { i int }

b1 := B{A{5}}
c1 := C{b1...} // c1 := C{i: b1.i}
b2 := B{c1...} // b2 := B{A{i: c1.i}}
fmt.Println(b1.i, c1.i, b2.i) // 5 5 5

Destructuring fields

type RequestWithExtraData struct {
  ID        string
  Value     string
  RequestID int
}
requestData := RequestWithExtraData{ "1", "foo", 5 }
RequestWithExtraData{ ID:id, Value:value } := requestData
fmt.Println(id, value) // 1 foo

Destructuring with rest.
The field RequestID will be ignored (_) and the rest of the fields it will assign the new DataModel instance, but is required to set Other and AnyID

requestWithExtraData := RequestWithExtraData{ ID:"3", Value:"extra", RequestID:42 }

RequestWithExtraData{
    RequestID: _, // ignore RequestID
    ...DataModel{
        Other: true,
        AnyID: 0,
    }: rest,
} := requestWithExtraData
fmt.Printf("%#v\n", rest) // DataModel{ID:"3", Value:"extra", Other:true, AnyID:0}

[ianlancetaylor]

Why not just make one type a field of the other type?

[rodcorsi]

We can embed a struct into another and reduce this problem, but it's not always that you have full control of the application. If you use a generator to create your model or if you need to pass configurations to a library or you are trying to connect a part of the application with another.

To understand this better I searched the pattern Field: any.Field, in some projects using this command:

grep -rnP --include \*.go "\s(\w+):\s+\w+\.\1[,}]" . | wc -l
prometheus 1527 occurrences
kubernetes 6982 occurrences
docker 1731 occurrences
minio 228 occurrences

[ianlancetaylor]

Thanks for looking at real code. Can you link to a couple of examples? That would help see what is going on there. Thanks.

[rodcorsi]

I want to show some real code examples:

• Kubernetes/pkg/scheduler/scheduler.go

Copy fields to different types of structs

return &Scheduler{
    SchedulerCache:      config.SchedulerCache,
    Algorithm:           config.Algorithm,
    GetBinder:           config.GetBinder,
    PodConditionUpdater: config.PodConditionUpdater,
    PodPreemptor:        config.PodPreemptor,
    Framework:           config.Framework,
    NextPod:             config.NextPod,
    WaitForCacheSync:    config.WaitForCacheSync,
    Error:               config.Error,
    Recorder:            config.Recorder,
    StopEverything:      config.StopEverything,
    VolumeBinder:        config.VolumeBinder,
    DisablePreemption:   config.DisablePreemption,
    SchedulingQueue:     config.SchedulingQueue,
}

Both types have exactly the same fields

return &Scheduler{config...}

• Prometheus/promql/engine.go

Copy and change a field of the same type of structs

newEv := &evaluator{
    endTimestamp:        ev.endTimestamp - offsetMillis,
    interval:            ev.defaultEvalInterval,
    ctx:                 ev.ctx,
    currentSamples:      ev.currentSamples,
    maxSamples:          ev.maxSamples,
    defaultEvalInterval: ev.defaultEvalInterval,
    logger:              ev.logger,
}

IMO short and keep the expressiveness

newEv := &evaluator{
    endTimestamp:   ev.endTimestamp - offsetMillis,
    ev...,
}

• Prometheus/discovery/openstack/openstack.go

Copy fields from type *SDConfig to gophercloud.AuthOptions

opts = gophercloud.AuthOptions{
    IdentityEndpoint:            conf.IdentityEndpoint,
    Username:                    conf.Username,
    UserID:                      conf.UserID,
    Password:                    string(conf.Password),
    TenantName:                  conf.ProjectName,
    TenantID:                    conf.ProjectID,
    DomainName:                  conf.DomainName,
    DomainID:                    conf.DomainID,
    ApplicationCredentialID:     conf.ApplicationCredentialID,
    ApplicationCredentialName:   conf.ApplicationCredentialName,
    ApplicationCredentialSecret: string(conf.ApplicationCredentialSecret),
}

Here we need explicitly ignore some fields of SDConfig
IMO this alternative is better because if new fields are created/updated/removed we need to handle it here

*SDConfig{
    Password:                    _,
    DomainName:                  _,
    DomainID:                    _,
    ApplicationCredentialSecret: _,
    Role:                        _,
    Region:                      _,
    RefreshInterval:             _,
    Port:                        _,
    AllTenants:                  _,
    TLSConfig:                   _,
    ...gophercloud.AuthOptions{
        Password:                    string(conf.Password),
        ApplicationCredentialSecret: string(conf.ApplicationCredentialSecret),
    }: opts, // the rest of field assign new instance of gophercloud.AuthOptions
} := conf

[beoran]

I like this idea, but how should this work for embedded structs? Does everything get copied recursively?

[rodcorsi]

I think the type embedding needs to be recursively copied different from a child struct

type Color struct {
    R byte
    G byte
    B byte
}

type Wheel struct {
    Size int
}

type Car struct {
    Wheel
    Color Color
}

type Vehicle struct {
    Size  int
    Color Color
}

func VehicleFromCar(car Car) Vehicle {
    // Expands all "Car" fields including "Wheel" fields
    // but "Color" fields won't expand
    // return Vehicle{
    //    Size:  car.Size,
    //    Color: car.Color,
    // }
    return Vehicle{bar...}
}

func CarFromVehicle(vehicle Vehicle) Car {
    // Expands all "Vehicle" fields to match with "Car" fields
    // a new instance of "Wheel" will be required
    // return Car{
    //     Wheel: Wheel{
    //         Size: vehicle.Size,
    //     },
    //     Color: vehicle.Color,
    // }
    return Car{vehicle...}
}

[ianlancetaylor]

If I understand this proposal correctly, changing a field name in one package can cause a silent behavior change in a different package. It could mean that the ... operation would stop, or start, assigning a field value. The same might be true for changing a field type. That seems like a problem. As a general guideline, supporting programming at scale means that changes to one package should not affect other packages, or they should affect other packages in an "obvious" way, or they should cause an error.

[rodcorsi]

The ... operation need to match with all fields, different fields need to be explicit ignored with _.
Changing a field in any package will cause an error.

Given the previous example, changing Vehicle to:

type Vehicle struct {
    Size  int
    Color Color
    Price float64 // new field
}

This function no longer compiles

func CarFromVehicle(vehicle Vehicle) Car {
    return Car{vehicle...} // now vehicle has Price field and this field don't exist in Car
}
// now is required to ignore the Price field
func CarFromVehicle(vehicle Vehicle) Car {
    Vehicle{Price:_, ...Car{}: rest} := vehicle
    return rest
}

In the current proposal this function continues to compile normally, but I agree with @ianlancetaylor this could introduce a bad behavior

func VehicleFromCar(car Car) Vehicle {  
    return Vehicle{car...}
}

I think we need to change the current proposal to allow the use of ... operator only if all fields are assign
With this change this function no longer compiles

func VehicleFromCar(car Car) Vehicle {  
    return Vehicle{car...}
}
// now is required add value to Price
func VehicleFromCar(car Car) Vehicle {  
    return Vehicle{
        Price: 0,
        car...,
    }
}

[ianlancetaylor]

Thanks, I managed to miss that.

[ianlancetaylor]

I just want to note that the newEv := &evaluator{ example in #33957 (comment) is not all that convincing, as it could be done with a simple assignment followed by a field change.

[ianlancetaylor]

The case data2 := DataModel{Value: "bar", data...} in the original comment seems potentially confusing. If I understand correctly, the presence of Value in the composite literal disables copying the Value field from data. That seems tricky.

Separately, consider

type S1 { i int }
type T { S1 }
type S2 { i int }
var v = S2{T{0}...}

Here the question is how the promoted field of T should be handled when using with .... Does it correspond to the normal field of S2?