cmd/cgo: permits reference to non-existent struct

[ianlancetaylor]

This code compiles and runs:

package main

import "C"

import "fmt"

var V C.struct_X

func main() {
        fmt.Println(V)
}

It prints {}. That makes no sense. There is no type X here. It should give an error somewhere.

[ghost]

Also applies to non-existent unions, but not enums.

[gopherbot]

CL https://golang.org/cl/44774 mentions this issue.

[hirochachacha]

Once we refer C.struct_undefined, cgo emits __typeof__(struct undefined) *__cgo__0;.
The problem is that struct undefined doesn't have complete definition. We can check completeness of the type by __typeof__(typename) _[0];, because C prohibits array of an incomplete type.

I think one solid solution here is rejecting all incomplete types.
Incomplete types are sort of tentative placeholder, however we cannot complete those types on Go side. Thus I think exporting incomplete types to Go world have no semantic meanings.

That said, those changes brake some backward compatibility. So it may be controversial. I sent my CL, but please feel free to reject it :)

[bcmills]

That said, those changes [break] some backward compatibility.

Not just backward compatibility, but also compatibility with a wide variety of C APIs. It's fine to pass around pointers to an incomplete struct type: the problem occurs when you're trying to actually instantiate a struct of that type.

[bcmills]

But perhaps that implies that cgo shouldn't generate a Go type at all for an incomplete C type.

Instead of doing the equivalent of

type struct_x struct { _ [0]byte }

perhaps we could emit a Go struct type for *C.struct_x and rewrite pointers to values of that type:

type ptr_struct_x struct { _ unsafe.Pointer }

Assuming that the Go compiler generates correct code to copy structs with blank-identifier fields, that would allow Go code to copy values of type ptr_struct_x but not dereference them, achieving the same effect as a pointer to incomplete type in C.

[hirochachacha]

Not just backward compatibility, but also compatibility with a wide variety of C APIs. It's fine to pass around pointers to an incomplete struct type: the problem occurs when you're trying to actually instantiate a struct of that type.

The CL only rejects direct use of incomplete type in Go. It doesn't prohibit indirect use of incomplete type.
For example:

package main

/*
#cgo LDFLAGS: -L${SRCDIR} opaque.dylib
typedef struct PersonImpl *Person;
Person createPerson(int age);
void destroyPerson(Person p);
void printPerson(Person p);
*/
import "C"

func main() {
	p := C.createPerson(10)
	C.printPerson(p)
	C.destroyPerson(p)
}

will produce

type _Ctype_Person *_Ctype_struct_PersonImpl
type _Ctype_struct_PersonImpl struct{}

into _cgo_gotypes.go. This is the same behavior as current code. I confirmed that opaque pointer pattern is still working.

I roughly understand that 'incomplete type' means that the type cannot determine its own size at compile time. All pointers have the exact size (e.g. 8 byte on amd64), so 'pointer to incomplete type' aren't 'incomplete type' themselves.

But perhaps that implies that cgo shouldn't generate a Go type at all for an incomplete C type.

I'm not correctly sure what are you suggesting here. But if you are suggesting

type _Ctype_Person unsafe.Pointer

I agree with you. Current code have already done the similar decision for void* . It is the generalized decision of that.

[bcmills]

The CL only rejects direct use of incomplete type in Go. It doesn't prohibit indirect use of incomplete type.

If I'm reading it correctly, it would break cgo bindings for any API that provides "create" and "destroy" functions. (cbrotli is one such example.) Given that the existing cgo code is correct as written and that style is fairly common for C bindings (see also libsdl), I don't think that's acceptable.

But if you are suggesting
type _Ctype_Person unsafe.Pointer

No, I'm suggesting that we not require the Person typedef at all. To use your example:

package main

/*
#cgo LDFLAGS: -L${SRCDIR} opaque.dylib
#include "opaque.h"
*/
import "C"

func main() {
	p := C.createPerson(10)
	C.printPerson(p)
	C.destroyPerson(p)
}

opaque.h:

typedef struct Person Person;
Person* createPerson(int age);
void destroyPerson(Person* p);
void printPerson(Person* p);

In the cgo-translated code, p would have type _Ctype_ptr_Person, which would be defined as

type _Ctype_ptr_Person struct { _ unsafe.Pointer } 

That way, Go code could still copy and assign Person pointers directly, but it would be impossible to construct a non-nil *C.Person in Go code without explicit use of unsafe.

The one caveat to that approach is that we would also have to translate:

var p *C.person = nil

to

var p _Ctype_ptr_Person = _Ctype_ptr_Person{}

since nil is not a valid zero-value for a struct type (see also #19642 (comment)).

[hirochachacha]

If I'm reading it correctly, it would break cgo bindings for any API that provides "create" and "destroy" functions. (cbrotli is one such example.) Given that the existing cgo code is correct as written and that style is fairly common for C bindings (see also libsdl), I don't think that's acceptable.

I understand I was wrong. Thank you.

No, I'm suggesting that we not require the Person typedef at all. To use your example:

I cannot understand why do you need to wrap unsafe.Pointer in struct.
Is unsafe.Pointer assignable and copyable, isn't it?

[hirochachacha]

Anyway, I have no idea to solve this problem now.
I can reject var _ C.struct_undefined, but I cannot reject var _ *C.struct_undefined.

[bcmills]

I cannot understand why do you need to wrap unsafe.Pointer in struct.
Is unsafe.Pointer assignable and copyable, isn't it?

Yes, but it allows for some potentially-surprising conversions (see #20171).

[bcmills]

Thinking about this further, the type _Ctype_ptr_Person struct { _ unsafe.Pointer } approach I described above would not allow pointers to incomplete types to be converted to unsafe.Pointer. That seems like a fairly significant property to lose.

[rsc]

I don't see that there's an obvious fix here. The current behavior might be the best of the possible choices. In any event, not for Go 1.10.