cmd/cgo: _GoStringLen/_GoStringPtr not available with exported functions in c-archive or c-shared modes

[theimpostor]

What version of Go are you using (go version)?

$ go version
go version go1.17.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/shoda/Library/Caches/go-build"
GOENV="/Users/shoda/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/shoda/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/shoda/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.17.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.1/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ct/x2gct7yn2bxfqs891n8h1dxr0000gn/T/go-build205412794=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Using this program:

package main

/*
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
void dumpGoString(_GoString_ s) {
    size_t len = _GoStringLen(s);
    const char * sp = _GoStringPtr(s); // may not be NULL terminated
    char * str = strndup(sp, len);
    fprintf(stderr, "go string: %s\n", str);
    free(str);
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}

What did you expect to see?

Doing go build -buildmode=c-archive -o gostring.a main.go should produce a library that is callable from C.

What did you see instead?

❯ go build -buildmode=c-archive -o gostring.a main.go
# command-line-arguments
In file included from _cgo_export.c:4:
main.go:8:18: error: implicit declaration of function '_GoStringLen' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
main.go:9:23: error: implicit declaration of function '_GoStringPtr' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
main.go:9:23: note: did you mean '_GoStringLen'?
main.go:8:18: note: '_GoStringLen' declared here
main.go:9:18: warning: incompatible integer to pointer conversion initializing 'const char *' with an expression of type 'int' [-Wint-conversion]

If I remove the line //export PrintWithC, the app runs fine:

❯ go run main-no-export.go
go string: 42

Also go build -buildmode=c-archive -o gostring.a main-no-export.go succeeds but there is no header file so not sure how I would invoke from C.

Ultimately what I would like to do is to be able to declare void dumpGoString(_GoString_ s); in main.go, build main.go as an archive or shared library, and define dumpGoString in a separate C library which is able to call _GoStringLen and _GoStringPtr on the passed in _GoString_ s from golang. I assumed this was possible from reading the cgo doc, but perhaps I am missing something.

[theimpostor]

Also, if I forward-declare _GoStringLen and _GoStringPtr to resolve the C compiler errors, I get undefined symbol errors later:

package main

/*
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
size_t _GoStringLen(_GoString_ s);
const char *_GoStringPtr(_GoString_ s);
void dumpGoString(_GoString_ s) {
    size_t len = _GoStringLen(s);
    const char * sp = _GoStringPtr(s); // may not be NULL terminated
    char * str = strndup(sp, len);
    fprintf(stderr, "go string: %s\n", str);
    free(str);
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}

❯ go build -buildmode=c-archive -o gostring.a main.go
# command-line-arguments
Undefined symbols for architecture x86_64:
  "__GoStringLen", referenced from:
      _dumpGoString in _x001.o
  "__GoStringPtr", referenced from:
      _dumpGoString in _x001.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[mknyszek]

The cgo docs indicate that these are only available in the C preamble of .go files.

I think that means that no, these symbols will not be available as part of a c-archive build. I'm not sure how you're supposed to make this work, though, or if it's possible at all.

CC @ianlancetaylor @aclements ?

[ianlancetaylor]

Quoting https://golang.org/cmd/cgo:

Using //export in a file places a restriction on the preamble: since it is copied into two different C output files, it must not contain any definitions, only declarations. If a file contains both definitions and declarations, then the two output files will produce duplicate symbols and the linker will fail. To avoid this, definitions must be placed in preambles in other files, or in C source files.

You have a definition in your C preamble, and that is not supported when using //export. Sorry.

[theimpostor]

You have a definition in your C preamble, and that is not supported when using //export. Sorry.

Thanks, you are correct.

Ultimately I think I was able to get this work by:

1. Following the advice here about C calling Go calling C
2. Using 'static inline' to be able to define code in the preamble and export at the same time, as demonstrated in the wiki
3. Manually declaring _GoStringLen and _GoStringPtr in the preamble.

Seems like 2 would be good to mention in the cgo documentation, and 3 should be done automatically?

Here is what I ended up with, which allows me to access a Go string from C with no extra allocations:

main.go:

package main

/*
#cgo linux LDFLAGS: -Wl,-unresolved-symbols=ignore-all
#cgo darwin LDFLAGS: -Wl,-undefined,dynamic_lookup
extern void dumpCString(const char *s, size_t length);
size_t _GoStringLen(_GoString_ s);
const char *_GoStringPtr(_GoString_ s);
static inline void dumpGoString(_GoString_ s)
{
    dumpCString(_GoStringPtr(s), _GoStringLen(s));
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}

main.c:

#include <stdio.h>

#include "libgostring.h"

void dumpCString(const char *s, size_t length)
{
    printf("go string: %.*s\n", (int)length, s);
}

int
main(int argc, char** argv)
{
    PrintWithC(42);
}

❯ go build -buildmode=c-archive -o libgostring.a main.go
❯ clang libgostring.a main.c
❯ ./a.out
go string: 42

[ianlancetaylor]

As far as I can tell _GoStringLen and _GoStringPtr are declared in the preamble--but only if you don't use //export. The static inline trick can often work but there is no intention of officially supporting it.