Memory leak with go-wasm-adapter/go-wasm

[BruceChoca]

Thanks for the bug report!

Describe the bug

I referenced "github.com/go-wasm-adapter/go-wasm" and updated it to support wasmer-go 1.0+.
When I create an instance with bridge(go-wasm) and execute it in batches, the memory is ever increasing.
I try to optimize go-wasm, but the effect is not good, the problem still exists.
Can you help me locate the cause?

Steps to reproduce

1. Go to github.com/BruceChoca/go-wasm, See "TestDemo"

package memory

import (
	"context"
	"encoding/json"
	"errors"
	"io/ioutil"
	"log"
	"runtime"
	"strconv"
	"sync"
	"testing"
)

const (
	FAILURE = 500
	SUCCESS = 200
)

type Response struct {
	Result  int    `json:"status"`
	Message string `json:"message"`
	Body    []byte `json:"body"`
}

func TestDemo(t *testing.T) {
	codebuf, _ := ioutil.ReadFile("./wasm/main.wasm")

	r, err := Start("test", codebuf)
	if err != nil {
		panic("Start Error")
	}

	// batch
	count := 30000
	wg := sync.WaitGroup{}
	wg.Add(count)
	for i := 0; i < count; i++ {
		go func(index int) {
			err := r.Invoke(index)
			if err != nil {
				println(err.Error())
			}
			wg.Done()
		}(i)
	}

	wg.Wait()

	println("done")

	r.runtime.Release()

	runtime.GC()

	select {}
}

type R struct {
	runtime Runtime
	lock    sync.Mutex
}

func Start(name string, codebuf []byte) (*R, error) {
	runtime, _ := RuntimeFromBytes("test", codebuf, nil)

	// import Call
	runtime.SetFunc("Sys_Call", func(args []interface{}) (i interface{}, e error) {
		// get params
		funcName, _ := String(args[0])
		byteArgs, _ := Bytes(args[1])

		byteData := [][]byte{}
		err := json.Unmarshal(byteArgs, &byteData)
		if err != nil {
			resByte, _ := json.Marshal(Response{
				Result:  FAILURE,
				Message: "args Unmarshal error, Error: " + err.Error(),
			})
			return FromBytes(resByte), nil
		}

		switch funcName {
		case "Logln":
			args := []interface{}{}
			json.Unmarshal(byteData[0], &args)
			log.Println(args...)
			return FromBytes(nil), nil
		default:
			log.Println(args...)
			return FromBytes(nil), nil
		}
	})

	// start
	init := make(chan error)
	ctx, cancF := context.WithCancel(context.Background())
	runtime.CancF = func() {
		runtime.ClearInstance()
		cancF()
	}

	go runtime.Run(ctx, init)

	return &R{
		runtime: *runtime,
	}, <-init
}

func (r *R) Invoke(index int) error {
	r.lock.Lock()
	defer r.lock.Unlock()

	res, err := r.runtime.CallFunc("logFunc", []interface{}{"test log " + strconv.Itoa(index)})
	if err != nil {
		return errors.New("CallFunc Error: " + err.Error())
	}
	resByte, err := Bytes(res)
	if err != nil {
		return errors.New("Response Convert Bytes Error: " + err.Error())
	}

	var response = Response{}
	json.Unmarshal(resByte, &response)

	println("invoke", index, ":", response.Result, response.Message)

	return nil
}

2. Run TestDemo
3. See error

GOOS=js GOARCH=wasm go build -o main.wasm ./wasm/main.go

Expected behavior

I expect the memory usage to grown a bit and then even out to a stable number.

Actual behavior

The memory usage keeps increasing with invoke.

[Hywan]

Hello,

So, you're compiling a Go program to Wasm, to execute it in Go. OK.

First, why doing that :-)? I'm curious.
Second, maybe it's a leak in your Go program, not in wasmer-go. Can you try with #277 please?

[BruceChoca]

Hello,

First, I want to make a tool to run go programs by wasm.
Second,
I think the leak may occur in the following parts:

1. Runtime

I update "syscall/js.finalizeRef" and optimized it.

func finalizeRef(env interface{}, v []wasmer.Value) (out []wasmer.Value, err error) {
	sp := v[0].I32()

	b := getRuntime(env)
	// log.Println("finalizeRef: ", len(b.valueMap), ", refs: ", len(b.refs), ", refcout: ", len(b.refCount))
	id := int(b.getUint32(sp + 8))

	b.valuesMu.RLock()
	b.refCount[id]--
	if b.refCount[id] == 0 {
		v, ok := b.valueMap[id]

		if ok {
			rt := reflect.TypeOf(v)
			if rt.Kind() == reflect.Ptr {
				rt = rt.Elem()
			}
			rv := v
			if !rt.Comparable() {
				rv = reflect.ValueOf(v)
			}

			delete(b.refs, rv)
			delete(b.valueMap, id)
			delete(b.refCount, id)
			// b.idPools = append(b.idPools, id)
		}
	}
	b.valuesMu.RUnlock()

	return
}

The end of demo, I run Release(), set them "nil"

func (b *Runtime) ClearInstance() {
	b.instance = nil
	b.valueMap = nil
	b.refs = nil
	b.valueIDX = 8
	b.refCount = nil
	b.memory = nil
}

2. Wasm Memory, I called "Grow"

func (b *Runtime) mem(offset int32) []byte {
	m, _ := b.instance.Exports.GetMemory("mem")
	if b.memory == nil {
		b.memory = m.Data()
	}

	if len(b.memory) <= int(offset) {
		m.Grow(1)
		b.memory = m.Data()
	}

	return b.memory
}

3. Wasmer-go

I can't judge the real cause.
my go program referenced "github.com/go-wasm-adapter/go-wasm",
I also tested "github.com/go-wasm-adapter/go-wasm", it also has similar problems.
I saw this 8#issue go-wasm-adapter/go-wasm

So, can you help me find the reason?

Hello,

So, you're compiling a Go program to Wasm, to execute it in Go. OK.

First, why doing that :-)? I'm curious.
Second, maybe it's a leak in your Go program, not in wasmer-go. Can you try with #277 please?

[Hywan]

I'm sorry but you should open a PR on go-wasm-adapter/go-wasm, not this repository. I'm not the owner of the go-wasm project. I proposed my help to migrate to wasmer-go 1.0, but the discussion must not happen here :-).

I'm closing the issue. Please reopen it at https://github.com/go-wasm-adapter/go-wasm/. Thank you.