Calls to plugin hang when passing large amounts of data back and forth #14
Comments
|
ack there's something memory leak might be happening in host generated code, as I confirmed the following test code can succeed fast with var p TestPlugin
func Test_In_TinyGo(t *testing.T) {
ctx := context.Background()
const entrySize = 10000
// Initialize.
response, err := p.PassData(ctx, host.PassDataRequest{
Entries: entrySize,
Sequence: 0,
})
if err != nil {
t.Fatal(err)
}
if len(response.Data) != entrySize {
t.Fatalf("data size mismatch: %d != %d", len(response.Data), entrySize)
}
log.Printf("Size of data being passed %d (%d bytes)", len(response.Data), response.SizeVT())
for i := uint64(0); i < 20; i++ {
log.Printf("calling plugin %d", i)
start := time.Now()
response, err = p.PassData(context.Background(), host.PassDataRequest{
Sequence: i,
Entries: entrySize,
Data: response.Data,
})
log.Printf("call execution time: %s", time.Since(start))
if err != nil {
t.Error(err)
return
}
if len(response.Data) != int(entrySize) {
t.Errorf("expected %d entries, got %d", entrySize, len(response.Data))
return
}
}
} |
|
well, the test case is not enough to simulate the behavior -- I have to copy the entire |
|
Updated, though it's till finishing in a decent time vs go test package main
import (
"context"
host "github.com/cchamplin/plugin-bug/plugin"
"log"
"strings"
"testing"
"time"
)
var p TestPlugin
func Test_In_TinyGo(t *testing.T) {
ctx := context.Background()
const entrySize = 10000
// Initialize.
response, err := p.PassData(ctx, host.PassDataRequest{
Entries: entrySize,
Sequence: 0,
})
if err != nil {
t.Fatal(err)
}
if len(response.Data) != entrySize {
t.Fatalf("data size mismatch: %d != %d", len(response.Data), entrySize)
}
log.Printf("Size of data being passed %d (%d bytes)", len(response.Data), response.SizeVT())
for i := uint64(0); i < 20; i++ {
log.Printf("calling plugin %d", i)
start := time.Now()
req := &host.PassDataRequest{
Sequence: i,
Entries: entrySize,
Data: make(map[string]*host.State, entrySize),
}
for k, v := range response.Data {
state := &host.State{
FieldA: strings.Clone(v.FieldA),
FieldB: strings.Clone(v.FieldB),
FieldC: strings.Clone(v.FieldC),
FieldD: strings.Clone(v.FieldD),
FieldE: strings.Clone(v.FieldE),
FieldF: strings.Clone(v.FieldF),
FieldG: strings.Clone(v.FieldG),
FieldH: strings.Clone(v.FieldH),
FieldI: strings.Clone(v.FieldI),
FieldJ: strings.Clone(v.FieldJ),
FieldK: strings.Clone(v.FieldK),
FieldL: strings.Clone(v.FieldL),
FieldM: strings.Clone(v.FieldM),
FieldN: strings.Clone(v.FieldN),
}
req.Data[k] = state
}
response, err = p.PassData(context.Background(), *req)
log.Printf("call execution time: %s", time.Since(start))
if err != nil {
t.Error(err)
return
}
if len(response.Data) != int(entrySize) {
t.Errorf("expected %d entries, got %d", entrySize, len(response.Data))
return
}
}
} |
|
I am pretty confident that this is an TinyGo issue. Could you try https://github.com/wasilibs/nottinygc to see problems will be gone? |
|
Just stumbled across this, was wondering if anyone ever got this figured out @mathetake @cchamplin ? Looking to use this library with Wazero in a data intensive application, with plugins written in TinyGo, but want to see if this got resolved. |
|
@brennanjl because large amounts of data have to be re-serialized into protobuf each time, it will be difficult to support with this model I think. First, really consider what @mathetake said because I have seen one project drop 1/2 its overhead by using https://github.com/wasilibs/nottinygc After that, if you still have issues, I think the choices are..
|
When calling into plugins the behavior is not consistent depending on how much data is being passed. For small amounts of data everything seems fine. For larger amounts of data each successive call gets slower. For even larger amounts of data only one or two calls is possible before the callee hangs indefinitely. When the hang occurs it seems like we never make it into the plugin function, so it appears to be an issue with memory allocation or garbage collection within in the wasm boundary.
Version: f7d9444
Reproducible Test: https://github.com/cchamplin/plugin-bug
The text was updated successfully, but these errors were encountered: