gRPC c library versioning and leaks on the client side

[jamesthompson]

Firstly thanks a lot for this great work, we're really excited to be using it.

Recently we've been stress testing some services and have noticed what we think is some memory leakage in the underlying c code, but only on client requests (used from high level generated code), not seemingly on server handlers.

• I'm planning on doing some profiling, but I'm not an expert on debugging c from haskell, any advice on how I might go about diagnosing if my theory is correct would be greatly appreciated.

Having had a cursory look at the grpc c-lib releases over the last year, it seems there's been a bunch of memory leak fixes (pretty much all releases actually).

• We were wondering if there was any plan to try to upgrade this new core library to use the later c-lib versions?

I've just had a look at building this against v.1.8.3 but the c-api has changed a bit:

src/Network/GRPC/Unsafe.chs:228: (column 7) [ERROR]  >>> Unknown identifier!
  Cannot find a definition for `grpc_call_destroy' in the header file.
src/Network/GRPC/Unsafe.chs:145: (column 3) [ERROR]  >>> Function arity mismatch!
  Parameter marshallers are missing.

I'm not really au fait with the nuts and bolts of the low-level c interface, so I'm not sure I can immediately help in this endeavour, or indeed, if it might be desired at all.

Any comments from you folks on this would be greatly appreciated. Thanks in advance for reading and helping!

[Gabriella439]

@jamesthompson: Just to clarify: you're saying that the server is stable memory-wise, but a long-lived client is not?

Also, I think you can run valgrind on a Haskell binary (I have not tried this in a very long time, though, so I could be wrong)

We do plan on upgrading to a newer grpc. It's something I'm slowly working on

[jamesthompson]

@Gabriel439 That's correct, the server seems to run very happily with a small memory footprint, but long lived clients using withGRPCClient and the high-level generated code exhibits continually expanding memory consumption, despite there being multiple GC cycles (from an +RTS -s analysis).

I'll look into valgrind to see if I can get anything diagnosed.

[gbaz]

One question is if you occasionally exit the withGRPCClient and reinitiate that, if that helps...

[CMCDragonkai]

Any news on this?

[Gabriella439]

@CMCDragonkai: No news yet. I haven't had a chance to look into this and likely won't within the next month at least

[jamesthompson]

@CMCDragonkai I haven't been using this library for a while now and have recently been trying out https://hackage.haskell.org/package/http2-client-grpc and associated libs to great success. Sorry I can't be of more help right now.

[CMCDragonkai]

@jamesthompson we were using that library and suffered from really slow streaming bandwidth.