Express + PG Streamed response hangs

[ngauthier]

Hi folks,

This issue occurred starting with 1.7.0 (I'm assuming from the pg instrumentation). It only occurs with streaming queries and responses, not regular queries.

We're using express and knex, but with knex in this case it's a raw query knex.raw(QUERY).stream() with pipeline into express's res response object.

I was going to make a failing test case but I noticed there are no tests for the pg instrumentation.

The result of this bug is that requests hang until they time out upstream and are cancelled.

[ngauthier]

Additionally, (and this is probably a separate issue, but not sure). These streamed responses don't seem to have the correct duration_ms. Even using async/await for the express handler it seems honeycomb considers them complete as soon as they start streaming, not when they finish.

[allyjweir]

Hi @ngauthier, testing of these instrumentations isn't particularly easy. When developing the pg instrumentation I developed a small Express app that would run it through its paces. Manual testing sucks but sure is better than nothing.

Perhaps you could submit a PR to that project including your new breaking test? If you can provide a test case then I'd be more than happy to take a look and try fix the problem!

In terms of a short-term workaround, I'd recommend disabling the pg instrumentation when you initialise beelines. Here is a snippet from the beeline docs:

const beeline = require("honeycomb-beeline");
beeline({
  enabledInstrumentations: beeline
    .getInstrumentations()
    .filter(i => i !== "pg")
  /* ... additional configuration ... */
});

[giulianob]

@allyjweir It looks like the honeycomb sdk could potentially cause serious hangs in our application and there seems to be several bugs preventing pg from working well. Wondering if we could get an update on these bugs thanks.

[allyjweir]

Hi @giulianob, please see this comment for a way to disable the pg instrumentation. This will get your team moving without this faulty instrumentation causing any further issues.

I wasn't provided a reproduction of the issue so haven't tried to address this specific issue. The author of issue #171 has shared a reproduction so I will try to find some time to resolve that issue which might also fix this one.

[vreynolds]

Hello,

We will be closing this issue as it is a low priority for us. It is unlikely that we'll ever get to it, and so we'd like to set expectations accordingly.

As we enter 2022 Q1, we are trimming our OSS backlog. This is so that we can focus better on areas that are more aligned with the OpenTelemetry-focused direction of telemetry ingest for Honeycomb.

If this issue is important to you, please feel free to ping here and we can discuss/re-open.