Using tunneling with Remix does not save Session Replays

[rombrom]

Hi all!

We're facing an issue where session replays appear to not be propagated to Sentry when using the tunnel option. I've seen a few issues referencing a similar issue. The common theme appears to be the binary data throwing a wrench in the tunneling endpoint. However, even when setting useCompression: false session replays are still not showing up on our Sentry dashboards. All other errors and events seem to be proxied just fine.

Edit: I've set up a reproduction repository here: https://github.com/rombrom/sentry-tunnel-replay-issue

The code we have set up for our /tunnel endpoint is as follows:

import { type ActionFunctionArgs, json } from '@remix-run/node';

import { env } from '@/config';

export async function action({ request }: ActionFunctionArgs) {
  try {
    const body = await request.clone().text();
    const [piece] = body.split('\n');
    const header = JSON.parse(piece) as { dsn: string };
    const { hostname, pathname } = new URL(header.dsn);
    const project_id = pathname.replace('/', '');

    if (hostname !== env.SENTRY_HOST) {
      throw new Response(`Invalid sentry hostname: ${hostname}`, {
        status: 400,
        statusText: 'Bad Request',
      });
    }

    if (project_id !== env.SENTRY_PROJECT_ID) {
      throw new Error(`Invalid sentry project id: ${project_id}`);
    }

    await fetch(`https://${hostname}/api/${project_id}/envelope/`, {
      body: request.body,
      headers: { 'Content-Type': 'application/x-sentry-envelope' },
      method: 'POST',
    });
    return json({ ok: true });
  } catch (e) {
    return json({ error: 'Error tunneling to Sentry.' }, { status: 500 });
  }
}

Any help would be appreciated 🙏

[lforst]

Hi, can you verify that you are sending binary data correctly?

[rombrom]

This I'm unsure of. I've created a reproduction repo here: https://github.com/rombrom/sentry-tunnel-replay-issue

When I inspect the network call to the local /tunnel endpoint there's binary data. Logging the response.text() to terminal also shows binary data—although on the surface there appears to be a small difference. I haven't actually compared payloads. I assume converting the /tunnel binary payload and response.text() binary payload to ArrayBuffer or hexstr would allow me to do this.

In any case, like I mentioned in the other comment: I'd assume that disabling binary compression with replayIntegration({ useCompression: false }) would either fix this or show discrepancies. It neither fixes it and it appears both payloads are exactly the same.

[jjbayer]

@rombrom What does request.clone().text() return when the body contains binary data? Could the header parsing fail in that case?

[rombrom]

Hey thanks for the enquiry! I can verify the following:

1. Binary data is sent to the tunnel endpoint.
2. request.clone().text() does not fail header parsing.
3. Any other Sentry data (error events, profiling spans, etc.) is sent correctly to the Sentry platform.
4. Using replayIntegration({ useCompression: false }) makes no difference: replays still don't pop up on the Sentry platform.

I've created a reproduction repo here: https://github.com/rombrom/sentry-tunnel-replay-issue

[jjbayer]

@rombrom thank you for the repro case! With the version in the repo you linked, I was able to reproduce a BadRequest from Relay because of a missing newline. But when I change body back to body: request.body, my local Relay accepts the envelope as expected.

[smart]

Is there a final solution here of what to do? I'm also having the issue in sveltekit with my tunnel when replays are sent. 400 error. I can post my code but seeing if there was a working example.

[lforst]

Please share your code!

[smart]

I finally got it working rereading the thread and seeing I didn't have compression off in the replay settings. I had also set the tunnel url in my hooks.server.ts which was unnecessary. This looks like it's working, I modify 429s to 200s because I don't think those should show up as errors in visitors console.

export async function POST({ request }) {
try {

const body = await request.text();

const headers = new Headers();  
headers.set('X-Sentry-Auth', `Sentry sentry_key=mysentrykey, sentry_version=7`);
headers.set('Content-Type', 'application/x-sentry-envelope');

// Forward the event to Sentry
const upstream_sentry_url = `https://myingesturl`;

let response;

try {
  response = await fetch(upstream_sentry_url, {
    method: 'POST',
    headers: headers,
    body: body // Forward the entire body without modification
  });
} catch (e) {
  console.error("Error tunneling to sentry", e);
  return new Response(JSON.stringify({ error: "error tunneling to sentry" }), {
    status: 500,
    headers: {
      'Content-Type': 'application/json',
    },
  });
}

let resBody = await response.text();
console.log(response.status, resBody);

return new Response(resBody, {
  status: response.status == 429 ? 200 : response.status,
  headers: {
    'Content-Type': 'application/json',
  }
});

} catch (e) {
console.error("Error processing the request", e);
return new Response(JSON.stringify({ error: "error processing the request" }), {
status: 500,
headers: {
'Content-Type': 'application/json',
},
});
}
}

[fnimick]

Here's a much easier solution for Sveltekit, leveraging the kitql library's handleProxies function:

const SENTRY_UPSTREAM_URL = `https://${SENTRY_HOST}/api/${SENTRY_PRODUCT_ID}/envelope/`;
const proxies = handleProxies([
  [
    "/api/sentry-tunnel",
    {
      to: SENTRY_UPSTREAM_URL,
      requestHook: (event) => {
        const request = event.request.clone();
        request.headers.set(
          "X-Sentry-Auth",
          `Sentry sentry_key=${SENTRY_KEY}, sentry_version=7`,
        );
        request.headers.set("Content-Type", "application/x-sentry-envelope");
        return request;
      },
    },
  ],
]);

// your other hooks here

export const handle = sequence([..., proxies, ...]);

where the variables are set based on your DSN, which is formatted as https://<SENTRY_KEY>@<SENTRY_INGEST_ENDPOINT>/<SENTRY_PRODUCT_ID.