req.body returns a Buffer

[jeremydrichardson]

For some reason in some version change (apologize I don't know from which version this started, but sometime after the switch from AWS to Vendia as the maintainer), req.body now returns a Buffer.

"body":{"type":"Buffer","data":[123,34,119,104,97,116,116,104,101,34,58,34,105,116,39,115,32,104,101,114,101,49,34,125]}}

Is this intended? This is even after using bodyParser.json() as middleware.

I have found that I can use

const { event, context } = getCurrentInvoke();
console.log(event.body)

That seems to give me a better response but takes place outside the Express environment.

I'm curious as to why req.body returns a Buffer instead of a json object.

[brettstack]

Hi @jeremydrichardson, could you provide some information such as:

1. package.json entry for this package
2. Lambda handler
3. Snippet of the route where you're trying to use req.body
4. Request content-type

[jeremydrichardson]

Yep.

1. package.json entry for this package

@vendia/serverless-express": "^4.0.0-rc.2

but I have updated recently so yarn.lock entry is

"@vendia/serverless-express@^4.0.0-rc.2":
  version "4.0.0-rc.6"
  resolved "https://registry.yarnpkg.com/@vendia/serverless-express/-/serverless-express-4.0.0-rc.6.tgz#63a201adbcd9d24b9a35135199e4a9dd3280b859"
  integrity sha512-OBd5Gwp4gD2EnKm3g64rv+4CPj2LJvpvd2iw3B32R366yMyImaCKPVPdb+y03DfBmxkQslZqK7T7owdPTJGZCQ==
  dependencies:
    type-is "^1.6.16"

2. Lambda handler

const serverlessExpress = require("@vendia/serverless-express");
const Sentry = require("@sentry/serverless");
const app = require("./app");

const sentryDSN =
  process.env.NODE_ENV !== "dev"
    ? "[omitted sentry address]"
    : null;

Sentry.init({
  dsn: sentryDSN,
  environment: process.env.NODE_ENV,
});

const se = serverlessExpress({
  app,
  respondWithErrors: process.env.NODE_ENV !== "prod",
  log: {
    info(message, additional) {
      console.info(message, additional);
    },
    debug(message, additional) {
      // This was making console really hard to read while developing
      //console.debug(message, additional);
    },
    error(message, additional) {
      console.error(message, additional);
    },
  },
});
exports.lambdaHandler = Sentry.AWSLambda.wrapHandler(se.handler);

3. Snippet of the route where you're trying to use req.body
   I've included most of our app.js so you can get context of the middleware and such that is being used with just the test route

const express = require("express");
const Sentry = require("@sentry/node");
const app = express();
const { getCurrentInvoke } = require("@vendia/serverless-express");

const bodyParser = require("body-parser");

const sentryDSN =
  process.env.NODE_ENV !== "devoff"
    ? "[sentry url omitted"
    : null;

Sentry.init({
  dsn: sentryDSN,
  environment: process.env.NODE_ENV,
});

app.use(Sentry.Handlers.requestHandler());

app.use(bodyParser.json());

// Enable CORS for all methods
app.use(function (req, res, next) {
  res.header("Access-Control-Allow-Origin", "*");
  res.header("Access-Control-Allow-Headers", "*");
  next();
});

app.post("/v1/test", (req, res) => {
  const { event, context } = getCurrentInvoke();
  res.json({
    expressBody: req.body,
    seBody: event.body,
    contentType: req.headers["content-type"]
  });
});

app.use(Sentry.Handlers.errorHandler());

module.exports = app;

Using postman POST with raw json payload

{"mykey": "this is the test key"}

Returns

{
    "expressBody": {
        "mykey": "this is the test key"
    },
    "seBody": "{\"mykey\": \"this is the test key\"}",
    "contentType": "application/json"
}

4. Request content-type
   It's application/json based on the code above and what the express req object is returning.

Let me know if you have any other questions.

[jeremydrichardson]

Well, sort of good and bad news.

Somehow our app started working again and the req.body is fine. It makes me wary that I'm not sure what it was but we were having some issue with the version of this package so maybe that's all it was.

I'll close this for now and re-open if the issue reappears.

[zunkelty]

Hi, I am currently experiencing the same issue. In my use case, I trigger a Lambda function directly (not via API Gateway) but pass an event that looks like an AWS API Gateway V1 proxy event. When I log the event before @vendia/serverless-express is executed, it prints correctly. In the execution of Express, I print req.body again and see a Buffer {"type":"Buffer","data":[...]}.

I have found a way to remove the error by changing node_modules/@vendia/serverless-express/src/event-sources/utils.js:

function getEventBody ({
  event,
  body = event.body,
  isBase64Encoded = event.isBase64Encoded
}) {
  return Buffer.from(body, isBase64Encoded ? 'base64' : 'utf8');
}

and just completely removing the conversion to a Buffer:

function getEventBody ({
  event,
  body = event.body,
  isBase64Encoded = event.isBase64Encoded
}) {
  return body;
}

I am using version 4.10.1 (currently latest).

1. My lambda handler

import serverlessExpress from "@vendia/serverless-express";
import { createApp } from "./app";

let serverlessExpressInstance: any;
async function setup() {
  const app = await createApp();
  serverlessExpressInstance = serverlessExpress({ app });
}

export async function handler(event: any, context: any) {
  console.log(JSON.stringify({ event }));

  if (!serverlessExpressInstance) {
    await setup();
  }

  return serverlessExpressInstance(event, context);
}

{
  "body": "{\"name\":\"Soenke\"}",
  "resource": "/{proxy+}",
  "path": "v1/echo.echo",
  "httpMethod": "POST",
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "v1/echo.echo"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate, br",
    "Content-Type": "application/json",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0"
  },
  "multiValueHeaders": {},
  "requestContext": {
    "body": "{\"name\":\"Soenke\"}"
  }
}

I am happy to provide more information if needed!

[rfink]

This is happening to us as well, 4.10.1

[manuelhonoredesousa]

Hi guys, I hope this solution to be useful for you...

You can get your body data like that...

app.post("/email",(req, res)=>{
const jsonBodyData = req.apiGateway.event.body;
console.log(JSON.parse(jsonBodyData));
})

[stegano]

Hi there,
I had the same issue and it seems to be resolved now.
Don't forget to include Content-Type: application/json in your request header.

[02strich]

This is still happening and breaks existing express applications.

[IndikaUdagedara]

I added this middleware as a temp workaround

const bufferToJSONMiddleware = (req: Request, res: Response, next: NextFunction) => {
  if (req.body instanceof Buffer) {
    try {
      req.body = JSON.parse(req.body.toString());
    } catch (err) {
      return res.status(400).json({ error: 'Invalid JSON data' });
    }
  }

  next();
};


app.use(bufferToJSONMiddleware)
``

[jreyesv]

The solution provided by @manuelhonoredesousa worked for me if you use this other package as well https://github.com/dougmoscrop/serverless-http/blob/master/docs/EXAMPLES.md

[aaronatbissell]

I've been following this thread pretty closely as we have been seeing similar problems.

It turns out our problem was that our Content-Type header was upper-case (should be content-type). There are some packages down the line that make some decisions based on the content-type header and they assume all header names will be lower-case.

Maybe an enhancement request for this library - if we are trying to make the req object look like a Node.js HTTP request object, then it should probably include the logic to lower-case all the HTTP headers. Even though we are satisfying the shape of an HTTP object, we aren't satisfying the behavior.

https://nodejs.org/api/http.html#messageheaders