Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty Body Sent to HTTP Subscribers #45

Open
sgtoj opened this issue Mar 22, 2018 · 6 comments
Open

Empty Body Sent to HTTP Subscribers #45

sgtoj opened this issue Mar 22, 2018 · 6 comments

Comments

@sgtoj
Copy link

sgtoj commented Mar 22, 2018

Sending messages to http subscribers do not work. A POST request is made to the endpoint, but no data is sent in the body. However, the content-length header is set the expected value as if a message was sent. I got the same result for two different http servers: netcat and server.js (nodejs express wrapper). Additionally, I confirmed messages sent to the same topic but different types of subscribers (i.e. file protocol) does work as expected.

command sent

aws sns --endpoint-url http://localhost:9911 publish --topic-arn arn:aws:sns:us-east-1:1465414804035:test-topic --message "test"

verbose logging of received msg from sns

POST /message HTTP/1.1
MessageExchangeId: 9a874dc5-3dc5-477f-90d2-4c4dd6e77c35
x-amz-sns-message-id: 9a874dc5-3dc5-477f-90d2-4c4dd6e77c35
x-amz-sns-message-type: Notification
x-amz-sns-subscription-arn: 6df4ed2b-a650-4f7c-910a-1a89c7cae5a6
x-amz-sns-topic-arn: arn:aws:sns:us-east-1:1465414804035:test-topic
User-Agent: Jakarta Commons-HttpClient/3.1
Host: subscriber:8010
Content-Length: 91

# no body data?

environment setup

./docker-compose.yml

version: "3.4"
services:
  sns:
    image: s12v/sns
    environment:
    - SUBSCRIBER_HTTP=subscriber:8010
    ports:
    - 9911:9911
    volumes:
    - ./sns:/etc/sns
  subscriber:
    image: subfuzion/netcat
    ports:
    - 8010:8010
    command: "-vl 8010"

./sns/db.json

{
    "version": 1,
    "timestamp": 1465414804110,
    "subscriptions": [
        {
            "arn": "6df4ed2b-a650-4f7c-910a-1a89c7cae5a6",
            "topicArn": "arn:aws:sns:us-east-1:1465414804035:test-topic",
            "endpoint": "http:{{env:SUBSCRIBER_HTTP}}/message",
            "owner": "",
            "protocol": "http"
        }
    ],
    "topics": [
        {
            "arn": "arn:aws:sns:us-east-1:1465414804035:test-topic",
            "name": "tms-session-api-events"
        }
    ]
}
@kfelichko
Copy link

kfelichko commented May 15, 2018

I was able to work around this by setting the content-type of the request on the receiving end, which is missing from the SNS publish.

const app = express();
app.use((req, res, next) => {
      req.headers['content-type'] = req.headers['content-type'] || 'application/json';
      next();
});
app.use(bodyParser.json());

HTH.

@victorsferreira
Copy link

in my case I just used bodyParser.text()

because it was sending a json content with a content type "plain/text". i also had to parse the json by myself

@damionvega
Copy link

Was banging my head on my keyboard for a while on this. Posting for anyone using Express v4+:

app.use(
  express.json({
    type: [
      'application/json',
      'text/plain', // AWS sends this content-type for its messages/notifications
    ],
  })
)

@alannesta
Copy link

alannesta commented Mar 6, 2020

I was able to work around this by setting the content-type of the request on the receiving end, which is missing from the SNS publish.

const app = express();
app.use((req, res, next) => {
      req.headers['content-type'] = req.headers['content-type'] || 'application/json';
      next();
});
app.use(bodyParser.json());

HTH.

I was able to work around this by setting the content-type of the request on the receiving end, which is missing from the SNS publish.

const app = express();
app.use((req, res, next) => {
      req.headers['content-type'] = req.headers['content-type'] || 'application/json';
      next();
});
app.use(bodyParser.json());

HTH.

The content type is text/plain, so req.headers['content-type'] = req.headers['content-type'] || 'application/json'; will not work.

x-forwarded-proto': 'https',
 | 2020-03-06 04:24:43 +00:00:      'cf-visitor': '{"scheme":"https"}',
 | 2020-03-06 04:24:43 +00:00:      'x-amz-sns-message-type': 'SubscriptionConfirmation',
  | 2020-03-06 04:24:43 +00:00:      'x-amz-sns-message-id': 'acdcb964-a042-434f-9746-036713666477',
 | 2020-03-06 04:24:43 +00:00:      'x-amz-sns-topic-arn': 'arn:aws:sns:us-east-1:722522486928:xxxx',
 | 2020-03-06 04:24:43 +00:00:      'content-type': 'text/plain; charset=UTF-8',
 | 2020-03-06 04:24:43 +00:00:      'user-agent': 'Amazon Simple Notification Service Agent',

@diegofcornejo
Copy link

Was banging my head on my keyboard for a while on this. Posting for anyone using Express v4+:

app.use(
  express.json({
    type: [
      'application/json',
      'text/plain', // AWS sends this content-type for its messages/notifications
    ],
  })
)

This worked for me, thanks

@geekhunger
Copy link

Solution from @damionvega still applies in January 2022! - However, I didn't want to always parse every "text/plain" request as JSON. Instead I wanted this workaround to only happen for Amazon Services (like SES or SNS). I figured that I could simply use the "user-agent" to identify these requests:

router.use(function(req, res, nxt) {
    const json_parser_options = {
        type: ["json", "*+json", "*/json", "application/json"],
        strict: true,
        limit: "5mb"
    }
    if(/Amazon/i.test(req.header("user-agent"))) {
        json_parser_options.type.push("text/*")
    }
    express.json(json_parser_options)(req, res, nxt)
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants