SNS not writing to SQS

[ptimson]

I have 2 docker containers running SNS / SQS using the ElasticMQ example. When I write to the topic it writes correctly out to the log file but doesn't send message to SQS.

I get the following WARN in the logs

10:04:40.528 [sns-akka.actor.default-dispatcher-3] WARN  akka.actor.ActorSystemImpl - Illegal header: Illegal 'authorization' header: Invalid input '/', expected tchar, OWS, listSep or 'EOI' (line 1, column 38): AWS4-HMAC-SHA256 Credential=ACCESSKEY/20180830/us-east-1/sns/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=e732dbb0746280abdbb0b2d3e7c7f7628c2d9926f1dc1a86b1b5f31a4f72a414
                                     ^
10:04:40.608 [sns-akka.actor.default-dispatcher-3] WARN  o.a.c.component.aws.sqs.SqsProducer - Cannot put the message header key=MessageExchangeId, value=9594974f-d2c7-4d30-bbb8-2ad55991600e into Sqs MessageAttribute
10:04:40.609 [sns-akka.actor.default-dispatcher-3] WARN  o.a.c.component.aws.sqs.SqsProducer - Cannot put the message header key=x-amz-sns-message-id, value=9594974f-d2c7-4d30-bbb8-2ad55991600e into Sqs MessageAttribute

Start-up log

10:04:38.285 [sns-akka.actor.default-dispatcher-3] INFO  akka.event.slf4j.Slf4jLogger - Slf4jLogger started
10:04:38.461 [sns-akka.actor.default-dispatcher-3] INFO  me.snov.sns.Main$ - SNS v0.4.1 is starting
10:04:38.501 [sns-akka.actor.default-dispatcher-3] INFO  me.snov.sns.Main$ - Loaded DB
10:04:38.513 [sns-akka.actor.default-dispatcher-5] INFO  me.snov.sns.actor.SubscribeActor - Loaded configuration
10:04:39.009 [sns-akka.actor.default-dispatcher-3] INFO  o.a.c.i.c.DefaultTypeConverter - Loaded 202 type converters
10:04:39.010 [sns-akka.actor.default-dispatcher-3] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 0.4.1 (CamelContext: sns) is starting
10:04:39.011 [sns-akka.actor.default-dispatcher-3] INFO  o.a.camel.impl.DefaultCamelContext - StreamCaching is enabled on CamelContext: sns
10:04:39.011 [sns-akka.actor.default-dispatcher-3] INFO  o.a.c.m.DefaultManagementStrategy - JMX is disabled
10:04:39.026 [sns-akka.actor.default-dispatcher-3] INFO  o.a.c.i.DefaultStreamCachingStrategy - StreamCaching in use with spool directory: /tmp/camel/camel-tmp-b35ce329-c557-4e09-8960-c2a80ad3f203 and rules: [Spool > 128K body size]
10:04:39.027 [sns-akka.actor.default-dispatcher-3] INFO  o.a.camel.impl.DefaultCamelContext - Total 0 routes, of which 0 are started.
10:04:39.028 [sns-akka.actor.default-dispatcher-3] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 0.4.1 (CamelContext: sns) started in 0.017 seconds
10:04:39.157 [sns-akka.actor.default-dispatcher-4] INFO  o.a.camel.util.CamelContextHelper - No existing PropertiesComponent has been configured, creating a new default PropertiesComponent with name: properties
10:04:40.194 [sns-akka.actor.default-dispatcher-2] WARN  akka.actor.ActorSystemImpl - Illegal header: Illegal 'authorization' header: Invalid input '/', expected tchar, OWS, listSep or 'EOI' (line 1, column 49): AWS4-HMAC-SHA256 Credential=AKIAJTVBYXZ67QPY6RNA/20180830/us-east-1/sns/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=6912ca33ab71531efb8958eb907d20c54e28b5ec365480b2b3cde29fd2424d94
                                                ^

Conf.json

{
  "version": 1,
  "timestamp": 1465414804110,
  "subscriptions": [
    {
      "arn": "e9126059-9eab-4b37-8194-e0d64dfb2045",
      "topicArn": "arn:aws:sns:us-east-1:123456789012:job-success",
      "endpoint": "aws-sqs://job-success?amazonSQSEndpoint={{env:SQS_ENDPOINT}}&accessKey=&secretKey=",
      "owner": "",
      "protocol": "sqs"
    },
    {
      "arn": "e9126059-9eab-4b37-8194-e0d64dfb2041",
      "topicArn": "arn:aws:sns:us-east-1:123456789012:job-error",
      "endpoint": "aws-sqs://job-error?amazonSQSEndpoint={{env:SQS_ENDPOINT}}&accessKey=&secretKey=",
      "owner": "",
      "protocol": "sqs"
    },
    {
      "arn": "6df4ed2b-a650-4f7c-910a-1a89c7cae5a6",
      "topicArn": "arn:aws:sns:us-east-1:123456789012:job-success",
      "endpoint": "file://tmp?fileName=sns.log",
      "owner": "",
      "protocol": "file"
    }
  ],
  "topics": [
    {
      "arn": "arn:aws:sns:us-east-1:123456789012:job-success",
      "name": "job-success"
    },
    {
      "arn": "arn:aws:sns:us-east-1:123456789012:job-error",
      "name": "job-error"
    }
  ]
}

db.json

include classpath("application.conf")

// node-address.host = "*" means the server will take "Host" header of incoming
// requests to generate queue URLs.
node-address {
    protocol = http
    host = "*"
    port = 9324
    context-path = ""
}

queues {
  job-success {}
  job-error {}
}

[jpaterso]

Hi, did you ever find a solution for this? I'm having the exact same issue

[ptimson]

No I don't think so. I will double check tomorrow

[itsa-sh]

Is there any updates on this? I've got the same problem ☹️

[Etherdaemon]

Mine appears to work - not sure what the differences are but I will post my setup in case someone else has an issue:

2 x docker containers spun up in docker-compose (I use alpine-sqs)

  fakesqs:
    image: roribio16/alpine-sqs
    ports:
      - 9324:9324
      - 9325:9325
    volumes:
      - ./fakesqs.conf:/opt/custom/elasticmq.conf

  fakesns:
    image: s12v/sns
    ports:
    - "9911:9911"
    volumes:
    - ./db.json:/etc/sns/db.json
    depends_on:
    - fakesqs

fakesqs.conf

include classpath("application.conf")

node-address {
    protocol = http
    host = "*"
    port = 9324
    context-path = ""
}

rest-sqs {
    enabled = true
    bind-port = 9324
    bind-hostname = "0.0.0.0"
    // Possible values: relaxed, strict
    sqs-limits = strict
}

queues {
  raw-telemetries {}
  deadletterdevelopment {}
  post-processing {}
}

db.json

{
  "version": 1,
  "timestamp": 1465414804110,
  "subscriptions": [
    {
      "arn": "e9126059-9eab-4b37-8194-e0d64dfb2045",
      "topicArn": "arn:aws:sns:ap-southeast-2:1234567890123:device-telemetries",
      "endpoint": "aws-sqs://post-processing?amazonSQSEndpoint=http://fakesqs:9324&accessKey=&secretKey=",
      "owner": "",
      "protocol": "sqs",
      "subscriptionAttributes": {
        "RawMessageDelivery": "true"
      }
    },
    {
      "arn": "6df4ed2b-a650-4f7c-910a-1a89c7cae5a6",
      "topicArn": "arn:aws:sns:ap-southeast-2:1234567890123:device-telemetries",
      "endpoint": "file://tmp?fileName=sns.log",
      "owner": "",
      "protocol": "file"
    }
  ],
  "topics": [
    {
      "arn": "arn:aws:sns:ap-southeast-2:1234567890123:device-telemetries",
      "name": "device-telemetries"
    }
  ]
}

[mcblum]

I'm also having the same issues here! @ptimson did you ever get this working?

[dany74q]

Do note that this warning does not indicate that dispatching does not work;
In my case, I've had the warning, but restarting both the sns & sqs brokers fixed the underlying issue.

[davioliveira-dev]

any resolution to this?

[rathiume]

@Etherdaemon I tried to setup sqs and elasticmq with steps you mentioned, but I am still not receiving message in sqs after publishing a message to sns topic.
I did not get any error while subscribe/Publish, its just queue is not getting any message on sqs.receive_message call after publish without any error.
Does anything comes to your mind on this issue?
@mcblum, @dany74q, @davioliveira-dev Or anyone else were able to find any resolution to issue?