NotImplemented - A header you provided implies functionality that is not implemented

[tristanls]

I am consistently receiving a NotImplemented error when trying to write a simple text file to S3 using the sdk.

var AWS = require( 'aws-sdk' );

AWS.config.update( { 
  accessKeyId : 'ACCESS_KEY_ID', 
  secretAccessKey : 'SECRET_ACCESS_KEY' 
});

var s3 = new AWS.S3({ endpoint : 'https://s3.amazonaws.com' });

s3.client.putObject({
  ACL : 'private',
  Body : 'some string',
  Bucket : 'my-bucket',
  Key : 'some/path/here/stuff.txt',
  ServerSideEncryption : 'AES256'
}, function ( error, response ) {
 // error listed below
});

results in the following error:

{
  "code":"NotImplemented",
  "message":"A header you provided implies functionality that is not implemented",
  "statusCode":501,
  "retryable":true
}

which, in itself seems weird considering it is NotImplemented but is retryable.

[lsegal]

I can't reproduce this error on my side. I am getting a successful result when I put an object with that set of values.

The note about NotImplemented is valid, though. We don't (yet) have special rules for handling this error code-- the current logic is to simply retry 500 errors. This can be fixed.

Can you provide any more data about how you are getting this error? Probably unhelpful would be your node version and aws-sdk version (log AWS.VERSION)-- perhaps a little more helpful could be output from npm list.

[tristanls]

node: 0.8.16
AWS.VERSION: 0.9.1-pre.2
npm list:

├─┬ aws-sdk@0.9.1-pre.2
│ ├─┬ xml2js@0.2.2
│ │ └── sax@0.4.2
│ └── xmlbuilder@0.4.2

Another condition necessary for failure is multiple processes ( say 4 ) running on the same machine all making the putObject call within a short period of time:

2012-12-23T20:15:05.010Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:05.013Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:06.284Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:06.431Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}

I can confirm that a manually triggered individual call succeeds.

It's a somewhat arcane failure case. Those putObject() calls are created using setInterval() (30 seconds in this particular failure case).

I will continue digging and try to provide more information.

[lsegal]

Seems like one of the headers is invalid, try printing the results of response.httpRequest.headers in your callback.

[tristanls]

response.httpRequest.headers does not exist / is empty (I'm displaying it via DEBUG below after transport response)

2012-12-24T21:19:34.312Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.314Z DEBUG transport response 
2012-12-24T21:19:34.314Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.588Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.588Z DEBUG transport response 
2012-12-24T21:19:34.588Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.981Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.981Z DEBUG transport response 

I'll keep looking.

[tristanls]

Ok, I have a contrast between successful and failed operations. Well, there's really no contrast, looks the same to me.

The _endpoint property is for my internal use.

successful operation

2012-12-24T21:28:48.811Z DEBUG AWS.config {"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false}
2012-12-24T21:28:48.811Z DEBUG s3 {"client":{"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}},"_endpoint":"https://s3.amazonaws.com"}
2012-12-24T21:28:48.812Z DEBUG s3.client {"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}}
2012-12-24T21:28:48.954Z DEBUG whole response {"ServerSideEncryption":"AES256","ETag":"\"ETAG_VALUE_HERE\"","RequestId":"56E04C735CB16D95"}
2012-12-24T21:28:48.955Z ERROR transport error 
2012-12-24T21:28:48.955Z DEBUG transport response.httpRequest.headers

failed operation

2012-12-24T21:29:19.725Z DEBUG AWS.config {"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false}
2012-12-24T21:29:19.725Z DEBUG s3 {"client":{"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}},"_endpoint":"https://s3.amazonaws.com"}
2012-12-24T21:29:19.725Z DEBUG s3.client {"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}}
2012-12-24T21:29:20.891Z DEBUG whole response 
2012-12-24T21:29:20.891Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:29:20.891Z DEBUG transport response.httpRequest.headers

[tristanls]

I have a hint. If Body is an empty string, that would probably result in Content-Length header not being created, which is probably the culprit for NotImplemented error.

Will verify shortly.

[lsegal]

That would be an easy fix if true-- but this might be a symptom of a larger problem. Is there a reason why your requests would have intermittent empty bodies? Or were some of the files being uploaded empty?

[tristanls]

Confirmed, Body = '' // empty string will result in the errors I've reported.

---

Some files were being uploaded empty.

Allowing uploading empty files is useful from the perspective of having a "log heartbeat" if you will. Say, I want to dump activity/log data into s3 at a set time interval. If I have no data, the presence of an empty file still lets me know that the reporting agent is still alive (i.e. heartbeat).

[lsegal]

Supporting empty files doesn't need justifying, I was just confirming that there was no other issue at play here. Thanks for looking into this and confirming, this is an easy fix, see: https://github.com/aws/aws-sdk-js/blob/master/lib/rest_xml_client.js#L69-L72

[lsegal]

Thanks for getting to this, I was just about to commit my own-- there are some extra things to adjust in addition to that patch though, which I am dealing with now :)

[tristanls]

Ah, sorry for stepping on you :D . You're more familiar with the code base, so I'm sure I missed a bunch. Thanks!

[lsegal]

You only missed a minor change, actually.

[lsegal]

(pulling a valid body back out in the same scenario)