S3 Bucket no Encoding Type

[Cam1337]

Is it possible to make a request with no encoding type? I am performing:

s3 = boto3.resource("s3")
bucket = s3.Bucket("myBucket")
objs = bucket.objects.filter(EncodingType=None, Prefix="foo")

This is still performing a GET request with ?encoding-type=url. Interestingly, the keys returned are only being formatted in URL encoding if the encoding type is specified accordingly, but the request is sending the parameter to url-encode the results every time.

http://boto3.readthedocs.io/en/latest/reference/services/s3.html?highlight=encodingType#S3.Bucket.objects

[JordonPhillips]

Encoding type is automatically set because there are unicode characters which can break the XML parser. To work around the issue, we automatically set the encoding type parameter to url if the customer hasn't already set it. We then decode the keys on the way back, again only if we provided that parameter. See boto/botocore#726 for more details.

What's your reasoning for wanting to omit the parameter?

[Cam1337]

Is there a way to force the parameter to false or to avoid the XML parser and deal with raw output? None of the data that I am using has unicode characters, so I would like to force execution without url-encoding. Additionally, I don't know if s3 does any additional internal work depending on url-encoding specification, but it would be nice to be able to check that out.

[Cam1337]

Any update?

[JordonPhillips]

You can unregister the handler on the client like so:

import boto3
from boto3.session import Session
from botocore.handlers import set_list_objects_encoding_type_url


s3 = boto3.client('s3')
s3.meta.events.unregister(
    'before-parameter-build.s3.ListObjects',
    set_list_objects_encoding_type_url)

# Unregister from a session
sess = Session()
sess.events.unregister(
    'before-parameter-build.s3.ListObjects',
    set_list_objects_encoding_type_url)

[Cam1337]

@JordonPhillips,

Thank you for your response. I will give that a try.

[tachang]

The reason is because Google Cloud storage gives this:

<?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidArgument</Code><Message>Invalid argument.</Message><Details>Invalid query parameter(s): [encoding-type]</Details></Error>

[gleicon]

Put together a gist, hope it helps. I understand that boto3 belongs to aws and interoperability is an interim configuration but it required knowing more about s3 protocol than I wanted.

https://gist.github.com/gleicon/2b8acb9f9c0f22753eaac227ff997b34