writing to S3 requires unnecessary permissions

[caboteria]

I'm trying to write to an S3 bucket for which I have permission to write to a path in the bucket, but I don't have permission to list the root of the bucket, which is a fairly common use case when using S3 to transfer files to/from multiple organizations in the same bucket. The write fails because of a check on line 426 of s3.py: it tries a head_bucket operation which requires[1] the ListBucket permission, which isn't really needed to write an object.

Commenting out lines 425-428 allows the write to succeed since it no longer does any operations in the root of the bucket.

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketHEAD.html

[caboteria]

It looks like this code was added to fix issue #154 so I tried to write to a bucket that doesn't exist to see if smart_open tried to create it implicitly (which I agree would be bad). It didn't try to create the bucket so lines 425-428 may no longer be necessary:

>>> import smart_open
>>> with open('s3://non-existing-bucket-name-i-hope/test.txt', 'wb') as fout:
...   fout.write(b'this is test text')
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 's3://non-existing-bucket-name-i-hope/test.txt'
>>> with smart_open.open('s3://non-existing-bucket-name-i-hope/test.txt', 'wb') as fout:
...   fout.write(b'this is test text')
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tcabot/Code/smart_open/smart_open/smart_open_lib.py", line 348, in open
    binary, filename = _open_binary_stream(uri, binary_mode, transport_params)
  File "/home/tcabot/Code/smart_open/smart_open/smart_open_lib.py", line 542, in _open_binary_stream
    return _s3_open_uri(parsed_uri, mode, transport_params), filename
  File "/home/tcabot/Code/smart_open/smart_open/smart_open_lib.py", line 614, in _s3_open_uri
    return smart_open_s3.open(parsed_uri.bucket_id, parsed_uri.key_id, mode, **kwargs)
  File "/home/tcabot/Code/smart_open/smart_open/s3.py", line 126, in open
    resource_kwargs=resource_kwargs,
  File "/home/tcabot/Code/smart_open/smart_open/s3.py", line 431, in __init__
    self._mp = self._object.initiate_multipart_upload(**multipart_upload_kwargs)
  File "/home/tcabot/.virtualenvs/data-acquisition/local/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/home/tcabot/.virtualenvs/data-acquisition/local/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/home/tcabot/.virtualenvs/data-acquisition/local/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/tcabot/.virtualenvs/data-acquisition/local/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the CreateMultipartUpload operation: The specified bucket does not exist