diff --git a/docs/source/python/filesystems.rst b/docs/source/python/filesystems.rst index 40656f6b76f43..3fc10dc7718d3 100644 --- a/docs/source/python/filesystems.rst +++ b/docs/source/python/filesystems.rst @@ -153,8 +153,9 @@ PyArrow implements natively a S3 filesystem for S3 compatible storage. The :class:`S3FileSystem` constructor has several options to configure the S3 connection (e.g. credentials, the region, an endpoint override, etc). In addition, the constructor will also inspect configured S3 credentials as -supported by AWS (for example the ``AWS_ACCESS_KEY_ID`` and -``AWS_SECRET_ACCESS_KEY`` environment variables). +supported by AWS (such as the ``AWS_ACCESS_KEY_ID`` and +``AWS_SECRET_ACCESS_KEY`` environment variables, AWS configuration files, +and EC2 Instance Metadata Service for EC2 nodes). Example how you can read contents from a S3 bucket:: diff --git a/python/pyarrow/_s3fs.pyx b/python/pyarrow/_s3fs.pyx index e76c7b9ffa730..51c248d147828 100644 --- a/python/pyarrow/_s3fs.pyx +++ b/python/pyarrow/_s3fs.pyx @@ -140,14 +140,20 @@ cdef class S3FileSystem(FileSystem): """ S3-backed FileSystem implementation - If neither access_key nor secret_key are provided, and role_arn is also not - provided, then attempts to initialize from AWS environment variables, - otherwise both access_key and secret_key must be provided. + AWS access_key and secret_key can be provided explicitly. If role_arn is provided instead of access_key and secret_key, temporary credentials will be fetched by issuing a request to STS to assume the specified role. + If neither access_key nor secret_key are provided, and role_arn is also not + provided, then attempts to establish the credentials automatically. + S3FileSystem will try the following methods, in order: + + * ``AWS_ACCESS_KEY_ID``, ``AWS_SECRET_ACCESS_KEY``, and ``AWS_SESSION_TOKEN`` environment variables + * configuration files such as ``~/.aws/credentials`` and ``~/.aws/config`` + * for nodes on Amazon EC2, the EC2 Instance Metadata Service + Note: S3 buckets are special and the operations available on them may be limited or more expensive than desired.