Skip to content
This repository has been archived by the owner on Jul 30, 2020. It is now read-only.

Commit

Permalink
[AIRFLOW-4255] Replace Discovery based api with client based for GCS (a…
Browse files Browse the repository at this point in the history
  • Loading branch information
kaxil authored and dharamsk committed Aug 8, 2019
1 parent 04430bb commit a58adb8
Show file tree
Hide file tree
Showing 6 changed files with 336 additions and 639 deletions.
24 changes: 24 additions & 0 deletions UPDATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,30 @@ under the License.
This file documents any backwards-incompatible changes in Airflow and
assists users migrating to a new version.

## Airflow Master

### Changes to GoogleCloudStorageHook

* the discovery-based api (`googleapiclient.discovery`) used in `GoogleCloudStorageHook` is now replaced by the recommended client based api (`google-cloud-storage`). To know the difference between both the libraries, read https://cloud.google.com/apis/docs/client-libraries-explained. PR: [#5054](https://github.com/apache/airflow/pull/5054)
* as a part of this replacement, the `multipart` & `num_retries` parameters for `GoogleCloudStorageHook.upload` method has been removed:

**Old**:
```python
def upload(self, bucket, object, filename,
mime_type='application/octet-stream', gzip=False,
multipart=False, num_retries=0):
```

**New**:
```python
def upload(self, bucket, object, filename,
mime_type='application/octet-stream', gzip=False):
```

The client library uses multipart upload automatically if the object/blob size is more than 8 MB - [source code](https://github.com/googleapis/google-cloud-python/blob/11c543ce7dd1d804688163bc7895cf592feb445f/storage/google/cloud/storage/blob.py#L989-L997).

* the `generation` parameter is no longer supported in `GoogleCloudStorageHook.delete` and `GoogleCloudStorageHook.insert_object_acl`.

## Airflow 1.10.3

### RedisPy dependency updated to v3 series
Expand Down
Loading

0 comments on commit a58adb8

Please sign in to comment.