From 196b78d940bf2ca86be8179d2c23616a89b3ec6a Mon Sep 17 00:00:00 2001 From: Benjamin Sugden Date: Mon, 1 Jul 2024 15:04:52 +0200 Subject: [PATCH 1/3] PB-615: Spec multipart upload process Describe multipart upload process in more details. --- spec/static/spec/v1/openapitransactional.yaml | 28 +++++++++++++++++++ spec/transaction/tags.yaml | 28 +++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/spec/static/spec/v1/openapitransactional.yaml b/spec/static/spec/v1/openapitransactional.yaml index cc4e5561..5abe6765 100644 --- a/spec/static/spec/v1/openapitransactional.yaml +++ b/spec/static/spec/v1/openapitransactional.yaml @@ -92,6 +92,34 @@ tags: - *Files bigger than 10 MB should use compression, see [Compression](#section/Compression)* + ## Steps to upload Assets + + To upload assets files there are generally three steps involved: + +
    +
  1. Create new upload process
    2. +
  2. Upload parts/files
    3. +
  3. Complete upload process
    4. +
+ + Any file that is larger than 5 GB must be split into multiple parts. A file part must be at + least 5 MB except for the last one and at most 5 GB, otherwise the complete operation will fail. + If the file is less that 5 GB, you will only upload a single part, but must still start and complete the process as with multiple parts. + + (1) Use the [create new upload](#tag/Asset-Upload-Management/operation/createAssetUpload) request to start a new upload. + It will return a list of urls. + + (2) Use the urls to [upload asset file part](#tag/Asset-Upload-Management/operation/uploadAssetFilePart). Do this for each file part. + + (3) Once all parts have been uploaded, execute the [complete the upload](#tag/Asset-Upload-Management/operation/completeMultipartUpload) request. + The new asset file be available to users after you have successfully completed the upload. + + If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) + and restart the process. + + There can only be 1 upload in progress per asset. If you wish to start a new upload you must first abort any existing upload that is in progress. + Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads + are in progress. ## Authentication diff --git a/spec/transaction/tags.yaml b/spec/transaction/tags.yaml index f66b6ddf..0a8afd55 100644 --- a/spec/transaction/tags.yaml +++ b/spec/transaction/tags.yaml @@ -79,6 +79,34 @@ tags: - *Files bigger than 10 MB should use compression, see [Compression](#section/Compression)* + ## Steps to upload Assets + + To upload an assets file there are generally three steps involved: + +
    +
  1. Create new upload process
    2. +
  2. Upload parts/files
    3. +
  3. Complete upload process
    4. +
+ + Any file that is larger than 5 GB must be split into multiple parts. A file part must be at + least 5 MB except for the last one and at most 5 GB, otherwise the complete operation will fail. + If the file is less that 5 GB, you will only upload a single part, but must still start and complete the process as with multiple parts. + + (1) Use the [create new upload](#tag/Asset-Upload-Management/operation/createAssetUpload) request to start a new upload. + It will return a list of urls. + + (2) Use the urls to [upload asset file part](#tag/Asset-Upload-Management/operation/uploadAssetFilePart). Do this for each file part. + + (3) Once all parts have been uploaded, execute the [complete the upload](#tag/Asset-Upload-Management/operation/completeMultipartUpload) request. + The new asset file be available to users after you have successfully completed the upload. + + If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) + and restart the process. + + There can only be 1 upload in progress per asset. If you wish to start a new upload you must first abort any existing upload that is in progress. + Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads + are in progress. ## Authentication From 23717b255050c5b78209491f2eabdae96d56ffc2 Mon Sep 17 00:00:00 2001 From: Benjamin Sugden Date: Tue, 2 Jul 2024 08:35:12 +0200 Subject: [PATCH 2/3] Fix spec media type list. Update the supported media type list in the spec to match the actual implementation. Add mode information to the delete feature request description. --- spec/static/spec/v1/openapitransactional.yaml | 9 +++++++-- spec/transaction/paths.yaml | 4 +++- spec/transaction/tags.yaml | 3 +++ 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/spec/static/spec/v1/openapitransactional.yaml b/spec/static/spec/v1/openapitransactional.yaml index 5abe6765..20f80c91 100644 --- a/spec/static/spec/v1/openapitransactional.yaml +++ b/spec/static/spec/v1/openapitransactional.yaml @@ -30,6 +30,7 @@ tags: | Media Type | Description | File Extension(s) | | ---------- | ----------- | ----------------- | + | application/vnd.apache.parquet | Apache Parquet column-oriented data file format | .parquet | | application/x.ascii-grid+zip | Zipped ESRI ASCII raster format (.asc) | .zip | | application/x.ascii-xyz+zip | Zipped XYZ (.xyz) | .zip | | application/x.e00+zip | Zipped e00 | .zip | @@ -60,6 +61,8 @@ tags: | application/x.geojson+zip | Zipped GeoJSON | .zip | | application/x.interlis; version=2.3 | Interlis 2 | .xtf, .xml | | application/x.interlis+zip; version=2.3 | Zipped XTF (2.3) | .zip | + | application/x.interlis; version=2.4 | Interlis 2 | .xtf | .xml | + | application/x.interlis+zip; version=2.4 | Zipped XTF (2.4) | .zip | | application/x.interlis; version=1 | Interlis 1 | .itf | | application/x.interlis+zip; version=1 | Zipped ITF | .zip | | image/tiff; application=geotiff; profile=cloud-optimized | Cloud Optimized GeoTIFF (COG) | .tiff, .tif |` @@ -94,7 +97,7 @@ tags: ## Steps to upload Assets - To upload assets files there are generally three steps involved: + To upload an assets file there are generally three steps involved:
  1. Create new upload process
    2. @@ -599,7 +602,9 @@ paths: $ref: "#/components/responses/ServerError" delete: summary: Delete an existing feature by Id - description: Use this method to delete an existing feature/item. + description: | + Use this method to delete an existing feature/item. The feature can only be deleted if all + its assets have been deleted first. Also see [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). operationId: deleteFeature tags: - Data Management diff --git a/spec/transaction/paths.yaml b/spec/transaction/paths.yaml index a3044a5d..22f20ef3 100644 --- a/spec/transaction/paths.yaml +++ b/spec/transaction/paths.yaml @@ -231,7 +231,9 @@ paths: $ref: "../components/responses.yaml#/components/responses/ServerError" delete: summary: Delete an existing feature by Id - description: Use this method to delete an existing feature/item. + description: | + Use this method to delete an existing feature/item. The feature can only be deleted if all + its assets have been deleted first. Also see [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). operationId: deleteFeature tags: - Data Management diff --git a/spec/transaction/tags.yaml b/spec/transaction/tags.yaml index 0a8afd55..aad40bf7 100644 --- a/spec/transaction/tags.yaml +++ b/spec/transaction/tags.yaml @@ -16,6 +16,7 @@ tags: | Media Type | Description | File Extension(s) | | ---------- | ----------- | ----------------- | + | application/vnd.apache.parquet | Apache Parquet column-oriented data file format | .parquet | | application/x.ascii-grid+zip | Zipped ESRI ASCII raster format (.asc) | .zip | | application/x.ascii-xyz+zip | Zipped XYZ (.xyz) | .zip | | application/x.e00+zip | Zipped e00 | .zip | @@ -46,6 +47,8 @@ tags: | application/x.geojson+zip | Zipped GeoJSON | .zip | | application/x.interlis; version=2.3 | Interlis 2 | .xtf, .xml | | application/x.interlis+zip; version=2.3 | Zipped XTF (2.3) | .zip | + | application/x.interlis; version=2.4 | Interlis 2 | .xtf | .xml | + | application/x.interlis+zip; version=2.4 | Zipped XTF (2.4) | .zip | | application/x.interlis; version=1 | Interlis 1 | .itf | | application/x.interlis+zip; version=1 | Zipped ITF | .zip | | image/tiff; application=geotiff; profile=cloud-optimized | Cloud Optimized GeoTIFF (COG) | .tiff, .tif |` From 8773def071f3397b5fb7f153d1c5a51d1e9f6d7d Mon Sep 17 00:00:00 2001 From: Benjamin Sugden Date: Tue, 2 Jul 2024 09:32:13 +0200 Subject: [PATCH 3/3] PB-615: Spec multipart upload description Detailed description of upload process and error handling. Moved from existing files found in doc/ --- doc/stac-upload-error-handling.md | 84 --------------- .../assets/service-stac-upload-process.dia | Bin .../assets/service-stac-upload-process.svg | 0 spec/static/spec/v1/openapitransactional.yaml | 102 ++++++++++++++++-- spec/transaction/paths.yaml | 2 +- spec/transaction/tags.yaml | 100 +++++++++++++++-- 6 files changed, 190 insertions(+), 98 deletions(-) delete mode 100644 doc/stac-upload-error-handling.md rename {doc => spec/static}/assets/service-stac-upload-process.dia (100%) rename {doc => spec/static}/assets/service-stac-upload-process.svg (100%) diff --git a/doc/stac-upload-error-handling.md b/doc/stac-upload-error-handling.md deleted file mode 100644 index 1bc995e6..00000000 --- a/doc/stac-upload-error-handling.md +++ /dev/null @@ -1,84 +0,0 @@ -# STAC Asset Upload Error Handling - -## Glossary - -- Abort Upload: abort an upload in progress by sending and abort request to the service -- Cancel Upload: cancel the upload iteration without sending an abort request to the service - -## Recurrent upload - -If you have recurrent Asset uploads, you need to have a proper error handling otherwise the uploads might get stuck. Asset uploads operation are not stateless but statefull therefore the error handling is important. Here below is a simple practical example on which errors to handle in case of recurrent asset upload. - -Note this example is only recommended if the upload is recurrent (for example every hour). The number of retries below depends on the upload frequency, if the upload frequency is daily then you might want to have at least 3 retries with some exponential backoff time between each retries, in opposite if the upload is done at high frequency you might skip the retries and simply cancel the upload, using the next upload iteration as retry. - -1. Create Asset Upload - - ```text - POST https://data.geo.admin.ch/api/stac/v0.9/collections/{collection}/items/{item}/assets/{asset}/uploads - ``` - - - `201 OK` => Continue to step 2. - - `400 Bad Request` - - Response is `{"description": "Upload already in progress", "code": 400}` => Abort the upload - - To do so first get the `upload_id` of the `in-progress` upload via - - ```text - GET https://data.geo.admin.ch/api/stac/v0.9/collections/{collection}/items/{item}/assets/{asset}/uploads?status=in-progress - ``` - - - Then using this id abort the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v0.9/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - Then restart the step 1. - - Another `400 Bad Request` => Cancel upload - - Your request is not correct, analyze your request and correct it before retrying the step 1. - - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short amount of time and retry step 1. the amount of time to wait and the number of retries depends on the upload rate, but a minimum wait time of 100ms is recommended. -2. Upload the parts via the presigned URL - - ```text - PUT {presigned_url} - ``` - - - `200 OK` => Continue to step 3. - - `400 Bad Request` => Abort upload - - Abort upload using the current `upload_id` and contact service administrator. - - ```text - POST https://data.geo.admin.ch/api/stac/v0.9/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Retry step 2. with a short wait time (min 100ms). - -3. Complete the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v0.9/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/complete - ``` - - - `200 OK` => Upload successful - - `400 Bad Request` => Cancel upload - - Your request is invalid/incorrect, you need to cancel the upload script and verify its correctness. - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short moment (100ms), then retry the request. - - -The following figure shows the flow of a multipart upload process (raw version of this figure can be found [here](./assets/service-stac-upload-process.dia). You need [Dia](http://live.gnome.org/Dia) to edit the figure.) -![diagram](./assets/service-stac-upload-process.svg) diff --git a/doc/assets/service-stac-upload-process.dia b/spec/static/assets/service-stac-upload-process.dia similarity index 100% rename from doc/assets/service-stac-upload-process.dia rename to spec/static/assets/service-stac-upload-process.dia diff --git a/doc/assets/service-stac-upload-process.svg b/spec/static/assets/service-stac-upload-process.svg similarity index 100% rename from doc/assets/service-stac-upload-process.svg rename to spec/static/assets/service-stac-upload-process.svg diff --git a/spec/static/spec/v1/openapitransactional.yaml b/spec/static/spec/v1/openapitransactional.yaml index 20f80c91..a45a4674 100644 --- a/spec/static/spec/v1/openapitransactional.yaml +++ b/spec/static/spec/v1/openapitransactional.yaml @@ -95,7 +95,7 @@ tags: - *Files bigger than 10 MB should use compression, see [Compression](#section/Compression)* - ## Steps to upload Assets + ## Basic Steps to upload assets To upload an assets file there are generally three steps involved: @@ -120,10 +120,98 @@ tags: If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) and restart the process. - There can only be 1 upload in progress per asset. If you wish to start a new upload you must first abort any existing upload that is in progress. + Multiple parts (2) can be upload concurrently, but you cannot create a new upload (1) while another is still in progress. + If you wish to start a new upload you must first complete or abort any existing upload that is in progress. Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads are in progress. + ## Detailed example of asset upload + + This describes the process in more detail with focus on automated recurring uploads and error handling. + + ### Glossary + + - Abort Upload: abort an upload in progress by sending an abort request to the service + - Cancel Upload: cancel the upload iteration without sending an abort request to the service + + ### Recurrent upload + + If you have recurrent asset uploads, you need to have a proper error handling otherwise the uploads might get stuck. Asset uploads operation are not stateless but statefull therefore the error handling is important. Here below is a simple practical example on which errors to handle in case of recurrent asset upload. + + Note this example is only recommended if the upload is recurrent (for example every hour). The number of retries below depends on the upload frequency, if the upload frequency is daily then you might want to have at least 3 retries with some exponential backoff time between each retries, in opposite if the upload is done at high frequency you might skip the retries and simply cancel the upload, using the next upload iteration as retry. + + 1. Create Asset Upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads + ``` + + - `201 OK` => Continue to step 2. + - `400 Bad Request` + - Response is `{"description": "Upload already in progress", "code": 400}` => Abort the upload + - To do so first get the `upload_id` of the `in-progress` upload via + + ```text + GET https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads?status=in-progress + ``` + + - Then using this id abort the upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort + ``` + + - Then restart the step 1. + - Another `400 Bad Request` => Cancel upload + + Your request is not correct, analyze your request and correct it before retrying the step 1. + + - `500 Internal Server Error` => Cancel upload + + This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Service is momentarily not available, wait a short amount of time and retry step 1. the amount of time to wait and the number of retries depends on the upload rate, but a minimum wait time of 100ms is recommended. + 2. Upload the parts via the presigned URL + + ```text + PUT {presigned_url} + ``` + + - `200 OK` => Continue to step 3. + - `400 Bad Request` => Abort upload + + Abort upload using the current `upload_id` and contact service administrator. + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort + ``` + + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Retry step 2. with a short wait time (min 100ms). + + 3. Complete the upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/complete + ``` + + - `200 OK` => Upload successful + - `400 Bad Request` => Cancel upload + + Your request is invalid/incorrect, you need to cancel the upload script and verify its correctness. + - `500 Internal Server Error` => Cancel upload + + This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Service is momentarily not available, wait a short moment (100ms), then retry the request. + + The following figure shows the flow of a multipart upload process. + ![diagram](/api/stac/static/assets/service-stac-upload-process.svg) + + ## Authentication POST/PUT requests require authentication as described in [here](#tag/Authentication). @@ -182,7 +270,7 @@ tags: # 1. Create a multipart upload response = requests.post( - f"{scheme}://{hostname}/api/stac/v0.9/{asset_path}/uploads", + f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads", auth=(user, password), json={ "number_parts": 1, @@ -201,7 +289,7 @@ tags: # 3. Complete the upload response = requests.post( - f"{scheme}://{hostname}/api/stac/v0.9/{asset_path}/uploads/{upload_id}/complete", + f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads/{upload_id}/complete", auth=(user, password), json={'parts': [{'etag': etag, 'part_number': 1}]} ) @@ -229,7 +317,7 @@ tags: ``` curl --request POST \ --user MickeyMouse:I_love_Minnie_Mouse \ - --url https://data.geoadmin.ch/api/stac/v0.9/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ + --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ --header 'Content-Type: application/json' \ --data '{ "id": "fancy_unique_id", @@ -249,7 +337,7 @@ tags: ``` curl --request POST \ - --url https://data.geoadmin.ch/api/stac/v0.9/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ + --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ --header 'Authorization: Token ccecf40693bfc52ba090cd46eb7f19e723fe831f' \ --header 'Content-Type: application/json' \ --data '{ @@ -604,7 +692,7 @@ paths: summary: Delete an existing feature by Id description: | Use this method to delete an existing feature/item. The feature can only be deleted if all - its assets have been deleted first. Also see [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). + its assets have been deleted first. See also [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). operationId: deleteFeature tags: - Data Management diff --git a/spec/transaction/paths.yaml b/spec/transaction/paths.yaml index 22f20ef3..20def7fa 100644 --- a/spec/transaction/paths.yaml +++ b/spec/transaction/paths.yaml @@ -233,7 +233,7 @@ paths: summary: Delete an existing feature by Id description: | Use this method to delete an existing feature/item. The feature can only be deleted if all - its assets have been deleted first. Also see [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). + its assets have been deleted first. See also [Delete an existing asset by id](#tag/Data-Management/operation/deleteAsset). operationId: deleteFeature tags: - Data Management diff --git a/spec/transaction/tags.yaml b/spec/transaction/tags.yaml index aad40bf7..a170808b 100644 --- a/spec/transaction/tags.yaml +++ b/spec/transaction/tags.yaml @@ -82,7 +82,7 @@ tags: - *Files bigger than 10 MB should use compression, see [Compression](#section/Compression)* - ## Steps to upload Assets + ## Basic Steps to upload assets To upload an assets file there are generally three steps involved: @@ -107,10 +107,98 @@ tags: If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) and restart the process. - There can only be 1 upload in progress per asset. If you wish to start a new upload you must first abort any existing upload that is in progress. + Multiple parts (2) can be upload concurrently, but you cannot create a new upload (1) while another is still in progress. + If you wish to start a new upload you must first complete or abort any existing upload that is in progress. Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads are in progress. + ## Detailed example of asset upload + + This describes the process in more detail with focus on automated recurring uploads and error handling. + + ### Glossary + + - Abort Upload: abort an upload in progress by sending an abort request to the service + - Cancel Upload: cancel the upload iteration without sending an abort request to the service + + ### Recurrent upload + + If you have recurrent asset uploads, you need to have a proper error handling otherwise the uploads might get stuck. Asset uploads operation are not stateless but statefull therefore the error handling is important. Here below is a simple practical example on which errors to handle in case of recurrent asset upload. + + Note this example is only recommended if the upload is recurrent (for example every hour). The number of retries below depends on the upload frequency, if the upload frequency is daily then you might want to have at least 3 retries with some exponential backoff time between each retries, in opposite if the upload is done at high frequency you might skip the retries and simply cancel the upload, using the next upload iteration as retry. + + 1. Create Asset Upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads + ``` + + - `201 OK` => Continue to step 2. + - `400 Bad Request` + - Response is `{"description": "Upload already in progress", "code": 400}` => Abort the upload + - To do so first get the `upload_id` of the `in-progress` upload via + + ```text + GET https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads?status=in-progress + ``` + + - Then using this id abort the upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort + ``` + + - Then restart the step 1. + - Another `400 Bad Request` => Cancel upload + + Your request is not correct, analyze your request and correct it before retrying the step 1. + + - `500 Internal Server Error` => Cancel upload + + This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Service is momentarily not available, wait a short amount of time and retry step 1. the amount of time to wait and the number of retries depends on the upload rate, but a minimum wait time of 100ms is recommended. + 2. Upload the parts via the presigned URL + + ```text + PUT {presigned_url} + ``` + + - `200 OK` => Continue to step 3. + - `400 Bad Request` => Abort upload + + Abort upload using the current `upload_id` and contact service administrator. + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort + ``` + + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Retry step 2. with a short wait time (min 100ms). + + 3. Complete the upload + + ```text + POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/complete + ``` + + - `200 OK` => Upload successful + - `400 Bad Request` => Cancel upload + + Your request is invalid/incorrect, you need to cancel the upload script and verify its correctness. + - `500 Internal Server Error` => Cancel upload + + This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. + - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry + + Service is momentarily not available, wait a short moment (100ms), then retry the request. + + The following figure shows the flow of a multipart upload process. + ![diagram](/api/stac/static/assets/service-stac-upload-process.svg) + + ## Authentication POST/PUT requests require authentication as described in [here](#tag/Authentication). @@ -169,7 +257,7 @@ tags: # 1. Create a multipart upload response = requests.post( - f"{scheme}://{hostname}/api/stac/v0.9/{asset_path}/uploads", + f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads", auth=(user, password), json={ "number_parts": 1, @@ -188,7 +276,7 @@ tags: # 3. Complete the upload response = requests.post( - f"{scheme}://{hostname}/api/stac/v0.9/{asset_path}/uploads/{upload_id}/complete", + f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads/{upload_id}/complete", auth=(user, password), json={'parts': [{'etag': etag, 'part_number': 1}]} ) @@ -216,7 +304,7 @@ tags: ``` curl --request POST \ --user MickeyMouse:I_love_Minnie_Mouse \ - --url https://data.geoadmin.ch/api/stac/v0.9/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ + --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ --header 'Content-Type: application/json' \ --data '{ "id": "fancy_unique_id", @@ -236,7 +324,7 @@ tags: ``` curl --request POST \ - --url https://data.geoadmin.ch/api/stac/v0.9/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ + --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ --header 'Authorization: Token ccecf40693bfc52ba090cd46eb7f19e723fe831f' \ --header 'Content-Type: application/json' \ --data '{