Add compression option to HttpTtansport

OpenLineage · Apr 15, 2024 · 20cf4a0 · 20cf4a0
1 parent 2132d73
commit 20cf4a0
Show file tree

Hide file tree

Showing 2 changed files with 6 additions and 0 deletions.
diff --git a/docs/client/java/partials/java_transport.md b/docs/client/java/partials/java_transport.md
@@ -14,6 +14,7 @@ Allows sending events to HTTP endpoint (with optional authorization headers).
 transport:
   type: http
   url: http://localhost:5000
+  compression: gzip
 ```
 
 With authorization:
@@ -26,6 +27,7 @@ transport:
   auth:
     type: api_key
     api_key: f38d2189-c603-4b46-bdea-e573a3b5a7d5
+  compression: gzip
 ```
 
 <details><summary>Override the default configuration of the HttpTransport within Java client</summary>
@@ -85,6 +87,7 @@ client.emit(startOrCompleteRun);
 | spark.openlineage.transport.urlParams.xyz   | A URL parameter (replace xyz) and value to be included in requests to the OpenLineage API server                                            | abcdefghijk           |
 | spark.openlineage.transport.url             | The hostname of the OpenLineage API server where events should be reported, it can have other properties embeded                            | http://localhost:5000 |
 | spark.openlineage.transport.headers.xyz     | Request headers (replace xyz) and value to be included in requests to the OpenLineage API server                                            | abcdefghijk           |
+| spark.openlineage.transport.compression     | Algorithm used by HTTP client to compress request body. Optional, default value `null`, allowed values: `gzip`                                            | gzip           |
 
 <details><summary>URL parsing within Spark integration</summary>
 <p>
@@ -113,6 +116,7 @@ example:
 | openlineage.transport.urlParams.xyz     | A URL parameter (replace xyz) and value to be included in requests to the OpenLineage API server                                            | abcdefghijk           |
 | openlineage.transport.url               | The hostname of the OpenLineage API server where events should be reported, it can have other properties embeded                            | http://localhost:5000 |
 | openlineage.transport.headers.xyz       | Request headers (replace xyz) and value to be included in requests to the OpenLineage API server                                            | abcdefghijk           |
+| openlineage.transport.compression       | Algorithm used by HTTP client to compress request body. Optional, default value `null`, allowed values: `gzip`                                            | gzip           |
 
 </TabItem>
 </Tabs>

diff --git a/docs/client/python.md b/docs/client/python.md
@@ -78,6 +78,7 @@ environment variables:
 - auth - dictionary specifying authentication options. Requires the type property. (optional)
   - type - string specifying the "api_key" or the fully qualified class name of your TokenProvider. (required if `auth` is provided)
   - apiKey - string setting the Authentication HTTP header as the Bearer. (required if `type` is `api_key`)
+- compression - algorithm used by HTTP client to compress request body. Optional, default value `null`, allowed values: `gzip`.
 
 Example:
 ```yaml
@@ -88,6 +89,7 @@ transport:
   auth:
     type: api_key
     apiKey: f048521b-dfe8-47cd-9c65-0cb07d57591e
+  compression: gzip
 ```
 
 ##### Console