Skip to content

Commit ae34f86

Browse files
feat(dataflow): update the api
#### dataflow:v1b3 The following keys were added: - schemas.RuntimeUpdatableParams.properties.workerUtilizationHint (Total Keys: 2)
1 parent d9dcc8a commit ae34f86

File tree

6 files changed

+119
-96
lines changed

6 files changed

+119
-96
lines changed

docs/dyn/dataflow_v1b3.projects.jobs.html

Lines changed: 42 additions & 35 deletions
Large diffs are not rendered by default.

docs/dyn/dataflow_v1b3.projects.locations.flexTemplates.html

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ <h3>Method Details</h3>
229229
&quot;clientRequestId&quot;: &quot;A String&quot;, # The client&#x27;s unique identifier of the job, re-used across retried attempts. If this field is set, the service will ensure its uniqueness. The request to create a job will fail if the service has knowledge of a previously submitted job with the same client&#x27;s ID and job name. The caller may use this field to ensure idempotence of job creation across retried attempts to create a job. By default, the field is empty and, in that case, the service ignores it.
230230
&quot;createTime&quot;: &quot;A String&quot;, # The timestamp when the job was initially created. Immutable and set by the Cloud Dataflow service.
231231
&quot;createdFromSnapshotId&quot;: &quot;A String&quot;, # If this is specified, the job&#x27;s initial state is populated from the given snapshot.
232-
&quot;currentState&quot;: &quot;A String&quot;, # The current state of the job. Jobs are created in the `JOB_STATE_STOPPED` state unless otherwise specified. A job in the `JOB_STATE_RUNNING` state may asynchronously enter a terminal state. After a job has reached a terminal state, no further state updates may be made. This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
232+
&quot;currentState&quot;: &quot;A String&quot;, # The current state of the job. Jobs are created in the `JOB_STATE_STOPPED` state unless otherwise specified. A job in the `JOB_STATE_RUNNING` state may asynchronously enter a terminal state. After a job has reached a terminal state, no further state updates may be made. This field might be mutated by the Dataflow service; callers cannot mutate it.
233233
&quot;currentStateTime&quot;: &quot;A String&quot;, # The timestamp associated with the current state.
234234
&quot;environment&quot;: { # Describes the environment in which a Dataflow Job runs. # The environment for the job.
235235
&quot;clusterManagerApiService&quot;: &quot;A String&quot;, # The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. &quot;compute.googleapis.com&quot;.
@@ -360,7 +360,7 @@ <h3>Method Details</h3>
360360
},
361361
},
362362
},
363-
&quot;id&quot;: &quot;A String&quot;, # The unique ID of this job. This field is set by the Cloud Dataflow service when the Job is created, and is immutable for the life of the job.
363+
&quot;id&quot;: &quot;A String&quot;, # The unique ID of this job. This field is set by the Dataflow service when the job is created, and is immutable for the life of the job.
364364
&quot;jobMetadata&quot;: { # Metadata available primarily for filtering jobs. Will be included in the ListJob response and Job SUMMARY view. # This field is populated by the Dataflow service to support filtering jobs by the metadata values provided here. Populated for ListJobs and all GetJob views SUMMARY and higher.
365365
&quot;bigTableDetails&quot;: [ # Identification of a Cloud Bigtable source used in the Dataflow job.
366366
{ # Metadata for a Cloud Bigtable connector used by the job.
@@ -421,7 +421,7 @@ <h3>Method Details</h3>
421421
&quot;a_key&quot;: &quot;A String&quot;,
422422
},
423423
&quot;location&quot;: &quot;A String&quot;, # The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
424-
&quot;name&quot;: &quot;A String&quot;, # The user-specified Cloud Dataflow job name. Only one Job with a given name can exist in a project within one region at any given time. Jobs in different regions can have the same name. If a caller attempts to create a Job with the same name as an already-existing Job, the attempt returns the existing Job. The name must match the regular expression `[a-z]([-a-z0-9]{0,1022}[a-z0-9])?`
424+
&quot;name&quot;: &quot;A String&quot;, # The user-specified Dataflow job name. Only one active job with a given name can exist in a project within one region at any given time. Jobs in different regions can have the same name. If a caller attempts to create a job with the same name as an active job that already exists, the attempt returns the existing job. The name must match the regular expression `[a-z]([-a-z0-9]{0,1022}[a-z0-9])?`
425425
&quot;pipelineDescription&quot;: { # A descriptive representation of submitted pipeline as well as the executed form. This data is provided by the Dataflow service for ease of visualizing the pipeline and interpreting Dataflow provided metrics. # Preliminary field: The format of this data may change at any time. A description of the user pipeline and stages through which it is executed. Created by Cloud Dataflow service. Only retrieved with JOB_VIEW_DESCRIPTION or JOB_VIEW_ALL.
426426
&quot;displayData&quot;: [ # Pipeline level display data.
427427
{ # Data provided with a pipeline or transform to provide descriptive info.
@@ -510,13 +510,14 @@ <h3>Method Details</h3>
510510
],
511511
&quot;stepNamesHash&quot;: &quot;A String&quot;, # A hash value of the submitted pipeline portable graph step names if exists.
512512
},
513-
&quot;projectId&quot;: &quot;A String&quot;, # The ID of the Cloud Platform project that the job belongs to.
513+
&quot;projectId&quot;: &quot;A String&quot;, # The ID of the Google Cloud project that the job belongs to.
514514
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
515515
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
516516
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. Applies to `UpdateJob` requests. Set `requested_state` with `UpdateJob` requests to switch between the states `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING`. You can also use `UpdateJob` requests to change a job&#x27;s state from `JOB_STATE_RUNNING` to `JOB_STATE_CANCELLED`, `JOB_STATE_DONE`, or `JOB_STATE_DRAINED`. These states irrevocably terminate the job if it hasn&#x27;t already reached a terminal state. This field has no effect on `CreateJob` requests.
517517
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
518518
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
519519
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
520+
&quot;workerUtilizationHint&quot;: 3.14, # Target worker utilization, compared against the aggregate utilization of the worker pool by autoscaler, to determine upscaling and downscaling when absent other constraints such as backlog.
520521
},
521522
&quot;satisfiesPzi&quot;: True or False, # Output only. Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
522523
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
@@ -544,7 +545,7 @@ <h3>Method Details</h3>
544545
&quot;transformNameMapping&quot;: { # The map of transform name prefixes of the job to be replaced to the corresponding name prefixes of the new job.
545546
&quot;a_key&quot;: &quot;A String&quot;,
546547
},
547-
&quot;type&quot;: &quot;A String&quot;, # The type of Cloud Dataflow job.
548+
&quot;type&quot;: &quot;A String&quot;, # The type of Dataflow job.
548549
},
549550
}</pre>
550551
</div>

0 commit comments

Comments
 (0)