Skip to content

Commit

Permalink
address comments
Browse files Browse the repository at this point in the history
  • Loading branch information
vishalkarve15 committed Aug 7, 2023
1 parent abe2d78 commit 16cf7b6
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 8 deletions.
27 changes: 21 additions & 6 deletions README-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ df.write \

Writing to existing partitioned tables (date partitioned, ingestion time partitioned and range
partitioned) in APPEND save mode is fully supported by the connector and the BigQuery Storage Write
API. Partition overwrite and the use of `datePartition`, `partitionField` and `partitionType` as
API. Partition overwrite and the use of `datePartition`, `partitionField`, `partitionType`, `partitionRangeStart`, `partitionRangeEnd`, `partitionRangeInterval` as
described below is not supported at this moment by the direct write method.

**Important:** Please refer to the [data ingestion pricing](https://cloud.google.com/bigquery/pricing#data_ingestion_pricing)
Expand Down Expand Up @@ -618,10 +618,12 @@ word-break:break-word
<tr valign="top">
<td><code>partitionField</code>
</td>
<td>If field is specified together with `partitionType`, the table is partitioned by this field.
The field must be a top-level TIMESTAMP or DATE field. Its mode must be <strong>NULLABLE</strong>
<td>If this field is specified, the table is partitioned by this field.
<br/>For Time partitioning, specify together with the option `partitionType`.
<br/>For Integer-range partitioning, specify together with the 3 options: `partitionRangeStart`, `partitionRangeEnd, `partitionRangeInterval`.
<br/>The field must be a top-level TIMESTAMP or DATE field for Time partitioning, or INT64 for Integer-range partitioning. Its mode must be <strong>NULLABLE</strong>
or <strong>REQUIRED</strong>.
If the option is not set for a partitioned table, then the table will be partitioned by pseudo
If the option is not set for a Time partitioned table, then the table will be partitioned by pseudo
column, referenced via either<code>'_PARTITIONTIME' as TIMESTAMP</code> type, or
<code>'_PARTITIONDATE' as DATE</code> type.
<br/>(Optional).
Expand All @@ -642,13 +644,26 @@ word-break:break-word
<tr valign="top">
<td><code>partitionType</code>
</td>
<td>Supported types are: <code>HOUR, DAY, MONTH, YEAR</code>
<br/> This option is <b>mandatory</b> for a target table to be partitioned.
<td>Used to specify Time partitioning.
<br/>Supported types are: <code>HOUR, DAY, MONTH, YEAR</code>
<br/> This option is <b>mandatory</b> for a target table to be Time partitioned.
<br/>(Optional. Defaults to DAY if PartitionField is specified).
<br/><i>Not supported by the `DIRECT` write method.</i>
</td>
<td>Write</td>
</tr>
<tr valign="top">
<td><code>partitionRangeStart</code>,
<code>partitionRangeEnd</code>,
<code>partitionRangeInterval</code>
</td>
<td>Used to specify Integer-range partitioning.
<br/>These options are <b>mandatory</b> for a target table to be Integer-range partitioned.
<br/>All 3 options must be specified.
<br/><i>Not supported by the `DIRECT` write method.</i>
</td>
<td>Write</td>
</tr>
<tr valign="top">
<td><code>clusteredFields</code>
</td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -607,8 +607,8 @@ public void loadDataIntoTable(
}
if (options.getPartitionField().isPresent() && options.getPartitionRange().isPresent()) {
RangePartitioning.Builder rangePartitionBuilder = RangePartitioning.newBuilder();
rangePartitionBuilder.setField(options.getPartitionField().get());
rangePartitionBuilder.setRange(options.getPartitionRange().get());
options.getPartitionField().ifPresent(rangePartitionBuilder::setField);
options.getPartitionRange().ifPresent(rangePartitionBuilder::setRange);
jobConfiguration.setRangePartitioning(rangePartitionBuilder.build());
}

Expand Down

0 comments on commit 16cf7b6

Please sign in to comment.