Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CDPCP-12793 Added polling capability to user sycn resource #162

Merged
merged 1 commit into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:
run: |
go test -count=1 -parallel=4 -timeout 10m -json -v ./... 2>&1 | tee TestResults-${{ matrix.os }}_${{ matrix.go-version }}.log
- name: Upload test log
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
if: always()
with:
name: TestResults-${{ matrix.os }}_${{ matrix.go-version }}.log
Expand Down
12 changes: 12 additions & 0 deletions docs/resources/dw_aws_cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,15 @@ output "name" {
- `database_backup_retention_days` (Number) The number of days to retain database backups.
- `instance_settings` (Attributes) (see [below for nested schema](#nestedatt--instance_settings))
- `node_role_cdw_managed_policy_arn` (String) The managed policy ARN to be attached to the created node instance role.
- `polling_options` (Attributes) Polling related configuration options that could specify various values that will be used during CDP resource creation. (see [below for nested schema](#nestedatt--polling_options))

### Read-Only

- `cluster_id` (String) The id of the cluster.
- `id` (String) The ID of this resource.
- `last_updated` (String) Timestamp of the last Terraform update of the order.
- `name` (String) The name of the cluster matches the environment name.
- `status` (String) The status of the cluster.

<a id="nestedatt--network_settings"></a>
### Nested Schema for `network_settings`
Expand Down Expand Up @@ -124,3 +126,13 @@ Optional:
- `enable_spot_instances` (Boolean) Whether to use spot instances for worker nodes.


<a id="nestedatt--polling_options"></a>
### Nested Schema for `polling_options`

Optional:

- `async` (Boolean) Boolean value that specifies if Terraform should wait for resource creation/deletion.
- `call_failure_threshold` (Number) Threshold value that specifies how many times should a single call failure happen before giving up the polling.
- `polling_timeout` (Number) Timeout value in minutes that specifies for how long should the polling go for resource creation/deletion.


15 changes: 14 additions & 1 deletion docs/resources/environments_user_sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ This approach allows a fine-grain control of the sync operation.

resource "cdp_environments_user_sync" "example-user_sync" {
environment_names = ["example-cdp-environment-1", "example-cdp-environment-2"]
polling_options = {
async = true
}
}
```

Expand All @@ -42,7 +45,17 @@ resource "cdp_environments_user_sync" "example-user_sync" {
### Optional

- `environment_names` (Set of String) List of environments to be synced. If not present, all environments will be synced.
- `polling_options` (Attributes) Polling related configuration options that could specify various values that will be used during CDP resource creation. (see [below for nested schema](#nestedatt--polling_options))

### Read-Only

- `id` (String) The ID of this resource.
- `id` (String) The ID of this resource.

<a id="nestedatt--polling_options"></a>
### Nested Schema for `polling_options`

Optional:

- `async` (Boolean) Boolean value that specifies if Terraform should wait for resource creation/deletion.
- `call_failure_threshold` (Number) Threshold value that specifies how many times should a single call failure happen before giving up the polling.
- `polling_timeout` (Number) Timeout value in minutes that specifies for how long should the polling go for resource creation/deletion.
3 changes: 3 additions & 0 deletions examples/resources/cdp_environments_user_sync/resource.tf
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,7 @@

resource "cdp_environments_user_sync" "example-user_sync" {
environment_names = ["example-cdp-environment-1", "example-cdp-environment-2"]
polling_options = {
async = true
}
}
110 changes: 106 additions & 4 deletions resources/environments/resource_user_sync.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,26 @@ package environments

import (
"context"
"fmt"
"time"

"github.com/cloudera/terraform-provider-cdp/cdp-sdk-go/cdp"
"github.com/cloudera/terraform-provider-cdp/cdp-sdk-go/gen/environments/client"
"github.com/cloudera/terraform-provider-cdp/cdp-sdk-go/gen/environments/client/operations"
environmentsmodels "github.com/cloudera/terraform-provider-cdp/cdp-sdk-go/gen/environments/models"
"github.com/cloudera/terraform-provider-cdp/utils"
"github.com/google/uuid"
"github.com/hashicorp/terraform-plugin-framework/diag"
"github.com/hashicorp/terraform-plugin-framework/resource"
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
"github.com/hashicorp/terraform-plugin-framework/resource/schema/booldefault"
"github.com/hashicorp/terraform-plugin-framework/resource/schema/boolplanmodifier"
"github.com/hashicorp/terraform-plugin-framework/resource/schema/int64default"
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
"github.com/hashicorp/terraform-plugin-framework/types"
"github.com/hashicorp/terraform-plugin-log/tflog"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry"
)

var (
Expand All @@ -45,6 +52,33 @@ var userSyncSchema = schema.Schema{
ElementType: types.StringType,
Optional: true,
},
"polling_options": schema.SingleNestedAttribute{
MarkdownDescription: "Polling related configuration options that could specify various values that will be used during CDP resource creation.",
Optional: true,
Attributes: map[string]schema.Attribute{
"async": schema.BoolAttribute{
MarkdownDescription: "Boolean value that specifies if Terraform should wait for resource creation/deletion.",
Optional: true,
Computed: true,
Default: booldefault.StaticBool(false),
PlanModifiers: []planmodifier.Bool{
boolplanmodifier.UseStateForUnknown(),
},
},
"polling_timeout": schema.Int64Attribute{
MarkdownDescription: "Timeout value in minutes that specifies for how long should the polling go for resource creation/deletion.",
Default: int64default.StaticInt64(90),
Computed: true,
Optional: true,
},
"call_failure_threshold": schema.Int64Attribute{
MarkdownDescription: "Threshold value that specifies how many times should a single call failure happen before giving up the polling.",
Default: int64default.StaticInt64(3),
Computed: true,
Optional: true,
},
},
},
},
}

Expand All @@ -61,9 +95,9 @@ func (r *userSyncResource) Metadata(ctx context.Context, req resource.MetadataRe
}

type userSyncResourceModel struct {
ID types.String `tfsdk:"id"`

EnvironmentNames types.Set `tfsdk:"environment_names"`
ID types.String `tfsdk:"id"`
EnvironmentNames types.Set `tfsdk:"environment_names"`
PollingOptions *utils.PollingOptions `tfsdk:"polling_options"`
}

func toSyncAllUsersRequest(ctx context.Context, model *userSyncResourceModel, diag *diag.Diagnostics) *environmentsmodels.SyncAllUsersRequest {
Expand Down Expand Up @@ -93,7 +127,7 @@ func (r *userSyncResource) Create(ctx context.Context, req resource.CreateReques

params := operations.NewSyncAllUsersParamsWithContext(ctx)
params.WithInput(toSyncAllUsersRequest(ctx, &state, &resp.Diagnostics))
_, err := client.Operations.SyncAllUsers(params)
res, err := client.Operations.SyncAllUsers(params)
if err != nil {
if isSyncAllUsersNotFoundError(err) {
resp.Diagnostics.AddError(
Expand All @@ -113,6 +147,74 @@ func (r *userSyncResource) Create(ctx context.Context, req resource.CreateReques
if resp.Diagnostics.HasError() {
return
}

opID := res.Payload.OperationID
tflog.Debug(ctx, fmt.Sprintf("User sync operation ID: %s", *opID))
if !(state.PollingOptions != nil && state.PollingOptions.Async.ValueBool()) {
tflog.Debug(ctx, "User sync polling starts")
err = waitForUserSync(*opID, time.Hour*1, callFailureThreshold, r.client.Environments, ctx, state.PollingOptions)
if err != nil {
return
}
}
}

func waitForUserSync(opID string, fallbackTimeout time.Duration, callFailureThresholdDefault int, client *client.Environments, ctx context.Context, pollingOptions *utils.PollingOptions) error {
timeout, err := utils.CalculateTimeoutOrDefault(ctx, pollingOptions, fallbackTimeout)
if err != nil {
return err
}
callFailureThreshold, failureThresholdError := utils.CalculateCallFailureThresholdOrDefault(ctx, pollingOptions, callFailureThresholdDefault)
if failureThresholdError != nil {
return failureThresholdError
}
callFailedCount := 0
stateConf := &retry.StateChangeConf{
Pending: []string{"NEVER_RUN",
"REQUESTED",
"REJECTED",
"RUNNING",
"COMPLETED",
"FAILED",
"TIMEDOUT"},
Target: []string{"COMPLETED"},
Delay: 5 * time.Second,
Timeout: *timeout,
PollInterval: 10 * time.Second,
Refresh: func() (interface{}, string, error) {
tflog.Debug(ctx, fmt.Sprintf("About to get sync status for operationID %s", opID))
params := operations.NewSyncStatusParamsWithContext(ctx)
params.WithInput(&environmentsmodels.SyncStatusRequest{OperationID: &opID})
resp, err := client.Operations.SyncStatus(params)
if err != nil {
if isEnvNotFoundError(err) {
tflog.Debug(ctx, fmt.Sprintf("Recoverable error getting user sync status: %s", err))
callFailedCount = 0
return nil, "", nil
}
callFailedCount++
if callFailedCount <= callFailureThreshold {
tflog.Warn(ctx, fmt.Sprintf("Error getting user sync status with call failure due to [%s] but threshold limit is not reached yet (%d out of %d).", err.Error(), callFailedCount, callFailureThreshold))
return nil, "", nil
}
tflog.Error(ctx, fmt.Sprintf("Error getting user sync status (due to: %s) and call failure threshold limit exceeded.", err))
return nil, "", err
}
callFailedCount = 0
tflog.Info(ctx, fmt.Sprintf("User sync status: %s", resp.GetPayload().Status))
return checkUserSyncResponseStatusForError(resp)
},
}
_, err = stateConf.WaitForStateContext(ctx)

return err
}

func checkUserSyncResponseStatusForError(resp *operations.SyncStatusOK) (interface{}, string, error) {
if utils.ContainsAsSubstring([]string{"FAILED", "ERROR"}, string(resp.GetPayload().Status)) {
return nil, "", fmt.Errorf("unexpected user sync status status: %s. ", resp.GetPayload().Status)
}
return resp, string(resp.GetPayload().Status), nil
}

func isSyncAllUsersNotFoundError(err error) bool {
Expand Down
Loading