Amazon OpenSearch Service: High Level Constructs For OpenSearch Coordinator (NodeOptions) Feature

[dubesar]

Describe the feature

Describe the feature
The OpenSearch Team has recently launched Coordinator Node Feature for OpenSearch Domain which can be specified under a parameter called NodeOptions in ClusterConfig. API Documentation Link. According to the CDK docs, currently there is no high level CDK construct for this feature.

Use Case

The general recommendation is to use high level constructs and due to lack of High level construct for this feature. The CDK template needs to be migrated to use CFN constructs if there is a need to create an OpenSearchDomain with NodeOptions. This serves as a hindrance to the adoption of the Coordinator Node feature for AWS OpenSearch Domains.

Proposed Solution

Add the NodeOptions to the Domain high level object under ClusterConfig CDK Doc for Domain Options .
A NodeOptions based OpenSearchDomain created using CDK high level constructs should look like

const domain = new Domain(this, 'Domain', {
  version: EngineVersion.OPENSEARCH_1_0,
  clusterConfig: {
    nodeOptions: [{
      nodeType: 'coordinator' (possible values: [coordinator])
      nodeConfig: {
        enabled: true, (boolean)
        type: 'm5.xlarge.search', (string)
        count: 1 (number)
      }
    }]
  }
});

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

NA (Yet to be updated)

Environment details (OS name and version, etc.)

macOS Venture 13.7.1

[khushail]

Hi @dubesar , thanks for requesting this. CDK high level construct are basically synthesized to Cloudformation L1 Constructs. I see that this NodeOptions is not currently provided by the Cloudformation construct, so unless this is supported by Cloudformation, would have to wait and once is done from their side, CDK support can be provided.

You can create a feature request with Cloudformation team on their coverage roadmap and follow that for updates.Hope that would be helpful.
Thanks.

[dubesar]

Thanks for the reply, the cloudformation change is still in deployment, will inform once it's completed.

[dubesar]

@khushail the NodeOptions seems to be available now in the Cloudformation construct, Can you please check once?

[khushail]

@dubesar , Thanks for keeping a check on the recent updates with Cloudformation team.

I see that NodeOptions is available on cloudformation-https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-opensearchservice-domain-clusterconfig.html#cfn-opensearchservice-domain-clusterconfig-nodeoptions but the corresponding changes have not been incorporated in regions cloudformation specifications and relevant CDK docs.

CDK Docs pending update - https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.CfnDomain.ClusterConfigProperty.html

Cloudformation resource specification doc missing the NodeOptions -https://d1uauaxba7bl26.cloudfront.net/latest/gzip/CloudFormationResourceSpecification.json

I am unable to see the cloudformation construct updated with NodeOptions(https://github.com/cdklabs/awscdk-service-spec/blob/%40aws-cdk/service-spec-types%40v0.0.63/sources/CloudFormationSchema/us-east-1/aws-opensearchservice-domain.json) could you please share the link as it seems like L1 construct is still not updated with this property.
Thanks

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[dubesar]

I am not sure when the CDK docs will be updated, but the deployment from our side has been completed. Will keep a track if this is eventually consistent and will be updated in a few days.

[dubesar]

Hey, I am still not sure why this hasn't been updated yet.

Can you help me from where does the link: https://d1uauaxba7bl26.cloudfront.net/latest/gzip/CloudFormationResourceSpecification.json gets updated?

Also as a proposed solution we would like CDK to work like -

const domain = new Domain(this, 'Domain', {
  version: EngineVersion.OPENSEARCH_1_0,
  clusterConfig: {
    nodeOptions: [{
      nodeType: 'coordinator' (possible values: [coordinator])
      nodeConfig: {
        enabled: true, (boolean)
        type: 'm5.xlarge.search', (string)
        count: 1 (number)
      }
    }]
  }
});

Can you please help as the documentation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-opensearchservice-domain-clusterconfig.html, has already been updated.

[dubesar]

Hi Team, can we look into this please?

[khushail]

@dubesar ,

Cloduformation resource spec -https://d1uauaxba7bl26.cloudfront.net/latest/gzip/CloudFormationResourceSpecification.json

Cloudformation doc - https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.CfnDomain.ClusterConfigProperty.html#nodeoptions

I see that Cloudformation specification docs are also updated with this NodeOptions property as well, so you can now use the L1 construct for this feature.

[dubesar]

Thanks @khushail, so are we good with the above implementation then? Are we good to close this ticket as well?

[khushail]

@dubesar , The NodeOptions property is available through L1 construct only ,which means you can not use it as you suggested here. For clusterConfig to work like that, it has to available as DomainProps, which is currently not feasible. Here is how you can do this -

const domain = new Domain(this, 'Domain', {
      version: EngineVersion.OPENSEARCH_1_0,
      capacity: {
        masterNodes: 3,
        masterNodeInstanceType: 'm5.large.search',
        dataNodes: 2,
        dataNodeInstanceType: 'm5.large.search'
      },
      ebs: {
        volumeSize: 100
      },
      zoneAwareness: {
        availabilityZoneCount: 3
      }
    });

    const cfndomain = domain.node.defaultChild as opensearchservice.CfnDomain;
    cfndomain.clusterConfig = {
      coldStorageOptions: {
        enabled: false,
      },
      dedicatedMasterCount: 123,
      dedicatedMasterEnabled: false,
      dedicatedMasterType: 'dedicatedMasterType',
      instanceCount: 123,
      instanceType: 'instanceType',
      multiAzWithStandbyEnabled: false,
      nodeOptions: [{
        nodeConfig: {
          count: 123,
          enabled: false,
          type: 'type',
        },
        nodeType: 'nodeType',
      }],
      warmCount: 123,
      warmEnabled: false,
      warmType: 'warmType',
      zoneAwarenessConfig: {
        availabilityZoneCount: 123,
      },
      zoneAwarenessEnabled: false,
    };
  }

This would be synthesized into this json template -

{
 "Resources": {
  "Domain66AC69E0": {
   "Type": "AWS::OpenSearchService::Domain",
   "Properties": {
    "ClusterConfig": {
     "ColdStorageOptions": {
      "Enabled": false
     },
     "DedicatedMasterCount": 123,
     "DedicatedMasterEnabled": false,
     "DedicatedMasterType": "dedicatedMasterType",
     "InstanceCount": 123,
     "InstanceType": "instanceType",
     "MultiAZWithStandbyEnabled": false,
     "NodeOptions": [
      {
       "NodeConfig": {
        "Count": 123,
        "Enabled": false,
        "Type": "type"
       },
       "NodeType": "nodeType"
      }
     ],
     "WarmCount": 123,
     "WarmEnabled": false,
     "WarmType": "warmType",
     "ZoneAwarenessConfig": {
      "AvailabilityZoneCount": 123
     },
     "ZoneAwarenessEnabled": false
    },
    "DomainEndpointOptions": {
     "EnforceHTTPS": false,
     "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07"
    },
    "EBSOptions": {
     "EBSEnabled": true,
     "VolumeSize": 100,
     "VolumeType": "gp2"
    },
    "EncryptionAtRestOptions": {
     "Enabled": false
    },
    "EngineVersion": "OpenSearch_1.0",
    "LogPublishingOptions": {},
    "NodeToNodeEncryptionOptions": {
     "Enabled": false
    }
   },
   "UpdateReplacePolicy": "Retain",
   "DeletionPolicy": "Retain",
   "Metadata": {
    "aws:cdk:path": "NodeOptionsStack/Domain/Resource"
   }
  },

Hope that would be useful.
Please feel free to close this issue if your ask is fulfilled.