Amazon OpenSearch Service: High Level Constructs For OpenSearch MultiAZWithStandBy Feature

[AmanRajAWS]

Describe the feature

The OpenSearch Team has recently launched MultiAZWithStandby Feature for OpenSearch Domain. Feature Documentation Link. According to the CDK docs currently there is no high level CDK construct for this feature.CDK Docs link for the MultiAZWIthStandbyFeature

Use Case

The general recommendation is to use high level constructs and due to lack of High level construct for this feature, the CDK template needs to be migrated to use CFN constructs if there is a need to Create an OpenSearchDomain with MultiAZWithStandBy. This serves as a hinderance to the adoption of the MultiAZWithStandBy feature for AWS OpenSearch Domains.

Proposed Solution

Add the MultiAZWithStandByAttribute to the capacityConfig object CDK Doc for CapacityConfig Attribute .
A MultiAZWithStandBy based OpenSearchDomain created using CDK high level constructs should look like
const domain = new Domain(this, 'Domain', {
version: EngineVersion.OPENSEARCH_1_0,
capacity: {
masterNodes: 3,
multiAZWithStandbyEnabled: true (possible values: [true,false])
},
});

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.84.0

Environment details (OS name and version, etc.)

macOS Ventura 13.4

[pahud]

Is this feature supported by cloudformation? If not, it sounds like we need to create a L3(or at least L2.5) construct for that?

[AmanRajAWS]

yes this feature is supported by cloudformation
docs link

[peterwoodworth]

You should be able to easily add this now to your template with an escape hatch.

Labeled as good first issue, all we need to do is add a boolean prop it appears

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[tmokmss]

@AmanRajAWS
Hi, I always get the following error with multiAZWithStandbyEnabled: true. But I think Auto-Tune feature cannot be controlled from CloudFormation for now. Should multiAZWithStandbyEnabled be true by default then? I'm not sure how this option can be used.

Resource handler returned message: "Invalid request provided: You must turn on Auto-Tune for domains with standby. 
(Service: OpenSearch, Status Code: 400, Request ID: eac0100d-5014-43b3-8af0-84e2d0e10197)" (RequestToken: 51cb5e17-12ae-508e-0eb8-de5a14a75639, HandlerErrorCode: InvalidRequest)

[Aman199825]

@tmokmss AutoTuneOptions should be enabled by default on the Cluster can you please share the CDK construct or CFN template where you are facing this issue

[tmokmss]

@Aman199825 Thanks. I also found the doc explaining that:

OpenSearch Service enables Auto-Tune by default on new domains.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/auto-tune.html#auto-tune-enable

So I checked my configuration and turned out that I'm using t3.medium.search as instance type, which does not support Multi-AZ with standby feature:

Multi-AZ with Standby only works with the m5, c5, r5, r6g, c6g, m6g, r6gd and i3 instance types.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-multiaz.html

I think this is the reason why I'm seeing the error. I also think CDK should validate this limitation, so I'll submit a PR or issue for this.

code to reproduce:

import * as cdk from 'aws-cdk-lib';
import { Domain, EngineVersion } from 'aws-cdk-lib/aws-opensearchservice';
import { EbsDeviceVolumeType, Vpc } from 'aws-cdk-lib/aws-ec2';

const stack = new cdk.Stack(app, 'OpenSearchAutoTuneReproduce', {
  env: {
    region: 'ap-northeast-1',
  },
});

const vpc = new Vpc(stack, 'Vpc', {natGateways: 1});
const targetSubnets = vpc.privateSubnets;

new Domain(stack, 'Domain', {
  version: EngineVersion.OPENSEARCH_2_11,
  capacity: {
    dataNodeInstanceType: 't3.medium.search',
    dataNodes: targetSubnets.length,
    multiAzWithStandbyEnabled: true, // not usable with t3
  },
  zoneAwareness: {
    enabled: true,
    availabilityZoneCount: targetSubnets.length,
  },
  ebs: {
    volumeSize: 30,
    volumeType: EbsDeviceVolumeType.GP3,
    throughput: 125,
    iops: 3000,
  },
  enforceHttps: true,
  fineGrainedAccessControl: {
    masterUserName: 'admin',
  },
  nodeToNodeEncryption: true,
  encryptionAtRest: {
    enabled: true,
  },
  vpc,
  vpcSubnets: [{ subnets: targetSubnets }],
  logging: {
    auditLogEnabled: true,
    slowSearchLogEnabled: true,
    appLogEnabled: true,
    slowIndexLogEnabled: true,
  },
  removalPolicy: cdk.RemovalPolicy.DESTROY,
});

[tmokmss]

@Aman199825 Do you know if OR1 and Im4gn instance type supports multi-AZ with standby feature?

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-instance-types.html

It's not listed in this doc but I suspect the doc is wrong because these instance types looks new and there's no reason not to support the feature.