Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rds): add support for database instances #2187

Merged
merged 42 commits into from
May 31, 2019
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ef9a9cf
Remove unused cluster-parameter-group-ref.ts
jogold Apr 2, 2019
1c17af4
Refactor parameter group
jogold Apr 2, 2019
e55b6fc
Extract endpoint
jogold Apr 3, 2019
8c31724
Add option group
jogold Apr 3, 2019
cc49f71
Make parameter group description optional
jogold Apr 5, 2019
d798e0c
Add instance
jogold Apr 5, 2019
6cc4e88
Update README
jogold Apr 5, 2019
c890989
Merge branch 'master' into rds-instance
jogold Apr 5, 2019
2e5b611
Fix import for option group
jogold Apr 5, 2019
ee391c8
Use cdk.SecretValue
jogold Apr 5, 2019
fe17f60
Add metrics
jogold Apr 7, 2019
2aed438
Add connections to option groups
jogold Apr 8, 2019
9249aff
README
jogold Apr 9, 2019
3b9bc6e
Merge branch 'master' into rds-instance
jogold Apr 9, 2019
4150e19
selectSubnets
jogold Apr 9, 2019
8078dcf
Expose instanceArn
jogold Apr 9, 2019
b91e3a3
Props and defaults
jogold Apr 9, 2019
737bcaa
JSDoc
jogold Apr 9, 2019
b817d92
Refactor engine and secret rotation to avoid mapping functions and al…
jogold Apr 10, 2019
6dd669d
README
jogold Apr 10, 2019
a9b8792
JSDoc
jogold Apr 10, 2019
34627f8
Fix sem ver of rotation app
jogold Apr 10, 2019
305db7b
JSDoc
jogold Apr 10, 2019
ab5c6f3
Improve integ test
jogold Apr 10, 2019
96f14ee
Merge branch 'master' into rds-instance
jogold Apr 17, 2019
7c5526a
use IResource and Resource
jogold Apr 17, 2019
e7b62b0
Merge branch 'master' into rds-instance
jogold Apr 24, 2019
4578cca
use aws-events-targets in tests
jogold Apr 24, 2019
b589ed4
fix awslint:props-struct-name
jogold Apr 24, 2019
ee8d067
update README
jogold Apr 24, 2019
c2a2c92
remove awkward engine.engine
jogold Apr 24, 2019
e06616f
improve comment
jogold Apr 29, 2019
c8b185e
Merge branch 'master' into rds-instance
jogold May 1, 2019
25e406f
private base and local import
jogold May 1, 2019
f8ac7cb
Merge branch 'master' into rds-instance
jogold May 2, 2019
13fb8c7
fromOptionGroupName and fromParameterGroupName
jogold May 2, 2019
e852acf
JSDoc
jogold May 3, 2019
c392fbb
Merge branch 'master' into rds-instance
jogold May 29, 2019
28b0d59
instance parameter group for cluster
jogold May 30, 2019
6f8eec8
Merge branch 'master' into rds-instance
jogold May 30, 2019
d044e64
set GP2 as default storage type
jogold May 30, 2019
c904e99
update README for events
jogold May 30, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 69 additions & 26 deletions packages/@aws-cdk/aws-rds/README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,8 @@
## AWS RDS Construct Library

The `aws-cdk-rds` package contains Constructs for setting up RDS instances.

> Note: the functionality this package is currently limited, as the CDK team is
> focusing on other use cases first. If your use case is not listed below, you
> will have to use achieve it using CloudFormation resources.
>
> If you would like to help improve the state of this library, Pull Requests are
> welcome.

Supported:

* Clustered databases

Not supported:

* Instance databases
* Setting up from a snapshot


### Starting a Clustered Database

To set up a clustered database (like Aurora), create an instance of `DatabaseCluster`. You must
To set up a clustered database (like Aurora), define a `DatabaseCluster`. You must
always launch a database in a VPC. Use the `vpcSubnets` attribute to control whether
your instances will be launched privately or publicly:

Expand All @@ -45,33 +26,85 @@ By default, the master password will be generated and stored in AWS Secrets Mana
Your cluster will be empty by default. To add a default database upon construction, specify the
`defaultDatabaseName` attribute.

### Starting an Instance Database
To set up a instance database, define a `DatabaseInstance`. You must
always launch a database in a VPC. Use the `vpcSubnets` attribute to control whether
your instances will be launched privately or publicly:

```ts
const instance = new DatabaseInstance(stack, 'Instance', {
engine: rds.DatabaseInstanceEngine.OracleSE1,
instanceClass: new ec2.InstanceTypePair(ec2.InstanceClass.Burstable2, ec2.InstanceSize.Small),
masterUsername: 'syscdk',
vpc
});
```
By default, the master password will be generated and stored in AWS Secrets Manager.

Use `DatabaseInstanceFromSnapshot` and `DatabaseInstanceReadReplica` to create an instance from snapshot or
a source database respectively:

```ts
new DatabaseInstanceFromSnapshot(stack, 'Instance', {
eladb marked this conversation as resolved.
Show resolved Hide resolved
snapshotIdentifier: 'my-snapshot',
engine: rds.DatabaseInstanceEngine.Postgres,
instanceClass: new ec2.InstanceTypePair(ec2.InstanceClass.Burstable2, ec2.InstanceSize.Large),
vpc
});

new DatabaseInstanceReadReplica(stack, 'ReadReplica', {
eladb marked this conversation as resolved.
Show resolved Hide resolved
sourceDatabaseInstance: sourceInstance,
engine: rds.DatabaseInstanceEngine.Postgres,
instanceClass: new ec2.InstanceTypePair(ec2.InstanceClass.Burstable2, ec2.InstanceSize.Large),
vpc
});
```
Creating a "production" Oracle database instance with option and parameter groups:

[example of setting up a production oracle instance](test/integ.instance.lit.ts)


### Instance events
To define Amazon CloudWatch event rules for database instances, use the `onEvent`
method:

```ts
const rule = instance.onEvent('InstanceEvent');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an actual event name?

Copy link
Contributor Author

@jogold jogold Apr 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's the id:

public onEvent(id: string, target?: events.IEventRuleTarget, options?: events.EventRuleProps)

Only the generic onEvent has been implemented (there are way too many events possible for RDS instances: snapshots, failover, reboot, restart, logs, security groups...)

rule.addTarget(lambdaFunction);
```

### Connecting

To control who can access the cluster, use the `.connections` attribute. RDS database have
To control who can access the cluster or instance, use the `.connections` attribute. RDS databases have
a default port, so you don't need to specify the port:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it's not so much that there's a default port as it is that the RDS instance has an intrinsic (known) port.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be that a non-standard port was set at instance creation (e.g. not 3306 for MySQL).


```ts
cluster.connections.allowFromAnyIpv4('Open to the world');
```

The endpoints to access your database will be available as the `.clusterEndpoint` and `.readerEndpoint`
The endpoints to access your database cluster will be available as the `.clusterEndpoint` and `.readerEndpoint`
attributes:

```ts
const writeAddress = cluster.clusterEndpoint.socketAddress; // "HOSTNAME:PORT"
```

For an instance database:
```ts
const address = instance.instanceEndpoint.socketAddress; // "HOSTNAME:PORT"
```

### Rotating master password
When the master password is generated and stored in AWS Secrets Manager, it can be rotated automatically:

[example of setting up master password rotation](test/integ.cluster-rotation.lit.ts)
[example of setting up master password rotation for a cluster](test/integ.cluster-rotation.lit.ts)

Rotation of the master password is also supported for an existing cluster:
```ts
new RotationSingleUser(stack, 'Rotation', {
new SecretRotation(stack, 'Rotation', {
secret: importedSecret,
engine: DatabaseEngine.Oracle,
target: importedCluster,
application: SecretRotationApplication.OracleRotationSingleUser
target: importedCluster, // or importedInstance
vpc: importedVpc,
})
```
Expand All @@ -87,3 +120,13 @@ The `importedSecret` must be a JSON string with the following format:
"port": "<optional: if not specified, default port will be used>"
}
```

### Metrics
Database instances expose metrics (`cloudwatch.Metric`):
```ts
// The number of database connections in use (average over 5 minutes)
const dbConnections = instance.metricDatabaseConnections();

// The average amount of time taken per disk I/O operation (average over 1 minute)
const readLatency = instance.metric('ReadLatency', { statistic: 'Average', periodSec: 60 });
```
107 changes: 0 additions & 107 deletions packages/@aws-cdk/aws-rds/lib/cluster-parameter-group.ts

This file was deleted.

36 changes: 3 additions & 33 deletions packages/@aws-cdk/aws-rds/lib/cluster-ref.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
import ec2 = require('@aws-cdk/aws-ec2');
import secretsmanager = require('@aws-cdk/aws-secretsmanager');
import cdk = require('@aws-cdk/cdk');
import { Token } from '@aws-cdk/cdk';
import { IResource } from '@aws-cdk/cdk';
import { Endpoint } from './endpoint';

/**
* Create a clustered database with a given number of instances.
*/
export interface IDatabaseCluster extends cdk.IResource, ec2.IConnectable, secretsmanager.ISecretAttachmentTarget {
export interface IDatabaseCluster extends IResource, ec2.IConnectable, secretsmanager.ISecretAttachmentTarget {
/**
* Identifier of the cluster
*/
Expand Down Expand Up @@ -80,33 +80,3 @@ export interface DatabaseClusterAttributes {
*/
readonly instanceEndpointAddresses: string[];
}

/**
* Connection endpoint of a database cluster or instance
*
* Consists of a combination of hostname and port.
*/
export class Endpoint {
/**
* The hostname of the endpoint
*/
public readonly hostname: string;

/**
* The port of the endpoint
*/
public readonly port: number;

/**
* The combination of "HOSTNAME:PORT" for this endpoint
*/
public readonly socketAddress: string;

constructor(address: string, port: number) {
this.hostname = address;
this.port = port;

const portDesc = Token.isToken(port) ? '{IndirectPort}' : port;
this.socketAddress = `${address}:${portDesc}`;
}
}
40 changes: 12 additions & 28 deletions packages/@aws-cdk/aws-rds/lib/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@ import ec2 = require('@aws-cdk/aws-ec2');
import kms = require('@aws-cdk/aws-kms');
import secretsmanager = require('@aws-cdk/aws-secretsmanager');
import { Construct, DeletionPolicy, Resource, Token } from '@aws-cdk/cdk';
import { IClusterParameterGroup } from './cluster-parameter-group';
import { DatabaseClusterAttributes, Endpoint, IDatabaseCluster } from './cluster-ref';
import { DatabaseClusterAttributes, IDatabaseCluster } from './cluster-ref';
import { DatabaseSecret } from './database-secret';
import { Endpoint } from './endpoint';
import { IParameterGroup } from './parameter-group';
import { BackupProps, DatabaseClusterEngine, InstanceProps, Login } from './props';
import { CfnDBCluster, CfnDBInstance, CfnDBSubnetGroup } from './rds.generated';
import { DatabaseEngine, RotationSingleUser, RotationSingleUserOptions } from './rotation-single-user';
import { SecretRotation, SecretRotationApplication, SecretRotationOptions } from './secret-rotation';

/**
* Properties for a new database cluster
Expand Down Expand Up @@ -111,7 +112,7 @@ export interface DatabaseClusterProps {
*
* @default - No parameter group.
*/
readonly parameterGroup?: IClusterParameterGroup;
readonly parameterGroup?: IParameterGroup;

/**
* The CloudFormation policy to apply when the cluster and its instances
Expand Down Expand Up @@ -241,7 +242,7 @@ export class DatabaseCluster extends DatabaseClusterBase {
/**
* The database engine of this cluster
*/
public readonly engine: DatabaseClusterEngine;
private readonly secretRotationApplication: SecretRotationApplication;

/**
* The VPC where the DB subnet group is created.
Expand Down Expand Up @@ -286,11 +287,11 @@ export class DatabaseCluster extends DatabaseClusterBase {
});
}

this.engine = props.engine;
this.secretRotationApplication = props.engine.secretRotationApplication;

const cluster = new CfnDBCluster(this, 'Resource', {
// Basic
engine: this.engine,
engine: props.engine.name,
dbClusterIdentifier: props.clusterIdentifier,
dbSubnetGroupName: subnetGroup.ref,
vpcSecurityGroupIds: [this.securityGroupId],
Expand Down Expand Up @@ -347,7 +348,7 @@ export class DatabaseCluster extends DatabaseClusterBase {

const instance = new CfnDBInstance(this, `Instance${instanceIndex}`, {
// Link to cluster
engine: props.engine,
engine: props.engine.name,
dbClusterIdentifier: cluster.ref,
dbInstanceIdentifier: instanceIdentifier,
// Instance properties
Expand Down Expand Up @@ -375,13 +376,13 @@ export class DatabaseCluster extends DatabaseClusterBase {
/**
* Adds the single user rotation of the master password to this cluster.
*/
public addRotationSingleUser(id: string, options: RotationSingleUserOptions = {}): RotationSingleUser {
public addRotationSingleUser(id: string, options: SecretRotationOptions = {}): SecretRotation {
if (!this.secret) {
throw new Error('Cannot add single user rotation for a cluster without secret.');
}
return new RotationSingleUser(this, id, {
return new SecretRotation(this, id, {
secret: this.secret,
engine: toDatabaseEngine(this.engine),
application: this.secretRotationApplication,
vpc: this.vpc,
vpcSubnets: this.vpcSubnets,
target: this,
Expand All @@ -396,20 +397,3 @@ export class DatabaseCluster extends DatabaseClusterBase {
function databaseInstanceType(instanceType: ec2.InstanceType) {
return 'db.' + instanceType.toString();
}

/**
* Transforms a DatbaseClusterEngine to a DatabaseEngine.
*
* @param engine the engine to transform
*/
function toDatabaseEngine(engine: DatabaseClusterEngine): DatabaseEngine {
switch (engine) {
case DatabaseClusterEngine.Aurora:
case DatabaseClusterEngine.AuroraMysql:
return DatabaseEngine.Mysql;
case DatabaseClusterEngine.AuroraPostgresql:
return DatabaseEngine.Postgres;
default:
throw new Error('Unknown engine');
}
}
Loading