-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Snapshot Resource changes for Ack #14
Snapshot Resource changes for Ack #14
Conversation
c53c3e1
to
b671400
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anil, this is an amazing job! There has clearly been a lot of work put into this PR and it looks great. You definitely chose to tackle a harder API, given the complexity in the CopySnapshot
option.
I've left you some comments inline regarding some ways we can save on custom code, and to keep things more inline with other ACK controllers.
apis/v1alpha1/snapshot.go
Outdated
KMSKeyID *string `json:"kmsKeyID,omitempty"` | ||
// A name for the snapshot being created. | ||
// +kubebuilder:validation:Required | ||
SnapshotName *string `json:"snapshotName"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please rename this just Name
? Use the S3 controller for reference
apis/v1alpha1/generator.yaml
Outdated
- SnapshotAlreadyExistsFault | ||
- SnapshotQuotaExceededFault | ||
- TagQuotaPerResourceExceeded | ||
- ClusterNotFoundFault |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this from terminal. We want the controller to retry until the Cluster is found!
apis/v1alpha1/generator.yaml
Outdated
- SnapshotQuotaExceededFault | ||
- TagQuotaPerResourceExceeded | ||
- ClusterNotFoundFault | ||
- InvalidClusterStateFault |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should remove this one as well, if there is a chance the cluster will recover into a good state.
from: | ||
operation: CopySnapshot | ||
path: SourceSnapshotName | ||
ClusterName: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this field not automatically generated? Why do we have to add it to the list of fields?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is part of different API (copy-snapshot) so we would need to add it
apis/v1alpha1/snapshot.go
Outdated
// The name of the snapshot | ||
// +kubebuilder:validation:Optional | ||
Name *string `json:"name,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This field should disappear if we rename the spec field - we don't want it duplicated across spec and status.
//Not used currently but whenever SDKUpdate() is generated for non existing update() operation on resource this will be added then as override | ||
return latest, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're probably safe to just delete this for now. If update isn't supported yet, better to return the not implemented error.
|
||
|
||
@dataclass | ||
class Snapshots(Bootstrappable): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job writing a Bootstrappable
type! Really happy to see it being used.
This Bootstrappable
is actually bootstrapping a MemoryDB
cluster - so should probably be called Cluster
class Snapshots(Bootstrappable): | |
class Cluster(Bootstrappable): |
while True: | ||
clusters = mdb.describe_clusters(ClusterName=self.snapshotClusterName) | ||
cluster = clusters['Clusters'][0] | ||
if cluster.get("Status").casefold() == available_status.casefold(): | ||
return True | ||
if time.time() > timeout: | ||
break | ||
time.sleep(60) | ||
raise ValueError('cluster not created within expected time') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ouch there are no waiters supported in boto3 for memorydb yet
@dataclass | ||
class Snapshots(Bootstrappable): | ||
# Output | ||
snapshotClusterName: str |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Outputs need to be instantiated with an initialised value:
snapshotClusterName: str | |
cluster_name str = field(init=False) |
test/e2e/bootstrap_resources.py
Outdated
Snapshot1: Snapshots | ||
Snapshot2: Snapshots |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason we need two clusters? Are we able to run both tests on a single cluster? It would certainly save us some time when bootstrapping.
test/e2e/replacement_values.py
Outdated
"KMSKEY": get_bootstrap_resources().KMSKey.key | ||
"SNAPSHOT_CLUSTER_NAME1": get_bootstrap_resources().Cluster1.clusterName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems e2e test failed due to missing comma after the KMSKEY statement.
tests/test_scenarios.py:29: in <module>
from e2e.replacement_values import REPLACEMENT_VALUES
E File "/memorydb-controller/tests/e2e/replacement_values.py", line 26
E "SNAPSHOT_CLUSTER_NAME1": get_bootstrap_resources().Cluster1.clusterName,
E ^
E SyntaxError: invalid syntax
test/e2e/replacement_values.py
Outdated
"SNAPSHOT_CLUSTER_NAME1": get_bootstrap_resources().Cluster1.clusterName, | ||
"SNAPSHOT_CLUSTER_NAME2": get_bootstrap_resources().Cluster1.clusterName |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should one of them be Cluster 2? We cannot take two snapshot from same cluster at same time.
e2e tests have passed, merging the PR. |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: AnilYalavarthi, kumargauravsharma The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Description of changes: Adding Snapshot Resource for MemoryDB
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.