From 68439958344528ba8a6bd972df028d6ea2b7e4e8 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Mon, 29 Jan 2024 19:11:07 -0800 Subject: [PATCH 01/11] Document the crossplane.io/external-create-... annotations These annotations were introduced in https://github.com/crossplane/crossplane-runtime/pull/283. Per https://github.com/crossplane/crossplane/issues/3037 folks find these annotations hard to reason about. That's understandable, because they're doing a lot of subtle things. This section ended up super long, but I think this is an area where folks really need to understand what's happening in order to make good decisions when Crossplane refuses to proceed. Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 138 +++++++++++++++++-- 1 file changed, 127 insertions(+), 11 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index 75a0c7bb6..cac55d17b 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -624,7 +624,7 @@ kind: RDSInstance metadata: name: my-rds-instance annotations: - crossplane.io/external-name: my-custom-namee + crossplane.io/external-name: my-custom-name ``` ```shell {copy-lines="1"} @@ -635,21 +635,137 @@ my-rds-instance True True my-custom-name 11m ### Creation annotations -Providers create new managed resources with the +Providers use three annotations to avoid and detect leaking resources: + +* `crossplane.io/external-create-pending` +* `crossplane.io/external-create-failed` +* `crossplane.io/external-create-succeeded` + +```yaml {label="creation"} +apiVersion: ec2.aws.upbound.io/v1beta1 +kind: VPC +metadata: + name: my-vpc + annotations: + crossplane.io/external-name: vpc-1234567890abcdef0 + crossplane.io/external-create-pending: "2023-12-18T21:48:06Z" + crossplane.io/external-create-succeeded: "2023-12-18T21:48:40Z" +``` + +{{}} +A provider uses a managed resource's `crossplane.io/external-name` annotation to +know if it needs to create the resource in the external system. The annotation +uniquely identifies the resource in the external system, such as AWS. + +If the provider doesn't save this annotation, it forgets that it created the +resource. In fact, it doesn't know the resource ever existed and can't manage +it. Crossplane calls this a leaked resource. +{{}} + +Some external systems let the provider specify the name of the resource when +it's created. The provider can generate a name and save it to the +`crossplane.io/external-name` annotation before it creates the resource. This +guarantees the provider doesn't leak the resource. + +Other external systems don't let the provider specify the name of the resource. +Instead they generate an unpredictable name and include it in the response to +the create API call the provider makes. + +{{}} +For this second kind of external system, there is always a risk that a provider +leaks a newly created resource. +{{}} + +A provider can't guarantee that it can save the generated name to the +`crossplane.io/external-name` annotation. If the provider crashes, restarts, or +can't reach the network after creating the resource it can't save the +annotation. If this happens, the provider leaks the resource. + +A provider can detect that it might have leaked a resource. If the provider +thinks it might have leaked a resource, it stops and waits for human +intervention. + +When a provider thinks it may have leaked a resource it creates an event +associated with the managed resource: + +```console +cannot determine creation result - remove the crossplane.io/external-create-pending annotation if it is safe to proceed +``` + + +The provider uses annotations to detect that it might have leaked a resource: + +* Before creating a resource, it writes the current time to the + `crossplane.io/external-create-pending` annotation. +* If the create succeeds, it writes the current times to the + `crossplane.io/external-create-succeeded` annotation. +* If the create fails, it writes the current time to the + `crossplane.io/external-create-failed` annotation. + +Providers try hard to write these annotations to the managed resource. +If writing the annotations fails, the provider retries several times. + +When a provider sees a managed resource with: + +* A `crossplane.io/external-create-pending` annotation. +* No `crossplane.io/external-create-succeeded` or + `crossplane.io/external-create-failed` annotation. + +It knows that it might have created the resource and failed to record its +`crossplane.io/external-name` annotation. When this happens, the provider +refuses to create the resource and waits for human intervention. + +{{}} +The safest thing for a provider to do when it detects that it might have leaked +a resource is to stop and wait for human intervention. + +Waiting for human intervention ensures the provider doesn't create a duplicate +of the leaked resource. Creating duplicate resources can be costly and +dangerous. + +If you see the `cannot determine creation result` error, inspect the external +system for resources created around the time recorded in the `crossplane.io/external-create-pending` annotation. -The Provider applies the `crossplane.io/external-create-succeeded` or -`crossplane.io/external-create-failed` annotation after making the external API -call and receiving a response. +If you find a leaked resource, and it's safe to do so, delete it. Remove the +`crossplane.io/external-create-pending` annotation from the managed resource. -{{}} -If a Provider restarts before creating the `succeed` or `fail` annotations the -Provider can't reconcile the managed resource. +If you don't find a leaked resource, remove the +`crossplane.io/external-create-pending` annotation from the managed resource. +{{}} -Read Crossplane [issue #3037](https://github.com/crossplane/crossplane/issues/3037#issuecomment-1110142427) -for more details -{{< /hint >}} +{{}} +Providers never delete the `crossplane.io/external-create-pending`, +`crossplane.io/external-create-succeeded`, or +`crossplane.io/external-create-failed` annotations. + +If several of these annotations exist, a provider uses the timestamps to +determine the state of the resource. + +For example if a provider sees a managed resource with a +`crossplane.io/external-create-pending` annotation that's newer than its +`crossplane.io/external-create-succeded` annotation it knows it might have +failed to record its `crossplane.io/external-name` annotation. + +This could happen if a provider successfully created the resource at some point +in the past, the later failed to recreate it. +{{}} + +{{}} +Providers also use these annotations to avoid leaking resources in other ways. + +Writing the `crossplane.io/external-create-pending` annotation lets the provider +know it's reconciling the latest version of the managed resource. The write +would fail if the provider was reconciling a stale, cached version. + +Saving a timestamp to the `crossplane.io/external-create-succeeded` annotation +lets the provider know that it probably created the resource, even if the +external system says it doesn't exist. +Sometimes there is a delay between the provider creating a resource, and the +system reporting that it exists. If this is the case, the provider waits an +appropriate amount of time for the resource to appear in the external system. +{{}} ### Paused Manually applying the `crossplane.io/paused` annotation causes the Provider to From 6cc91a7d728ce67478ab49a34db835f0674cc799 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Tue, 30 Jan 2024 17:42:00 -0800 Subject: [PATCH 02/11] Refactor external creation annotations section Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 186 ++++++++++--------- 1 file changed, 97 insertions(+), 89 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index cac55d17b..d29feed06 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -635,13 +635,29 @@ my-rds-instance True True my-custom-name 11m ### Creation annotations -Providers use three annotations to avoid and detect leaking resources: +In some rare situations a provider can forget that it created a resource. When +this happens the provider can't manage the resource. -* `crossplane.io/external-create-pending` -* `crossplane.io/external-create-failed` -* `crossplane.io/external-create-succeeded` +{{}} +Crossplane calls resources that a provider creates but doesn't manage _leaked +resources_. +{{}} + +Providers set three creation annotations to avoid and detect leaked resources: + +* {{}}crossplane.io/external-create-pending{{}} - + The last time the provider was about to create the resource. +* {{}}crossplane.io/external-create-succeeded{{}} - + The last time the provider successfully created the resource. +* `crossplane.io/external-create-failed` - The last time the provider failed to + create the resource. -```yaml {label="creation"} + +Use `kubectl get` to view the annotations on a managed resource. For example, an +AWS VPC resource: + +```yaml {label="creation" copy-lines="2-9"} +$ kubectl get vpc my-vpc apiVersion: ec2.aws.upbound.io/v1beta1 kind: VPC metadata: @@ -652,120 +668,112 @@ metadata: crossplane.io/external-create-succeeded: "2023-12-18T21:48:40Z" ``` -{{}} -A provider uses a managed resource's `crossplane.io/external-name` annotation to -know if it needs to create the resource in the external system. The annotation -uniquely identifies the resource in the external system, such as AWS. +A provider uses the +{{}}crossplane.io/external-name{{}} +annotation to find a resource in an external system, for example AWS. -If the provider doesn't save this annotation, it forgets that it created the -resource. In fact, it doesn't know the resource ever existed and can't manage -it. Crossplane calls this a leaked resource. -{{}} +If the provider can't find a managed resource in an external system, it thinks +the resource doesn't exist. When the provider thinks a resource doesn't exist +it creates the resource. + +Some external systems don't let a provider specify a resource's name when the +provider creates it. Instead the external system generates an unpredictable name +and returns it to the provider. -Some external systems let the provider specify the name of the resource when -it's created. The provider can generate a name and save it to the -`crossplane.io/external-name` annotation before it creates the resource. This -guarantees the provider doesn't leak the resource. +When the external system generates the resource's name, it's critical that the +provider saves it to the managed resource's `crossplane.io/external-name` +annotation. If it doesn't, it leaks the resource. -Other external systems don't let the provider specify the name of the resource. -Instead they generate an unpredictable name and include it in the response to -the create API call the provider makes. +A provider can't guarantee that it can save the annotation. The provider could +restart or lose network connectivity between creating the resource and saving +the annotation. {{}} -For this second kind of external system, there is always a risk that a provider -leaks a newly created resource. +Anytime an external system generates a resource's name there is a risk the +provider could leak the resource. {{}} -A provider can't guarantee that it can save the generated name to the -`crossplane.io/external-name` annotation. If the provider crashes, restarts, or -can't reach the network after creating the resource it can't save the -annotation. If this happens, the provider leaks the resource. - A provider can detect that it might have leaked a resource. If the provider -thinks it might have leaked a resource, it stops and waits for human -intervention. - -When a provider thinks it may have leaked a resource it creates an event -associated with the managed resource: - -```console -cannot determine creation result - remove the crossplane.io/external-create-pending annotation if it is safe to proceed -``` - +thinks it might have leaked a resource, it stops reconciling it until you tell +the provider it's safe to proceed. -The provider uses annotations to detect that it might have leaked a resource: +When a provider thinks it might have leaked a resource it creates a `cannot +determine creation result` event associated with the managed resource. Use +`kubectl describe` to see the event. -* Before creating a resource, it writes the current time to the - `crossplane.io/external-create-pending` annotation. -* If the create succeeds, it writes the current times to the - `crossplane.io/external-create-succeeded` annotation. -* If the create fails, it writes the current time to the - `crossplane.io/external-create-failed` annotation. - -Providers try hard to write these annotations to the managed resource. -If writing the annotations fails, the provider retries several times. - -When a provider sees a managed resource with: +```shell {copy-lines="1"} +kubectl describe queue my-sqs-queue -* A `crossplane.io/external-create-pending` annotation. -* No `crossplane.io/external-create-succeeded` or - `crossplane.io/external-create-failed` annotation. +# Removed for brevity -It knows that it might have created the resource and failed to record its -`crossplane.io/external-name` annotation. When this happens, the provider -refuses to create the resource and waits for human intervention. +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning CannotInitializeManagedResource 29m (x19 over 19h) managed/queue.sqs.aws.crossplane.io cannot determine creation result - remove the crossplane.io/external-create-pending annotation if it is safe to proceed +``` {{}} The safest thing for a provider to do when it detects that it might have leaked a resource is to stop and wait for human intervention. -Waiting for human intervention ensures the provider doesn't create a duplicate -of the leaked resource. Creating duplicate resources can be costly and -dangerous. +This ensures the provider doesn't create duplicates of the leaked resource. +Duplicate resources can be costly and dangerous. +{{}} -If you see the `cannot determine creation result` error, inspect the external -system for resources created around the time recorded in the -`crossplane.io/external-create-pending` annotation. +Providers use the creation annotations to detect that they might have leaked a +resource. -If you find a leaked resource, and it's safe to do so, delete it. Remove the -`crossplane.io/external-create-pending` annotation from the managed resource. +Each time a provider reconciles a managed resource it checks the resource's +creation annotations. If the provider sees a create pending time that's more +recent than the most recent create succeeded or create failed time, it knows +that it might have leaked a resource. -If you don't find a leaked resource, remove the -`crossplane.io/external-create-pending` annotation from the managed resource. +{{}} +Providers don't remove the creation annotations. They use the timestamps to +determine which is most recent. It's normal for a managed resource to have +several creation annotations. {{}} -{{}} -Providers never delete the `crossplane.io/external-create-pending`, -`crossplane.io/external-create-succeeded`, or -`crossplane.io/external-create-failed` annotations. +The provider knows it might have leaked a resource because it updates all the +resource's annotations at the same time. If the provider couldn't update the +creation annotations after it created the resource, it also couldn't update the +`crossplane.io/external-name` annotation. + +{{}} +Inspect the external system when resources have a `cannot determine creation +result` error. -If several of these annotations exist, a provider uses the timestamps to -determine the state of the resource. +Use the timestamp from the `crossplane.io/external-create-pending` annotation to +determine when the provider might have leaked a resource. Look for resources +created around this time. -For example if a provider sees a managed resource with a -`crossplane.io/external-create-pending` annotation that's newer than its -`crossplane.io/external-create-succeded` annotation it knows it might have -failed to record its `crossplane.io/external-name` annotation. +If you find a leaked resource, and it's safe to do so, delete it from the +external system. -This could happen if a provider successfully created the resource at some point -in the past, the later failed to recreate it. +Remove the `crossplane.io/external-create-pending` annotation from the managed +resource after you're sure no leaked resource exists. This tells the provider to +resume reconciliation of the managed resource. {{}} -{{}} -Providers also use these annotations to avoid leaking resources in other ways. +Providers also use the creation annotations to avoid leaking resources. -Writing the `crossplane.io/external-create-pending` annotation lets the provider -know it's reconciling the latest version of the managed resource. The write -would fail if the provider was reconciling a stale, cached version. +When a provider writes the `crossplane.io/external-create-pending` annotation it +knows it's reconciling the latest version of the managed resource. The write +would fail if the provider was reconciling an old version of the managed +resource. -Saving a timestamp to the `crossplane.io/external-create-succeeded` annotation -lets the provider know that it probably created the resource, even if the -external system says it doesn't exist. +If the provider reconciled an old version with an outdated +`crossplane.io/external-name` annotation it could mistakenly determine that the +resource didn't exist. The provider would create a new resource, and leak the +existing one. -Sometimes there is a delay between the provider creating a resource, and the -system reporting that it exists. If this is the case, the provider waits an -appropriate amount of time for the resource to appear in the external system. -{{}} +Some external systems have a delay between when a provider creates a resource +and when the system reports that it exists. The provider uses the most recent +create succeeded time to account for this delay. + +If the provider didn't account for the delay, it could mistakenly determine +that the resource didn't exist. The provider would create a new resource, and +leak the existing one. ### Paused Manually applying the `crossplane.io/paused` annotation causes the Provider to From 1341e35e388c131877f4f7c1e37e62430dac9375 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 11:30:05 -0800 Subject: [PATCH 03/11] Be specific about circumstances where we'll leak Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index d29feed06..4ed0a97d0 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -635,7 +635,8 @@ my-rds-instance True True my-custom-name 11m ### Creation annotations -In some rare situations a provider can forget that it created a resource. When +When an external system like AWS generates nondeterministic resource names it's +possible for a provider to create a resource but not record that it did. When this happens the provider can't manage the resource. {{}} From 741929cacfc6529c92ed471047f2c59c8eb59c29 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 11:30:43 -0800 Subject: [PATCH 04/11] Use kubectl get -o yaml to get YAML manifest Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index 4ed0a97d0..cf4859877 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -658,7 +658,7 @@ Use `kubectl get` to view the annotations on a managed resource. For example, an AWS VPC resource: ```yaml {label="creation" copy-lines="2-9"} -$ kubectl get vpc my-vpc +$ kubectl get -o yaml vpc my-vpc apiVersion: ec2.aws.upbound.io/v1beta1 kind: VPC metadata: From 9a5ee38451685eac603af96056cadd92ef84c414 Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 11:32:24 -0800 Subject: [PATCH 05/11] Use 'lookup' rather than 'find' an MR in an external system Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index cf4859877..a2428e4bf 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -671,7 +671,7 @@ metadata: A provider uses the {{}}crossplane.io/external-name{{}} -annotation to find a resource in an external system, for example AWS. +annotation to lookup a managed resource in an external system. If the provider can't find a managed resource in an external system, it thinks the resource doesn't exist. When the provider thinks a resource doesn't exist From 26c4ff49576120b062cc82fa741a079858fbc6ad Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 11:36:19 -0800 Subject: [PATCH 06/11] Describe external name generation as nondeterministic, not unpredictable Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index a2428e4bf..458fc4bdc 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -678,8 +678,8 @@ the resource doesn't exist. When the provider thinks a resource doesn't exist it creates the resource. Some external systems don't let a provider specify a resource's name when the -provider creates it. Instead the external system generates an unpredictable name -and returns it to the provider. +provider creates it. Instead the external system generates an nondeterministic +name and returns it to the provider. When the external system generates the resource's name, it's critical that the provider saves it to the managed resource's `crossplane.io/external-name` From c0dbc0f5cf6ce980374bf38e8b564d12cb085a7d Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 12:27:21 -0800 Subject: [PATCH 07/11] Don't explicitly mention that saving annotations is critical Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index 458fc4bdc..14684d1b0 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -653,7 +653,6 @@ Providers set three creation annotations to avoid and detect leaked resources: * `crossplane.io/external-create-failed` - The last time the provider failed to create the resource. - Use `kubectl get` to view the annotations on a managed resource. For example, an AWS VPC resource: @@ -681,9 +680,9 @@ Some external systems don't let a provider specify a resource's name when the provider creates it. Instead the external system generates an nondeterministic name and returns it to the provider. -When the external system generates the resource's name, it's critical that the -provider saves it to the managed resource's `crossplane.io/external-name` -annotation. If it doesn't, it leaks the resource. +When the external system generates the resource's name, the provider attempts to +save it to the managed resource's `crossplane.io/external-name` annotation. If +it doesn't, it _leaks_ the resource. A provider can't guarantee that it can save the annotation. The provider could restart or lose network connectivity between creating the resource and saving From ecaf5c20dd07882309daa852f0a1a2eea881d27e Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 12:28:49 -0800 Subject: [PATCH 08/11] Use if-then for inspecting external system Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index 14684d1b0..f3ea52359 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -740,8 +740,8 @@ creation annotations after it created the resource, it also couldn't update the `crossplane.io/external-name` annotation. {{}} -Inspect the external system when resources have a `cannot determine creation -result` error. +If a resource has a `cannot determine creation result` error, inspect the +external system. Use the timestamp from the `crossplane.io/external-create-pending` annotation to determine when the provider might have leaked a resource. Look for resources From eafb5274ae69cdde02477954e422053e398ebcfc Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 12:29:44 -0800 Subject: [PATCH 09/11] Clarify that resuming reconciliation recreates MR Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index f3ea52359..ec8084031 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -752,7 +752,7 @@ external system. Remove the `crossplane.io/external-create-pending` annotation from the managed resource after you're sure no leaked resource exists. This tells the provider to -resume reconciliation of the managed resource. +resume reconciliation of and recreate the managed resource. {{}} Providers also use the creation annotations to avoid leaking resources. From c50047e7169a20ecd04ea7db6ca6134f733fb24a Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 12:31:29 -0800 Subject: [PATCH 10/11] Move hint about why we wait for human intervention Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index ec8084031..0313ced23 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -712,14 +712,6 @@ Events: Warning CannotInitializeManagedResource 29m (x19 over 19h) managed/queue.sqs.aws.crossplane.io cannot determine creation result - remove the crossplane.io/external-create-pending annotation if it is safe to proceed ``` -{{}} -The safest thing for a provider to do when it detects that it might have leaked -a resource is to stop and wait for human intervention. - -This ensures the provider doesn't create duplicates of the leaked resource. -Duplicate resources can be costly and dangerous. -{{}} - Providers use the creation annotations to detect that they might have leaked a resource. @@ -739,6 +731,14 @@ resource's annotations at the same time. If the provider couldn't update the creation annotations after it created the resource, it also couldn't update the `crossplane.io/external-name` annotation. +{{}} +The safest thing for a provider to do when it detects that it might have leaked +a resource is to stop and wait for human intervention. + +This ensures the provider doesn't create duplicates of the leaked resource. +Duplicate resources can be costly and dangerous. +{{}} + {{}} If a resource has a `cannot determine creation result` error, inspect the external system. From 0aef9a688ba9a418614a88cedd493cb152548f6d Mon Sep 17 00:00:00 2001 From: Nic Cope Date: Wed, 31 Jan 2024 12:45:29 -0800 Subject: [PATCH 11/11] Don't say "think", restructure important note about dupes Signed-off-by: Nic Cope --- content/master/concepts/managed-resources.md | 28 +++++++++----------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/content/master/concepts/managed-resources.md b/content/master/concepts/managed-resources.md index 0313ced23..c9ee2ef34 100644 --- a/content/master/concepts/managed-resources.md +++ b/content/master/concepts/managed-resources.md @@ -672,9 +672,9 @@ A provider uses the {{}}crossplane.io/external-name{{}} annotation to lookup a managed resource in an external system. -If the provider can't find a managed resource in an external system, it thinks -the resource doesn't exist. When the provider thinks a resource doesn't exist -it creates the resource. +The provider looks up the resource in the external system to determine if it +exists, and if it matches the managed resource's desired state. If the provider +can't find the resource, it creates it. Some external systems don't let a provider specify a resource's name when the provider creates it. Instead the external system generates an nondeterministic @@ -688,14 +688,20 @@ A provider can't guarantee that it can save the annotation. The provider could restart or lose network connectivity between creating the resource and saving the annotation. +A provider can detect that it might have leaked a resource. If the provider +thinks it might have leaked a resource, it stops reconciling it until you tell +the provider it's safe to proceed. + {{}} Anytime an external system generates a resource's name there is a risk the provider could leak the resource. -{{}} -A provider can detect that it might have leaked a resource. If the provider -thinks it might have leaked a resource, it stops reconciling it until you tell -the provider it's safe to proceed. +The safest thing for a provider to do when it detects that it might have leaked +a resource is to stop and wait for human intervention. + +This ensures the provider doesn't create duplicates of the leaked resource. +Duplicate resources can be costly and dangerous. +{{}} When a provider thinks it might have leaked a resource it creates a `cannot determine creation result` event associated with the managed resource. Use @@ -731,14 +737,6 @@ resource's annotations at the same time. If the provider couldn't update the creation annotations after it created the resource, it also couldn't update the `crossplane.io/external-name` annotation. -{{}} -The safest thing for a provider to do when it detects that it might have leaked -a resource is to stop and wait for human intervention. - -This ensures the provider doesn't create duplicates of the leaked resource. -Duplicate resources can be costly and dangerous. -{{}} - {{}} If a resource has a `cannot determine creation result` error, inspect the external system.