Undeploy an API without having to delete the Host #1532

Nanashi-lab · 2025-04-16T07:30:13Z

/closes #1494

Opened a new endpoint Delete /v1/api/deployments/{site}/{id}/{version} which undeploys a single api/version from the site
Make sure it properly deals with, Api not found or Site not found
Updated undeploy to be able to remove component constrain

Nanashi-lab · 2025-04-16T07:31:00Z

golem-worker-service/src/api/api_deployment.rs

@@ -182,4 +183,102 @@ impl ApiDeploymentApi {

        Ok(Json("API deployment deleted".to_string()))
    }
+


This is the endpoint. /:site/:id/:version

Nanashi-lab · 2025-04-16T07:32:06Z

golem-worker-service-base/src/service/gateway/api_deployment.rs

@@ -457,10 +461,32 @@ impl<Namespace: GolemNamespace, AuthCtx: GolemAuthCtx> ApiDeploymentService<Auth
        }

        if !remove_deployment_records.is_empty() {
+            // 3. Get the specific API definitions being undeployed


Removing component constraints during undeploy. There is already a test in service_test. Here we do two loops, one to filter by name, so there is no namespace conflict. And another loop to get the actual api definition, it is possible to do this in a single loop.

Nanashi-lab · 2025-04-16T07:33:38Z

I side stepped the Issue #1529 by adding those parts manually back in. Did Cargo Make fix to check if everything is working.

mschuwalow · 2025-04-16T10:55:15Z

Please also add a test for the new endpoint to https://github.com/golemcloud/golem/tree/main/integration-tests/tests/api

Nanashi-lab · 2025-04-16T15:56:31Z

@mschuwalow All local deployment test always fail for me, and this has been the case for a while, since 2-3 week old HEAD.

Failed tests:
 - api::api_security::get_api_security_scheme (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_security_scheme)
 - api::api_security::create_api_security_scheme (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_security_scheme)
 - api::api_deployment::create_multiple_api_deployments_and_update_component_1 (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)
 - api::api_deployment::create_multiple_api_deployments_and_update_component_2 (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)
 - api::api_deployment::get_all_api_deployments (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)
 - api::invocation_context::invocation_context_test (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)
�[2m2025-04-16T15:52:29.295200Z�[0m �[32m INFO�[0m �[2mgolem_test_framework::components�[0m�[2m:�[0m [componentsvc] �[2m2025-04-16T15:52:29.295101Z�[0m �[34mDEBUG�[0m �[2mtonic::transport::channel::service::connection�[0m�[2m:�[0m connection task error: hyper::Error(Io, Custom { kind: BrokenPipe, error: "connection closed because of a broken pipe" })
 - api::api_deployment::create_api_deployment_and_update_component (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)
 - api::api_deployment::create_and_get_api_deployment (called `Result::unwrap()` on an `Err` value: not available on GRPC API: create_api_deployment)

vigoo · 2025-04-17T10:44:50Z

Those tests need to be run in http mode which is currently controlled by an environment variable (this is not ideal but has its reasons). You can see the details of it in the makefile

…tests

Nanashi-lab · 2025-04-18T09:58:32Z

golem-test-framework/src/components/worker_service/mod.rs

@@ -1247,6 +1247,18 @@ pub trait WorkerService: WorkerServiceInternal {
    }

    async fn kill(&self);
+


added undeploy to test-framework

Nanashi-lab · 2025-04-19T12:04:48Z

@mschuwalow This PR is ready for review, All checks passed

mschuwalow · 2025-04-21T18:28:31Z

golem-worker-service/src/api/api_deployment.rs

+        let (host, subdomain) = if let Some(idx) = site.find('.') {
+            let (subdomain, host) = site.split_at(idx);
+            (
+                host.trim_start_matches('.').to_string(),
+                Some(subdomain.to_string()),
+            )
+        } else {
+            (site, None)
+        };
+
+        let api_site = ApiSite { host, subdomain };
+
+        // Check if the site exists
+        let site_exists = self
+            .deployment_service
+            .get_by_site(&ApiSiteString::from(&api_site))
+            .await?
+            .is_some();
+
+        if !site_exists {
+            return Err(ApiEndpointError::not_found(safe(
+                "Site not found".to_string(),
+            )));
+        }
+
+        // Check if the API definition exists
+        let api_definition_exists = self
+            .deployment_service
+            .get_by_id(&namespace, Some(api_definition_key.id.clone()))
+            .await?
+            .iter()
+            .any(|deployment| {
+                deployment.api_definition_keys.iter().any(|key| {
+                    key.id == api_definition_key.id && key.version == api_definition_key.version
+                })
+            });
+
+        if !api_definition_exists {
+            return Err(ApiEndpointError::not_found(safe(
+                "API definition not found".to_string(),
+            )));
+        }
+
+        let api_deployment = gateway_api_deployment::ApiDeploymentRequest {
+            namespace: namespace.clone(),
+            api_definition_keys: vec![api_definition_key],
+            site: api_site,
+        };
+
+        self.deployment_service
+            .undeploy(&api_deployment, auth_ctx)
+            .await?;
+


Please move this logic out of the route into the ApiDeploymentService. Try to only do simple data transformations here.
The new function in the ApiDeploymentService should take a generic AuthCtx as a parameter, the concrete EmptyAuthCtx should be provided to the function when calling it here.

Looks good otherwise, thanks!

Nanashi-lab · 2025-04-22T09:08:55Z

I moved the logic into service, The only thing in route, now is error/success message. {subdomain, domain} is converted into APIsite. All the logic of checking if a site or deployment is available has been moved to ApiDeploymentService.

I am not sure why the api-and-integration-tests failed, which passed in the previous check, I haven't changed the code, I dont think it is related to my PR. Could you re-run it ? Thank you.

or between this check and previous check all tests pass, and there has been no change between this and last. (Just removed a empty file)

afsalthaj · 2025-04-23T02:38:15Z

golem-worker-service-base/src/service/gateway/api_deployment.rs

+            }
+
+            // 4. Remove component constraints
+            self.remove_component_constraints(definitions_to_undeploy, auth_ctx)


I hope you have a test around these

Test for checking if component constraints have been removed, no not yet, I will add one

afsalthaj · 2025-04-23T02:40:02Z

golem-worker-service/src/api/api_deployment.rs

+        };
+
+        // Split the site string into host and subdomain parts
+        let (host, subdomain) = if let Some(idx) = site.find('.') {


This is unreliable. I don't think you can really split host and site like that.

I fix this, there should be a way to convert ApiSiteString to ApiSite already within golem and use that.

Finding host and subdomain by splitting dots is unreliable and may not work in all cases.

I don't think there is a way to convert ApiSiteString to ApiSite. I think it's good to avoid that need, if you see what I mean.

afsalthaj · 2025-04-23T02:45:25Z

golem-worker-service/src/api/api_deployment.rs

+            (site, None)
+        };
+
+        let api_site = ApiSite { host, subdomain };


I think ApiSiteString may be of your help. may be your undeploy function can take this one instead of ApiSite.

That to undeploy an API from a site, you really don't need to logically split domain and subdomain anymore.

afsalthaj · 2025-04-23T02:47:49Z

@Nanashi-lab thanks a lot for this addition. LGTM, except for one thing. We should avoid the need to convert a site string to a host and subdomain. Also, excuse for the delay in review :)

Nanashi-lab · 2025-04-24T07:13:45Z

@afsalthaj Finished with the changes, you would have to rerun checks, failure is not related to the PR. Tested the changes locally.

Original Golem

undeploy uses ApiDefinitionRequest which require APISite
undeploy is used in a single service-test

My Old PR

undeploy_api -> undeploy_api_internal(convert APIStringSite to APISite) -> undeploy_api(Service)-> undeploy(Service)

New PR

undeploy_api -> undeploy_api_internal-> undeploy(Service)
New undeploy accepts APIStringSite and undeploys
Made sure the service_test uses the new undeploy
New undeploy checks for site and deployment, undeploys, removes component constrains and changes it to draft
Added a Test for if undeploy properly removes component constrain
- Creates and Deploys API
- Tries to update the component, FAILS
- undeploys API
- Updates the component, SUCCEEDS

Nanashi-lab · 2025-04-24T07:17:40Z

golem-worker-service-base/src/service/gateway/api_deployment.rs

-                || deployment_record.subdomain != deployment.site.subdomain
-                || deployment_record.host != deployment.site.host
-            {
-                error!("Undeploying API definition - failed, site used by another API (under another namespace/API)");


I removed this check, delete also doesn't have this check. It shouldn't be possible to have different namespace, but same subdomain and host.

afsalthaj · 2025-04-24T11:22:59Z

Let's merge once the checks are good. LGTM now

Nanashi-lab commented Apr 16, 2025

View reviewed changes

Nanashi-lab marked this pull request as draft April 17, 2025 06:29

Nanashi-lab added 5 commits April 18, 2025 13:26

Add Rest API endpoint for undepoly and updated undeploy

ad935a1

Fix Show error when site or api definition is not available

7240530

Add undeploy_api to test-framework, Add undeploy test to integration-…

8514460

…tests

Fixed Flaky test

32b505d

Merge main, & cargo make fix

d174427

Nanashi-lab force-pushed the single-delete branch from f233a91 to d174427 Compare April 18, 2025 09:56

Nanashi-lab commented Apr 18, 2025

View reviewed changes

Nanashi-lab marked this pull request as ready for review April 18, 2025 10:16

Merge branch 'main' into single-delete

a77f5e1

mschuwalow reviewed Apr 21, 2025

View reviewed changes

Nanashi-lab added 3 commits April 22, 2025 13:07

Refactor logic into Service

18d5334

Merge branch 'main' into single-delete

cf6901f

Fixed Namespace change from merging with main

01f1d76

Cargo make fix

96d18c4

Nanashi-lab requested a review from mschuwalow April 22, 2025 12:39

mschuwalow approved these changes Apr 22, 2025

View reviewed changes

afsalthaj reviewed Apr 23, 2025

View reviewed changes

afsalthaj requested changes Apr 23, 2025

View reviewed changes

afsalthaj reviewed Apr 23, 2025

View reviewed changes

Merge branch 'main' into single-delete

d82da83

Nanashi-lab added 3 commits April 24, 2025 10:30

Refactored Undeploy, uses APIStringSite

28b7de9

Add component constrain undeploy test, Clean up code

58b4de1

Cargo make fix

a953b2b

Nanashi-lab commented Apr 24, 2025

View reviewed changes

Nanashi-lab requested a review from afsalthaj April 24, 2025 07:19

afsalthaj approved these changes Apr 24, 2025

View reviewed changes

mschuwalow merged commit 3eb7602 into golemcloud:main Apr 28, 2025
22 checks passed

Nanashi-lab deleted the single-delete branch May 21, 2025 03:27

		@@ -182,4 +183,102 @@ impl ApiDeploymentApi {

		Ok(Json("API deployment deleted".to_string()))
		}

		@@ -1247,6 +1247,18 @@ pub trait WorkerService: WorkerServiceInternal {
		}

		async fn kill(&self);

Undeploy an API without having to delete the Host #1532

Undeploy an API without having to delete the Host #1532

Uh oh!

Conversation

Nanashi-lab commented Apr 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Nanashi-lab Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Nanashi-lab commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mschuwalow commented Apr 16, 2025

Uh oh!

Nanashi-lab commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vigoo commented Apr 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Nanashi-lab commented Apr 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mschuwalow Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Nanashi-lab commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

afsalthaj Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

afsalthaj commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nanashi-lab commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

afsalthaj commented Apr 24, 2025

Uh oh!

Uh oh!

Uh oh!

Nanashi-lab Apr 16, 2025 •

edited

Loading

Nanashi-lab commented Apr 16, 2025 •

edited

Loading

Nanashi-lab commented Apr 16, 2025 •

edited

Loading

Nanashi-lab commented Apr 19, 2025 •

edited

Loading

mschuwalow Apr 21, 2025 •

edited

Loading

Nanashi-lab commented Apr 22, 2025 •

edited

Loading

afsalthaj Apr 23, 2025 •

edited

Loading

afsalthaj commented Apr 23, 2025 •

edited

Loading

Nanashi-lab commented Apr 24, 2025 •

edited

Loading