VMSS Flex VM instances cannot be deleted #19588

vishwa-krishnan · 2022-11-17T19:52:16Z

Bug Report

Imports:

	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.1.0
	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute v1.0.0

Package in question:
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute v1.0.0

Go version: 1.19.1

What happened?
I am trying to delete a VMSS Flex VM instance, but the SDK APIs dont seem functional.
What did you expect or want to happen?
VM instance deletion process should start and API should succeed.
How can we reproduce it?
Create VMSS Flex, create instances and try deleting using SDK.
Environment

Current code being tested:

force := true
	r, err := vmsClient.BeginDelete(ctx, rgname, vm, &armcompute.VirtualMachinesClientBeginDeleteOptions{
		ForceDeletion: &force,
	})
	if err != nil {
		fmt.Println(err)
		return
	}

I am using the vm client instead of the vmss client api since VMSS API dont seem to support deletion of Flex instances. The VMSS API requires instance IDs that are integers in the backend and those dont exist for the VM Flex instances:

vmssclient.BeginDeleteInstances(ctx, rgname, cpuVmssName, armcompute.VirtualMachineScaleSetVMInstanceRequiredIDs{
		InstanceIDs: <REQUIRED; The virtual machine scale set instance ids.>,
	}

Response from 1:

Deleting VM: az-cpu-test-wus3-flex-1_3166bfac
DELETE https://management.azure.com/subscriptions/.../providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac

--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE UNAVAILABLE
--------------------------------------------------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

The text was updated successfully, but these errors were encountered:

ghost · 2022-11-17T19:52:23Z

Hi @vishwa-krishnan. Thank you for your feedback and we will look into it soon. Meanwhile, feel free to share your experience using the Azure SDK in this survey.

jhendrixMSFT · 2022-11-17T20:33:31Z

If I understand correctly, you used the VirtualMachineScaleSets client to create the resource and are then attempting to delete it with the VirtualMachines client? If this is the case, I don't believe this is supported as the resource IDs are different (needs to be verified by the service team). Can you please confirm the APIs you're using?

vishwa-krishnan · 2022-11-17T20:48:23Z

The resources are created beforehand through Terraform, there is no golang code creating these. On deletion, initially i tried to use the VMSS API as stated above: vmssclient.BeginDeleteInstances(...), this did not work since the instanceIDs required are not available on the flex instances.

The resourceID for the VM Flex instance through the metadata endpoint is:

"/subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac"

Hence, it made sense to try out the VMs API vmsClient.BeginDelete(...). That fails with Response mentioned above - 400 Bad Request, Invalid URL

jhendrixMSFT · 2022-11-17T20:54:00Z

Thanks for clarifying.

Can you please enable logging and share the trace?

vishwa-krishnan · 2022-11-17T21:54:06Z

I dont see anything being printed do i need to use azure solution for tracing and monitoring in the portal ?

	os.Setenv("AZURE_SDK_GO_LOGGING", "all")

	ctx := context.Background()
	tracer := tracing.NewTracer(func(ctx context.Context, spanName string,
		options *tracing.SpanOptions) (context.Context, tracing.Span) {
		return ctx, tracing.NewSpan(tracing.SpanImpl{})
	}, nil)

	ctx, span := tracer.Start(ctx, "test-azure-sdk", nil)
	defer span.End()

The logging prints nothing, I just set the env var. Im not sure whether tracing is done right. Can you please tell me whether this is done right ?

jhendrixMSFT · 2022-11-17T21:58:37Z

You can't set the env var programmatically as it's read during an init() in azcore. You'll need to set it as part of the environment before you run your application.

Also, there's no need to create a tracer as the logging writes to stderr by default.

jhendrixMSFT · 2022-11-17T22:06:58Z

If you must set it programmatically, use the log.SetListener() function.

vishwa-krishnan · 2022-11-17T22:07:24Z

This is the logging:

Enter VM name to delete in VMSS az-cpu-test-wus3-flex-1:az-cpu-test-wus3-flex-1_3166bfac
Scaling down by 1 now ...
Deleting VM: az-cpu-test-wus3-flex-1_3166bfac
[Nov 17 14:06:30.708809] Retry: =====> Try=1
[Nov 17 14:06:30.709340] Authentication: *azidentity.ClientSecretCredential.GetToken() acquired a token for scope https://management.core.windows.net//.default

[Nov 17 14:06:30.709402] Request: ==> OUTGOING REQUEST (Try=1)
   DELETE https://management.azure.com/subscriptions/3358be8c-ffc1-4165-965c-5a07f7a420a8/resourceGroups/rg-ecp-hydra-prod-1-wus3-1/providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac%0A?api-version=2022-03-01&forceDeletion=REDACTED
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armcompute/v1.0.0 (go1.19.1; darwin)

[Nov 17 14:06:30.774220] Response: ==> REQUEST/RESPONSE (Try=1/64.704292ms, OpTime=64.757167ms) -- RESPONSE RECEIVED
   DELETE https://management.azure.com/subscriptions/3358be8c-ffc1-4165-965c-5a07f7a420a8/resourceGroups/rg-ecp-hydra-prod-1-wus3-1/providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac%0A?api-version=2022-03-01&forceDeletion=REDACTED
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armcompute/v1.0.0 (go1.19.1; darwin)
   --------------------------------------------------------------------------------
   RESPONSE Status: 400 Bad Request
   Content-Length: 324
   Content-Type: text/html; charset=us-ascii
   Date: Thu, 17 Nov 2022 22:06:29 GMT
   Server: Microsoft-HTTPAPI/2.0

[Nov 17 14:06:30.774274] Retry: response 400
[Nov 17 14:06:30.774290] Retry: exit due to non-retriable status code
DELETE https://management.azure.com/subscriptions/3358be8c-ffc1-4165-965c-5a07f7a420a8/resourceGroups/rg-ecp-hydra-prod-1-wus3-1/providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac

--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE UNAVAILABLE
--------------------------------------------------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

--------------------------------------------------------------------------------

vishwa-krishnan · 2022-11-17T22:10:00Z

I can see some character appended, let me try to fix it and check

vishwa-krishnan · 2022-11-17T22:13:29Z

Ok it looks like that character was getting appended in the go code. It seems to be working now.

[Nov 17 14:12:15.017928] Response: ==> REQUEST/RESPONSE (Try=1/475.825333ms, OpTime=475.996333ms) -- RESPONSE RECEIVED
   DELETE https://management.azure.com/subscriptions/3358be8c-ffc1-4165-965c-5a07f7a420a8/resourceGroups/rg-ecp-hydra-prod-1-wus3-1/providers/Microsoft.Compute/virtualMachines/az-cpu-test-wus3-flex-1_3166bfac?api-version=2022-03-01&forceDeletion=REDACTED
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armcompute/v1.0.0 (go1.19.1; darwin)
   --------------------------------------------------------------------------------
   RESPONSE Status: 202 Accepted
   Azure-Asyncnotification: REDACTED
   Azure-Asyncoperation: REDACTED
   Cache-Control: no-cache
   Content-Length: 0
   Date: Thu, 17 Nov 2022 22:12:14 GMT
   Expires: -1
   Location: REDACTED
   Pragma: no-cache
   Retry-After: 10
   Server: Microsoft-HTTPAPI/2.0
   Strict-Transport-Security: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Ratelimit-Remaining-Resource: REDACTED
   X-Ms-Ratelimit-Remaining-Subscription-Deletes: REDACTED
   X-Ms-Request-Id: 3744f871-dc05-43ec-bcf7-8aa59f765ab4
   X-Ms-Routing-Request-Id: REDACTED

jhendrixMSFT · 2022-11-17T22:14:24Z

Was it a copy/paste thing?

vishwa-krishnan · 2022-11-17T22:15:43Z

Yep looks like it. The logging helped a lot, thank you !
Also could you please tell me whether there is another API that can be used instead of VM Delete APIs for Flex instances that are tied to VMSS ?

jhendrixMSFT · 2022-11-17T22:18:00Z

That I don't know, I'll defer to @lirenhe team to answer.

fitzgeraldsteele · 2022-11-21T15:18:57Z

Hi @vishwa-krishnan - No need to unlink it from the VMSS before you delete it. You should be able to use either the VM delete or the VMSS Delete Instances (passing in the instanceIDs parameter). No preference from the platform side.

vishwa-krishnan · 2022-11-28T20:07:02Z

VMSS Delete Instances (passing in the instanceIDs parameter).
@fitzgeraldsteele the backend fails when this ID is passed stating its not an integer, for flex instance actually there is no such ID, if you consider the last part of the name as the ID - then its not an integer.

vishwa-krishnan · 2022-12-14T04:46:19Z

Ok i see it happening, in this case the name is the instance IDs. Thank you, you may close this issue.

ghost · 2022-12-14T14:55:48Z

Hi @vishwa-krishnan. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost · 2022-12-21T16:03:16Z

Hi @vishwa-krishnan, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. labels Nov 17, 2022

ghost added the question The issue doesn't require a change to the product in order to be resolved. Most issues start as that label Nov 17, 2022

jhendrixMSFT added Compute Mgmt This issue is related to a management-plane library. and removed needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Nov 17, 2022

ghost added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Nov 17, 2022

jhendrixMSFT assigned lirenhe Nov 17, 2022

lirenhe assigned ArcturusZhang Nov 18, 2022

This comment was marked as off-topic.

Sign in to view

jhendrixMSFT added the needs-author-feedback Workflow: More information is needed from author to address the issue. label Nov 22, 2022

ghost removed the needs-author-feedback Workflow: More information is needed from author to address the issue. label Nov 28, 2022

jhendrixMSFT added the issue-addressed Workflow: The Azure SDK team believes it to be addressed and ready to close. label Dec 14, 2022

ghost removed the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Dec 14, 2022

ghost closed this as completed Dec 21, 2022

tadelesh mentioned this issue Dec 26, 2022

Unable to get instance view for VMSS flex VMs #19707

Closed

github-actions bot locked and limited conversation to collaborators Apr 11, 2023

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VMSS Flex VM instances cannot be deleted #19588

VMSS Flex VM instances cannot be deleted #19588

vishwa-krishnan commented Nov 17, 2022 •

edited

Loading

ghost commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022 •

edited

Loading

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022 •

edited

Loading

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

This comment was marked as off-topic.

fitzgeraldsteele commented Nov 21, 2022

vishwa-krishnan commented Nov 28, 2022

vishwa-krishnan commented Dec 14, 2022

ghost commented Dec 14, 2022

ghost commented Dec 21, 2022

VMSS Flex VM instances cannot be deleted #19588

VMSS Flex VM instances cannot be deleted #19588

Comments

vishwa-krishnan commented Nov 17, 2022 • edited Loading

Bug Report

ghost commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022 • edited Loading

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022 • edited Loading

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

vishwa-krishnan commented Nov 17, 2022

jhendrixMSFT commented Nov 17, 2022

This comment was marked as off-topic.

fitzgeraldsteele commented Nov 21, 2022

vishwa-krishnan commented Nov 28, 2022

vishwa-krishnan commented Dec 14, 2022

ghost commented Dec 14, 2022

ghost commented Dec 21, 2022

vishwa-krishnan commented Nov 17, 2022 •

edited

Loading

vishwa-krishnan commented Nov 17, 2022 •

edited

Loading

jhendrixMSFT commented Nov 17, 2022 •

edited

Loading