Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DscConfigurationClient.GetContent is broken since 2020-01-13-preview/automation API version #17591

Closed
dz-sourced opened this issue Apr 14, 2022 · 19 comments
Assignees
Labels
Automation bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Previous Versions Work related to track1 and track1.5 SDKs Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@dz-sourced
Copy link

Bug Report

Summary:
When trying to read back a DSC Configuration content using func (client DscConfigurationClient) GetContent an error happens in the version of the API 2020-01-13-preview/automation and later while trying to unmarshall DSC Configuration content from JSON. DSC Configurations are not valid JSON objects and thus unmarshalling fails. This issue was not happening in 2018-06-30-preview/automation and earlier, where DSC Configuration content was treated as a plain string value. I have confirmed the issue on the main branch.

Details:
In the 2020-01-13-preview/automation the actual failure seems to happen when trying to unmarshall DSC Configuration content from JSON:

The error that I get when trying to read back a sample DSC Configuration like Configuration test {} is:

automation.DscConfigurationClient#GetContent: Failure responding to request: StatusCode=200 -- Original Error: Error occurred unmarshalling JSON - Error = 'invalid character 'C' looking for beginning of value' JSON = 'Configuration test{}'

This was not happening in the previous version of the API 2018-06-30-preview/automation and the content value was just passed as is:

Finally, the root cause can be traced to the change in the Swagger API definitions. The latest version of the GetContent API defines the output as a string type, which I assume is tried to be de-serialized from JSON by default:
https://github.com/Azure/azure-rest-api-specs/blob/3034ada77d465b317cda51250a92454824cca06b/specification/automation/resource-manager/Microsoft.Automation/stable/2019-06-01/dscConfiguration.json#L309

In the earlier version of the API spec the output type is set to file:
https://github.com/Azure/azure-rest-api-specs/blob/3026119ab41bbce77275cfa3a1afbabf43af5aea/specification/automation/resource-manager/Microsoft.Automation/stable/2015-10-31/dscConfiguration.json#L309

I don't know whether this was intentional API spec change, or whether there is a need to provide further type modifiers in the Swagger API or whether it's an issue with the SDK codegen tool.

Here is a small script that demostrates the problem https://gist.github.com/dz-sourced/11654206078fcc159c54564de6758659
It creates a resource group, an automation account, a sample empty DSC Configuration and the tries to read it using older and new API versions. You should be able to see that read is succesful in 2018-06-30-preview/automation and fails in the 2020-01-13-preview/ version.

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Apr 14, 2022
@jhendrixMSFT jhendrixMSFT added Automation Mgmt This issue is related to a management-plane library. labels Apr 14, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Apr 14, 2022
@lirenhe lirenhe assigned ArcturusZhang and unassigned lirenhe Apr 19, 2022
@ArcturusZhang
Copy link
Member

ArcturusZhang commented Apr 19, 2022

Hi @dz-sourced thank you for this issue!

By looking at the error message, it is very likely to be an issue that the service is returning something uncompatible with its swagger. But could you please paste the request log here so that we could confirm where does this issue come from?
You could get the request logs by exporting an env var AZURE_GO_SDK_LOG_FILE=DEBUG

@dz-sourced
Copy link
Author

Hey @ArcturusZhang, do I need to initialize the logger in any way? I set then env variable as you asked, but I don't see any debug logs. You can see the code I'm using here https://gist.github.com/dz-sourced/11654206078fcc159c54564de6758659

@ghost ghost added needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team and removed needs-author-feedback Workflow: More information is needed from author to address the issue. labels Apr 19, 2022
@dz-sourced
Copy link
Author

Actually, I have figured it out, need to set AZURE_SDK_GO_LOG_LEVEL=DEBUG first. @ArcturusZhang you can find the log here https://gist.github.com/dz-sourced/050d6b14a80c231a13840171dcfd7a6b I have redacted the subscription id.

@ghost
Copy link

ghost commented May 9, 2022

Thank you for your feedback. This has been routed to the support team for assistance.

@ArcturusZhang ArcturusZhang added bug This issue requires a change to an existing behavior in the product in order to be resolved. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels May 9, 2022
@ArcturusZhang
Copy link
Member

Hey @dz-sourced based on the information you shared, it is pretty sure that this issue was introduced by the swagger changes - therefore I added some labels to loop the correct persons to answer your questions

@SatishBoddu-MSFT SatishBoddu-MSFT added Service Attention Workflow: This issue is responsible by Azure service team. and removed CXP Attention labels May 17, 2022
@ghost
Copy link

ghost commented May 17, 2022

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @zjalexander.

Issue Details

Bug Report

Summary:
When trying to read back a DSC Configuration content using func (client DscConfigurationClient) GetContent an error happens in the version of the API 2020-01-13-preview/automation and later while trying to unmarshall DSC Configuration content from JSON. DSC Configurations are not valid JSON objects and thus unmarshalling fails. This issue was not happening in 2018-06-30-preview/automation and earlier, where DSC Configuration content was treated as a plain string value. I have confirmed the issue on the main branch.

Details:
In the 2020-01-13-preview/automation the actual failure seems to happen when trying to unmarshall DSC Configuration content from JSON:

The error that I get when trying to read back a sample DSC Configuration like Configuration test {} is:

automation.DscConfigurationClient#GetContent: Failure responding to request: StatusCode=200 -- Original Error: Error occurred unmarshalling JSON - Error = 'invalid character 'C' looking for beginning of value' JSON = 'Configuration test{}'

This was not happening in the previous version of the API 2018-06-30-preview/automation and the content value was just passed as is:

Finally, the root cause can be traced to the change in the Swagger API definitions. The latest version of the GetContent API defines the output as a string type, which I assume is tried to be de-serialized from JSON by default:
https://github.com/Azure/azure-rest-api-specs/blob/3034ada77d465b317cda51250a92454824cca06b/specification/automation/resource-manager/Microsoft.Automation/stable/2019-06-01/dscConfiguration.json#L309

In the earlier version of the API spec the output type is set to file:
https://github.com/Azure/azure-rest-api-specs/blob/3026119ab41bbce77275cfa3a1afbabf43af5aea/specification/automation/resource-manager/Microsoft.Automation/stable/2015-10-31/dscConfiguration.json#L309

I don't know whether this was intentional API spec change, or whether there is a need to provide further type modifiers in the Swagger API or whether it's an issue with the SDK codegen tool.

Here is a small script that demostrates the problem https://gist.github.com/dz-sourced/11654206078fcc159c54564de6758659
It creates a resource group, an automation account, a sample empty DSC Configuration and the tries to read it using older and new API versions. You should be able to see that read is succesful in 2018-06-30-preview/automation and fails in the 2020-01-13-preview/ version.

Author: dz-sourced
Assignees: ArcturusZhang
Labels:

bug, Automation, Service Attention, Mgmt, customer-reported, needs-team-attention

Milestone: -

@tombuildsstuff
Copy link
Contributor

ping @zjalexander

@zjalexander
Copy link
Member

I'm not the owner, but I've reached out to the responsible team to 1) get a response to this issue and 2) get my name taken off the owner list for this service.

@ralfwanninkhof
Copy link

@zjalexander Did you got any response? We would love to have this issue fixed, is there anything we can do to help?

@zjalexander
Copy link
Member

Thank you for the ping! I did get a response, but it needs some follow up on the correct owner's side. I have reached out to them again to get action taken.

@wuxu92
Copy link

wuxu92 commented Aug 26, 2022

@zjalexander hi is there any update on this issue? looking forward to having this issue fixed asap.

@zjalexander
Copy link
Member

I have forwarded your comment to the Automation service owners.

@wuxu92
Copy link

wuxu92 commented Sep 1, 2022

there is another similar break change in the runbook.GetCotent from 2018-06-30 to 2019-06-01. could you please forward this to the service team too? @zjalexander or should I raise another issue to track this

https://github.com/Azure/azure-rest-api-specs/blob/55458b642a7cdc8dd9b2928942d13c6f076399bc/specification/automation/resource-manager/Microsoft.Automation/stable/2018-06-30/runbook.json#L82

        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "type": "file"  <-- file
            }

==>

https://github.com/Azure/azure-rest-api-specs/blob/55458b642a7cdc8dd9b2928942d13c6f076399bc/specification/automation/resource-manager/Microsoft.Automation/stable/2019-06-01/runbook.json#L82

        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "type": "string"  <-- string
            }
          },

@tadelesh
Copy link
Member

tadelesh commented Sep 5, 2022

@dz-sourced We've depreacated services/preview/automation/mgmt. A replacement package is available github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/automation/armautomation. See Migration Guide for guidance on upgrading. And in the new version, the response is desterilized correctly.

@wuxu92
Copy link

wuxu92 commented Sep 5, 2022

but we can not migrate to the new SDK package in the short term(or even in the long term)

@lirenhe
Copy link
Member

lirenhe commented Sep 5, 2022

Please consider the migration and we could help if you have any specific asks. We already stopped regular release of old version of the SDKs.
cc @xboxeer

@tombuildsstuff
Copy link
Contributor

@wuxu92 since this is related to a Terraform issue, this method is now available in hashicorp/go-azure-sdk - so it'd be worth confirming if that works for this issue.

@wuxu92
Copy link

wuxu92 commented Sep 6, 2022

@tombuildsstuff hashicorp/go-azure-sdk generated the same code by autorest, so the line autorest.ByUnmarshallingJSON(&result.Model), will always cause an error.

@tadelesh
Copy link
Member

We have retired support for Azure SDK for Golang libraries which do not conform to our current Azure SDK guidelines (see announcement). Please migrate to the latest version according to the migration guide. If you could still repo this problem, please submit a support ticket in Azure directly.

@tadelesh tadelesh closed this as not planned Won't fix, can't repro, duplicate, stale Dec 26, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Mar 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Automation bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Previous Versions Work related to track1 and track1.5 SDKs Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests