Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate test recording, for mgmt-plane #34243

Closed
weidongxu-microsoft opened this issue Mar 29, 2023 · 10 comments
Closed

Migrate test recording, for mgmt-plane #34243

weidongxu-microsoft opened this issue Mar 29, 2023 · 10 comments
Assignees
Labels
Mgmt This issue is related to a management-plane library. Mgmt-EngSys Engineering System (Management Plane Specific) test-enhancement

Comments

@weidongxu-microsoft
Copy link
Member

weidongxu-microsoft commented Mar 29, 2023

link #33342

We'd need to check if these work

  • normal recording
  • recording with random name/guid (i.e. they are saved to json, and loaded at playback)
  • redact of secret (need to know whether we can still run script to redact it, before e.g. upload to somewhere we no longer control; or we can use TestBase to redact); if now request body is also recorded, we may need to add more redact rules

We may try

  1. resources (random name)
  2. storage (secret)
  3. appservice or ACI/ACR (different kind of secret)

We also need to see if we can still batch modify api-version in recorded data, after phase2
We need this e.g. after upgrade resources RP or network RP
https://github.com/Azure/azure-sdk-assets/blob/main/README.md
e.g. upgrade Network would modify more tests #34243 (comment)


Another question on the "azure-sdk-assets" repo is: how it handles different recording from different branch (e.g. one branch may release a preview api-version, but main on stable)

@weidongxu-microsoft weidongxu-microsoft added Mgmt This issue is related to a management-plane library. test-enhancement Mgmt-EngSys Engineering System (Management Plane Specific) labels Mar 29, 2023
@ghost
Copy link

ghost commented Mar 29, 2023

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @weidongxu-microsoft, @XiaofeiCao

@weidongxu-microsoft
Copy link
Member Author

weidongxu-microsoft commented Mar 29, 2023

@weidongxu-microsoft
Copy link
Member Author

weidongxu-microsoft commented Mar 29, 2023

Most recording would be under "sdk/resourcemanager".

Very few other places, as mgmt recommend live tests for simpler RP
One of them:
https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/confidentialledger/azure-resourcemanager-confidentialledger/src/test/resources/session-records
<-- this can be used as a pilot migration, however it is not representative for complex cases

@weidongxu-microsoft
Copy link
Member Author

weidongxu-microsoft commented Mar 31, 2023

You can contact Sameeksha, if more question.

@haolingdong-msft
Copy link
Member

Hey @samvaity, may I know do we have timeline of migrating to the new test recording framework?

@haolingdong-msft
Copy link
Member

Reference links:
issue on test proxy: #33342
example pr: #33902

@haolingdong-msft
Copy link
Member

Met issue on connecting to local test proxy server. Question here: #33342 (comment)

@haolingdong-msft
Copy link
Member

Started working on azure-resourcemanager-resources

@haolingdong-msft
Copy link
Member

haolingdong-msft commented Jul 13, 2023

@haolingdong-msft
Copy link
Member

haolingdong-msft commented Jul 13, 2023

Todos:

  1. try remove TextReplacementPolicy
  2. retry-after = 0, make sure long lro can run successfully, e.g. sql server
  3. storage, response body secret sanitize
    solution here: https://github.com/samvaity/azure-sdk-for-java/tree/add-docs-test-proxy/sdk/core/azure-core-test#sanitize-secrets
    image
  4. credcheck before push to asset repo? (there is already DefaultSanitizer, not sure if it includes all we want)
  5. migration for other RPs.

@github-actions github-actions bot locked and limited conversation to collaborators Oct 29, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Mgmt This issue is related to a management-plane library. Mgmt-EngSys Engineering System (Management Plane Specific) test-enhancement
Projects
None yet
Development

No branches or pull requests

2 participants