Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test][Epic] Create unified python test onboarding guidelines and coalesce around them. #14294

Closed
4 tasks
KieranBrantnerMagee opened this issue Oct 6, 2020 · 3 comments
Assignees
Labels
Docs EngSys This issue is impacting the engineering system. MQ This issue is part of a "milestone of quality" initiative.
Milestone

Comments

@KieranBrantnerMagee
Copy link
Member

KieranBrantnerMagee commented Oct 6, 2020

Goals
This issue will track the work to do the following high level goals.

  1. Create a set of documentation for onboarding onto and writing proper tests for azure-sdk-for-python.
  2. The steps necessary to allow SDK owners to coalesce their libraries to the guidance decided in 1.
  3. Any work arising from 2 that occurs as needed. (e.g. framework instability, additional features, etc, preventing coalescing)

Bullets, subtasks etc should be free to be created and modified as needed, as the early steps of this process are highly discovery-oriented.

Phase 1 action items:

  • Create a home for the interim doc.
  • Define "from 0" test rampup steps in the new doc, noting gaps.
  • Define conceptual, semantic goals for "test acceptance." (runtime, flakiness, unit/integration split, etc).
  • When document is solid, share with SDK owners for feedback and to gather deltas-to-conformance.

Phase 2 action items:
TBD

Phase 3 action items:
TBD

@KieranBrantnerMagee KieranBrantnerMagee added Docs EngSys This issue is impacting the engineering system. MQ This issue is part of a "milestone of quality" initiative. labels Oct 6, 2020
@KieranBrantnerMagee KieranBrantnerMagee added this to the Backlog milestone Oct 6, 2020
@dave-read
Copy link

Does the target audience for this include external consumers of the SDK? We have migrations from AWS where the teams are familiar with the moto framework for unit testing and can't find something similar for Azure SDKs. There are some older issues pointing external users to how SDK teams do their testing, but nothing that I can find that clearly points to end-to-end unit testing guidelines/samples for external consumers of the Python SDK.

@seankane-msft
Copy link
Member

@dave-read These guidelines are a target for the library developers, although there may be helpful points for you and other external customers doing end-to-end testing. There is no Azure counterpart to moto, nor one on the roadmap afaik. For our own test infrastructure we use vcrpy to mock our HTTP transactions. vcrpy is not as comprehensive/specialized as moto, but it is a tool that can enable you to build out your end-to-end unit testing framework. Please let me know if this answer yields more questions than answers.

@lmazuel
Copy link
Member

lmazuel commented Dec 28, 2020

Closing, since doc is in PR now

@lmazuel lmazuel closed this as completed Dec 28, 2020
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this issue May 11, 2021
[Hub Generated] Review request for Microsoft.DevTestLab to add version stable/2018-09-15 (Azure#14294)

* Microsoft.DevTestLab S360 OpenApi Spec Corrections v2018-09-15

* Update isAuthenticationWithSshKey Mutability

* Correct dataDiskParameters mutability

* Fix Prettier Issues
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this issue May 11, 2021
[Hub Generated] Review request for Microsoft.DevTestLab to add version stable/2018-09-15 (Azure#14294)

* Microsoft.DevTestLab S360 OpenApi Spec Corrections v2018-09-15

* Update isAuthenticationWithSshKey Mutability

* Correct dataDiskParameters mutability

* Fix Prettier Issues
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Docs EngSys This issue is impacting the engineering system. MQ This issue is part of a "milestone of quality" initiative.
Projects
None yet
Development

No branches or pull requests

4 participants