Skip to content

Upgrades the openapi python client to v6.2.1 #1887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

spacether
Copy link

@spacether spacether commented Sep 12, 2022

What type of PR is this?

/kind feature
/kind api-change

What this PR does / why we need it:

This PR switches to using the python generator from openapi-generator v6.2.1
Please do not review this PR into I move it out of WIP/Draft State
This PR updates the python client which now has:

  • type hints everywhere
  • more robust composition(oneOf/anyOf/allOf/not) support
  • payload someProp case used rather than python some_prop used to ensure compliance when imposing json schema keywords like properties/additionalProperties/composition. This allows a payload to contain the separate keys someProp and some_prop
  • supports newer openapi features like handling of multiple request body content types and response body content types

I am seeking feedback from your project on what you think about the python client in the latest openapi-generator and if you prefer to use it going forward.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

I am seeking feedback from your project on how you feel about using the code generated from the 6.2.1 python generator.

Does this PR introduce a user-facing change?

Action required. This is a breaking change as the new python client is structured differently.
Python min version changed from 3.6 to 3.7

### Changes
1. This generator uses spec case for all (object) property names and parameter names.
    - So if the spec has a property name like camelCase, it will use camelCase rather than camel_case
    - So you will need to update how you input and read properties to use spec case
2. Endpoint parameters are stored in dictionaries to prevent collisions (explanation below)
    - So you will need to update all of your endpoint calls that have query/path/header/cookie params 
      to pass that parameter data in query_params, header_params, path_params, cookie_params 
      named argument inputs with values of type dict
3. Endpoint responses now include the original response, the deserialized response body, and (todo)the deserialized headers
    - So you will need to update your code to use response.body to access deserialized data
4. All validated data is instantiated in an instance that subclasses all validated Schema classes and Decimal/str/list/tuple/frozendict/NoneClass/BoolClass/bytes/io.FileIO
    - This means that you can use isinstance to check if a payload validated against a schema class
    - This means that no data will be of type None/True/False
        - ingested None will subclass NoneClass
        - ingested True will subclass BoolClass
        - ingested False will subclass BoolClass
        - So if you need to check is True/False/None, instead use instance.is_true_oapg()/.is_false_oapg()/.is_none_oapg()
5. All validated class instances are immutable except for ones based on io.File
    - This is because if properties were changed after validation, that validation would no longer apply
    - So no changing values or property values after a class has been instantiated
6. String + Number types with formats
    - String type data is stored as a string and if you need to access types based on its format like date,
    date-time, uuid, number etc then you will need to use accessor functions on the instance
    - type string + format: See .as_date_oapg, .as_datetime_oapg, .as_decimal_oapg, .as_uuid_oapg
    - type number + format: See .as_float_oapg, .as_int_oapg
    - this was done because openapi/json-schema defines constraints. string data may be type string with no format
    keyword in one schema, and include a format constraint in another schema
    - So if you need to access a string format based type, use as_date_oapg/as_datetime_oapg/as_decimal_oapg/as_uuid_oapg
    - So if you need to access a number format based type, use as_int_oapg/as_float_oapg
7. Property access on AnyType(type unset) or object(dict) schemas
    - Only required keys with valid python names are properties like .someProp and have type hints
    - All optional keys may not exist, so properties are not defined for them
    - One can access optional values with dict_instance['optionalProp'] and KeyError will be raised if it does not exist
    - Use get_item_oapg if you need a way to always get a value whether or not the key exists
        - If the key does not exist, schemas.unset is returned from calling dict_instance.get_item_oapg('optionalProp')
        - All required and optional keys have type hints for this method, and @typing.overload is used
        - A type hint is also generated for additionalProperties accessed using this method
    - So you will need to update you code to use some_instance['optionalProp'] to access optional property
    and additionalProperty values
8. The location of the api classes has changed
    - Api classes are located in your_package.apis.tags.some_api
    - This change was made to eliminate redundant code generation
    - Legacy generators generated the same endpoint twice if it had > 1 tag on it
    - This generator defines an endpoint in one class, then inherits that class to generate
      apis by tags and by paths
    - This change reduces code and allows quicker run time if you use the path apis
        - path apis are at your_package.apis.paths.some_path
    - Those apis will only load their needed models, which is less to load than all of the resources needed in a tag api
    - So you will need to update your import paths to the api classes

This is an excerpt from the included migration guide which shows what needs to be updated: https://github.com/spacether/kube_python/tree/feat_switches_to_python_exp_client/kubernetes#migration-from-other-generators-like-python-and-python-legacy

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot
Copy link
Contributor

@spacether: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Sep 12, 2022
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Sep 12, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Sep 12, 2022
@k8s-ci-robot
Copy link
Contributor

Welcome @spacether!

It looks like this is your first PR to kubernetes-client/python 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-client/python has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@spacether spacether marked this pull request as draft September 12, 2022 18:46
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 12, 2022
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 12, 2022
@spacether spacether changed the title Switches to openapi-generator 6.1.0 and uses python-experimental generator [WIP] Switches to openapi-generator 6.1.0 and uses python-experimental generator Sep 12, 2022
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Sep 12, 2022
@spacether spacether changed the title [WIP] Switches to openapi-generator 6.1.0 and uses python-experimental generator [WIP] Switches to using python-experimental generator and openapi-generator v6.1.0 Sep 12, 2022
@spacether spacether changed the title [WIP] Switches to using python-experimental generator and openapi-generator v6.1.0 [WIP] Switches to using python-experimental generator for client generation Sep 13, 2022
@yliaog
Copy link
Contributor

yliaog commented Sep 26, 2022

i think we should not be using the experimental generator. please file a PR to use the official generator release.

@spacether spacether changed the title [WIP] Switches to using python-experimental generator for client generation [WIP] Switches to using python generator for client generation Sep 26, 2022
@spacether spacether changed the title [WIP] Switches to using python generator for client generation [WIP] Upgrades the openapi python client to v6.2.0 Sep 26, 2022
@spacether spacether marked this pull request as ready for review September 26, 2022 19:52
@spacether
Copy link
Author

spacether commented Sep 26, 2022

@yliaog I just updated this PR to use the official 6.2.0 python generator

@spacether spacether changed the title [WIP] Upgrades the openapi python client to v6.2.0 Upgrades the openapi python client to v6.2.0 Sep 26, 2022
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 26, 2022
@spacether
Copy link
Author

@yliaog can you or others review this PR? Do I need to do anything else?

@yliaog
Copy link
Contributor

yliaog commented Oct 11, 2022

could you please squash the commits? I'll take a look afterwards. Thanks.

@spacether spacether force-pushed the feat_switches_to_python_exp_client branch from c4e18f9 to 6a1414f Compare October 11, 2022 16:14
@spacether
Copy link
Author

@yliaog just squashed them into one commit now

@yliaog
Copy link
Contributor

yliaog commented Oct 11, 2022

https://github.com/spacether/kube_python/tree/feat_switches_to_python_exp_client/kubernetes#migration-from-other-generators-like-python-and-python-legacy

so this PR introduces breaking change? for breaking change, we need to solicit community feedback.

also it's better to split the PR into at least two commits, one commit for generated files, the other commit is for the manual edited files.

@spacether
Copy link
Author

https://github.com/spacether/kube_python/tree/feat_switches_to_python_exp_client/kubernetes#migration-from-other-generators-like-python-and-python-legacy

so this PR introduces breaking change? for breaking change, we need to solicit community feedback.

also it's better to split the PR into at least two commits, one commit for generated files, the other commit is for the manual edited files.

Yes the updated client has breaking changes versus the old ones. I have not manually edited files.
Which files need to be manually edited/where in this code base uses the client?
How do we solicit community feedback?

@yliaog
Copy link
Contributor

yliaog commented Oct 11, 2022

https://github.com/kubernetes-client/gen/tree/master/openapi needs manual update?

please send email to:
kubernetes-dev@googlegroups.com

file an issue similar to: #974

@spacether
Copy link
Author

My plan is to wait until 6.2.1 is released to get community feedback

@yliaog
Copy link
Contributor

yliaog commented Oct 18, 2022

sounds good. thanks.

@spacether spacether changed the title Upgrades the openapi python client to v6.2.0 Upgrades the openapi python client to v6.2.1 Nov 1, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: spacether
Once this PR has been reviewed and has the lgtm label, please assign yliaog for approval by writing /assign @yliaog in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

… gen_kube_client so the different templating engine can be used
@spacether spacether force-pushed the feat_switches_to_python_exp_client branch from 49bd4f1 to aae3f84 Compare November 1, 2022 19:58
@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 1, 2022
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 1, 2022
@spacether
Copy link
Author

Just posted an email in https://groups.google.com/a/kubernetes.io/g/dev which looks like the newest google group to use

@tomplus
Copy link
Member

tomplus commented Nov 2, 2022

@spacether Could you update examples/ and e2e tests?

@spacether
Copy link
Author

@tomplus can you help me with my tox requirement installation?
I have added the needed dependencies to requirements.txt, but they are not being picked up when installing with tox. What am I missing?
Note: pip install -e . correctly installs the requirements

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 30, 2022
@k8s-ci-robot
Copy link
Contributor

@spacether: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@atti92
Copy link

atti92 commented Feb 8, 2023

Any update on this? openapi-generator v6.3.0 is released already.

@spacether
Copy link
Author

I need help getting tox working. Until someone helps this is stalled. Also openapi-generator has announced that the current python generator will be replaced with the python-nextgen generator in v 7.0.0

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 9, 2023
@spacether spacether closed this May 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants