Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix parsing of storage constraints #1053

Merged
merged 3 commits into from
May 31, 2024

Conversation

luissimas
Copy link
Contributor

Description

Fixes: #1052. Parses the storage constraints when deploying applications. Before this change the storage constraints where not parsing, resulting in an error when deploying bundles that contained applications with storage definitions.

QA Steps

The following python script can be used to verify both the bug in the current version as well as the fix implemented:

import asyncio
from juju.model import Model

bundle_file = "./bundle.yaml"

bundle = """
name: sample-bundle

series: jammy

machines:
  "0":
    constraints: instance-type=type1

applications:
  swift-storage:
    charm: swift-storage
    channel: yoga/stable
    num_units: 1
    to:
      - "0"
    storage:
      block-devices: cinder,1,5G
"""

async def main():
    with open(bundle_file, "w") as f:
        f.write(bundle)

    model = Model()
    await model.connect()
    await model.deploy(bundle_file)

asyncio.run(main())

All CI tests need to pass.

Notes & Discussion

I wasn't able to add a regression test for this fix since the juju-qa-test charm does not support the addition of storage devices. With that said, I'm open to suggestions on how to test this behavior to make sure it's correct and that it stays that way on future changes.

@jujubot
Copy link
Contributor

jujubot commented May 17, 2024

Thanks for opening a pull request! Please follow the instructions here to ensure your pull request is ready for review. Then, a maintainer will review your patch.

@hpidcock @anvial

1 similar comment
@jujubot
Copy link
Contributor

jujubot commented May 17, 2024

Thanks for opening a pull request! Please follow the instructions here to ensure your pull request is ready for review. Then, a maintainer will review your patch.

@hpidcock @anvial

Copy link

@kwmonroe kwmonroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this bit me as well on juju==3.4.0.0. confirmed this PR fixes this on that release.

Copy link
Contributor

@cderici cderici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change @luissimas! Code looks good 👍 Let's add the small bundle in your QA steps as an integration test, please use the tempfile so the file doesn't persist (e.g. with tempfile.NamedTemporaryFile() as temp_file:), you can just write, flush, then temp_file.seek(0), before running the deploy inside the context.

Alternatively, I'd be ok with adding this among our test bundles, if that's gonna be easier.

@cderici
Copy link
Contributor

cderici commented May 24, 2024

@jack-w-shaw can you add the following cases in your queue for fixing CI:

In this run for this PR:

(note to @luissimas, none of the CI failures seems to be related to your changes, so we can land this as soon as you add the regression test, thanks again for the change ❤️)

@luissimas
Copy link
Contributor Author

@cderici I've added a integration test in 132f96c. I had to make some changes to the bundle to be able to deploy it on an LXD cloud, which I assume is the expected environment for the integration tests. With that said, I was able to reproduce both the bug and the fix using this bundle.

luissimas and others added 3 commits May 31, 2024 11:41
Parses the storage constraints when deploying applications. Before this
change the storage constraints where not parsing, resulting in an error
when deploying bundles that contained applications with storage
definitions.
@cderici
Copy link
Contributor

cderici commented May 31, 2024

/build

@cderici
Copy link
Contributor

cderici commented May 31, 2024

/merge

@jujubot jujubot merged commit 53cd33d into juju:master May 31, 2024
7 of 9 checks passed
jujubot added a commit that referenced this pull request Jul 11, 2024
#1071

## What's Changed
* fix parsing of storage constraints by @luissimas in #1053
* Add setuptools to tox.ini by @Aflynn50 in #1058
* fix(refresh): bug with revisions by @jack-w-shaw in #1067
* feat: conventional commits static analysis by @SimonRichardson in #1068
* fix(series): add noble support by @jack-w-shaw in #1063
* fix zones constrains list parsing by @luissimas in #1054
* fix(model): fix wrong instanciation of list-secrets facade by @gboutry in #1065
* fix(makefile): run .tox before lint in makefile target by @cderici in #1069
jujubot added a commit that referenced this pull request Nov 26, 2024
…fy-handling-of-storage-constraints-in-deploy

#1213

#### Description

This PR unifies storage constraint parsing into a single method (`juju.constraints.parse_storage_constraints`), which is called in a single place (`juju.model.Model._deploy`), allowing both bundle and model deployments to specify storage constraints using either the [juju storage constraint directive format](https://juju.is/docs/juju/storage-constraint) (e.g. `{'label': 'ebs,100G'}`) or pre-parsed dictionaries (e.g. `{'label': {'count': 1, 'pool': 'ebs', 'size': 102400}`).

#### QA Steps

Unit tests have been updated to reflect the new parsing location. Integration tests have been added to verify that model deployment can request storage using either format. The existing bundle integration tests should continue to pass.


#### Notes & Discussion

This PR resolves the issues with storage constraint parsing identified in:
- #1052
- #1075

#1052 was initially addressed in #1053, which was included in the [3.5.2.0](https://github.com/juju/python-libjuju/releases/tag/3.5.2.0) release. This allowed bundle deployments (using `juju.bundle.AddApplicationChange.run`) to correctly handle storage constraints specified as strings in the [juju storage constraint directive format](https://juju.is/docs/juju/storage-constraint).

Unfortunately, this erroneously required that model deployments (using `juju.model.Model.deploy`) also use this string format, instead of the parsed dictionary format that was previously accepted. This was noticed in #1075 and initially fixed in #1105, which was merged into `main` but never released. This fix moved parsing of storage constraint strings to bundle deployment exclusively.

Due to the interim period in which `3.5.2` has (incorrectly) required model deployments to use the string format, I think the best fix at this point is to allow both bundle deployments and model deployments to use either format, and parse them into the parsed dictionary format in a single place in `juju.model.Model._deploy` (the private method called by both bundle and model deployments).

After merging, let's look at getting these changes out in a `3.5.2.2` bugfix release.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect parsing of storage constraints
4 participants