Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(integration): support v2 volume test on test_provisioner.py #2308

Merged
merged 1 commit into from
Feb 20, 2025

Conversation

chriscchien
Copy link
Contributor

@chriscchien chriscchien commented Feb 6, 2025

Which issue(s) this PR fixes:

Issue #9760

What this PR does / why we need it:

support v2 volume test on test_provisioner.py

Special notes for your reviewer:

Test result

Additional documentation or context

N/A

Summary by CodeRabbit

  • Tests
    • Updated provisioning validations to dynamically adjust disk configuration checks based on the current engine version.
    • Enhanced storage provisioning tests by adding an engine-specific parameter and categorizing them for clearer test coverage.

@chriscchien chriscchien self-assigned this Feb 6, 2025
@chriscchien chriscchien requested a review from a team as a code owner February 6, 2025 07:25
Copy link

coderabbitai bot commented Feb 6, 2025

Walkthrough

The changes adjust test logic in two files. In the common module, the assertion in the node_default_tags fixture now conditionally checks the number of node disks based on the DATA_ENGINE variable. In the provisioner tests, a new import of DATA_ENGINE is used to update the storage class parameters with a new dataEngine key, and several test functions are now marked with the @pytest.mark.v2_volume_test decorator for version-specific grouping.

Changes

File(s) Change Summary
manager/integration/tests/common.py Updated the node_default_tags fixture to conditionally assert the number of disks based on the DATA_ENGINE variable (2 disks if "v2", otherwise 1 disk).
manager/integration/tests/test_provisioner.py Added import of DATA_ENGINE; modified test_provisioner_params to include the dataEngine parameter in storage class; added @pytest.mark.v2_volume_test to multiple functions.

Sequence Diagram(s)

sequenceDiagram
    participant T as Test Function
    participant F as node_default_tags Fixture
    participant N as Node
    T->>F: Invoke fixture
    F->>F: Check DATA_ENGINE
    alt DATA_ENGINE=="v2"
        F->>N: Assert 2 disks
    else
        F->>N: Assert 1 disk
    end
Loading
sequenceDiagram
    participant TP as test_provisioner_params
    participant C as Common Module (DATA_ENGINE)
    participant SC as Storage Class
    TP->>C: Import DATA_ENGINE
    C-->>TP: Provide DATA_ENGINE value
    TP->>SC: Add 'dataEngine' to parameters
Loading

Possibly related PRs

Suggested reviewers

  • yangchiu

Poem

I'm a bunny coder, quick on my feet,
Hopping through tests with a rhythmic beat.
One disk or two, the logic now sings,
In burrows of code, innovation springs.
Cheers to changes that brighten the night,
With ASCII carrots and code shining bright!
🐰🥕


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 95c2516 and 1358f84.

📒 Files selected for processing (2)
  • manager/integration/tests/common.py (1 hunks)
  • manager/integration/tests/test_provisioner.py (7 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • manager/integration/tests/common.py
  • manager/integration/tests/test_provisioner.py
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build images
  • GitHub Check: Summary

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
manager/integration/tests/common.py (2)

332-336: Consider using constants for better maintainability.

The data engine selection logic is clear, but consider defining constants for the engine types to avoid magic strings.

+DATA_ENGINE_V1 = "v1"
+DATA_ENGINE_V2 = "v2"

enable_v2 = os.environ.get('RUN_V2_TEST')
if enable_v2 == "true":
-    DATA_ENGINE = "v2"
+    DATA_ENGINE = DATA_ENGINE_V2
else:
-    DATA_ENGINE = "v1"
+    DATA_ENGINE = DATA_ENGINE_V1
🧰 Tools
🪛 Ruff (0.8.2)

333-336: Use ternary operator DATA_ENGINE = "v2" if enable_v2 == "true" else "v1" instead of if-else-block

Replace if-else-block with DATA_ENGINE = "v2" if enable_v2 == "true" else "v1"

(SIM108)


1687-1690: Enhance assertion error message for disk count check.

The disk count assertion is correct but could benefit from a more descriptive error message to help with debugging.

-        if DATA_ENGINE == "v2":
-            assert len(node.disks) == 2
-        else:
-            assert len(node.disks) == 1
+        expected_disk_count = 2 if DATA_ENGINE == "v2" else 1
+        assert len(node.disks) == expected_disk_count, \
+            f"Expected {expected_disk_count} disk(s) for {DATA_ENGINE} engine, " \
+            f"but got {len(node.disks)} disk(s)"
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4f6efab and 53ce038.

📒 Files selected for processing (2)
  • manager/integration/tests/common.py (1 hunks)
  • manager/integration/tests/test_provisioner.py (7 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build images
  • GitHub Check: Summary
🔇 Additional comments (3)
manager/integration/tests/test_provisioner.py (3)

12-12: LGTM!

The import of DATA_ENGINE from the common module is correctly placed and used in the storage class parameters.


31-31: LGTM! Test categorization is correctly implemented.

The @pytest.mark.v2_volume_test decorator is consistently applied to all relevant test functions, aligning with the PR objective of supporting v2 volume testing.

Also applies to: 87-87, 132-132, 174-174, 226-226


112-116: Verify the impact of the new dataEngine parameter.

The addition of dataEngine parameter to the storage class is consistent with v2 volume support. However, let's verify its usage and potential impact.

Run the following script to check how the dataEngine parameter is used across the codebase:

✅ Verification successful

🏁 Script executed:

#!/bin/bash
# Description: Check usage of dataEngine parameter in storage class definitions and related code

# Search for dataEngine parameter usage
rg -A 5 'dataEngine'

# Search for storage class parameter handling
ast-grep --pattern $'storage_class.*parameters.*=.*{
  $$$
}'

Length of output: 65699


The new dataEngine parameter is consistently integrated across the codebase.

  • The shell script search reveals that the dataEngine parameter is passed from the DATA_ENGINE variable in tests such as test_provisioner.py, test_csi.py, test_basic.py, and many others.
  • It is also used in the storage class creation functions in modules like e2e/libs/storageclass/storageclass.py and related volume management implementations.
  • This wide usage indicates that the parameter is well-integrated and its impact has been accounted for across volume provisioning, stateful set handling, and other related functionalities.

Comment on lines 1687 to 1693
if DATA_ENGINE == "v2":
assert len(node.disks) == 2
else:
assert len(node.disks) == 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you leave a comment explaining why the len(node.disks) is 2 while enabling v2 data engine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, thank you.

@@ -1684,7 +1684,12 @@ def node_default_tags():

tag_mappings = {}
for tags, node in zip(DEFAULT_TAGS, nodes):
assert len(node.disks) == 1
if DATA_ENGINE == "v2":
# if v2 data engine is enabled, both a file system disk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on the purpose? For v2 backing image, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have updated the comment to below, because a v2 volume use block disk but a v2 backing image can not function without file system disk so the two types of disks need to coexist if we want to test v2 volume with v2 backing image.

            # if the v2 data engine is enabled, both a file system disk
            # and a block disk will coexist. This is because a v2 backing image
            # requires a file system disk to function.

ref: longhorn/longhorn 9760

Signed-off-by: Chris Chien <chris.chien@suse.com>
@derekbit derekbit force-pushed the v2_provisioner_test branch from 95c2516 to 1358f84 Compare February 20, 2025 03:22
Copy link
Member

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekbit derekbit merged commit cb80e91 into longhorn:master Feb 20, 2025
9 checks passed
@roger-ryao
Copy link
Contributor

@mergify backport v1.8.x

Copy link

mergify bot commented Feb 25, 2025

backport v1.8.x

✅ Backports have been created

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants