Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11311. Added Compatibility test for HSync #7400

Merged
merged 2 commits into from
Nov 7, 2024
Merged

HDDS-11311. Added Compatibility test for HSync #7400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
070e322
HDDS-11311. Added compatibility test for HSync
hemantk-12 Nov 6, 2024
447ad58
Changed HSync base version to 2.0.0 in test setup
hemantk-12 Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions hadoop-ozone/dist/src/main/smoketest/compatibility/read.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,9 @@ FSO Bucket Can Be Read
Execute ozone fs -get ofs://om/vol1/fso-bucket-${SUFFIX}/dir/subdir/file ${TEMP_DIR}/
Execute diff -q ${TESTFILE} ${TEMP_DIR}/file
[teardown] Execute rm -f ${TEMP_DIR}/file

HSync Lease Recover Can Be Used
Pass Execution If '${DATA_VERSION}' < '${HSYNC_VERSION}' Skipped the test case
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like write and read test cases for HSync are independent. This read test case uses the file created by the FSO test case as input. Therefore here we do not need skip based on ${HSYNC_VERSION}, but ${FSO_VERSION}.

Suggested change
Pass Execution If '${DATA_VERSION}' < '${HSYNC_VERSION}' Skipped the test case
Pass Execution If '${DATA_VERSION}' < '${FSO_VERSION}' Skipped the test case

Pass Execution If '${CLIENT_VERSION}' < '${HSYNC_VERSION}' Client does not support HSYNC
Pass Execution If '${CLUSTER_VERSION}' < '${HSYNC_VERSION}' Cluster does not support HSYNC
Execute ozone debug recover --path=ofs://om/vol1/fso-bucket-${SUFFIX}/dir/subdir/file
1 change: 1 addition & 0 deletions hadoop-ozone/dist/src/main/smoketest/compatibility/setup.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Resource ../ozone-lib/shell.robot
*** Variables ***
${SUFFIX} ${EMPTY}
${FSO_VERSION} 1.3.0
${HSYNC_VERSION} 2.0.0


*** Keywords ***
Expand Down
14 changes: 14 additions & 0 deletions hadoop-ozone/dist/src/main/smoketest/compatibility/write.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,15 @@
Documentation Write Compatibility
Resource ../ozone-lib/shell.robot
Resource setup.robot
Resource ../lib/fs.robot
Resource ../ozone-lib/freon.robot
Test Timeout 5 minutes
Suite Setup Create Local Test File

*** Variables ***
${SUFFIX} ${EMPTY}
${VOL} comp-hsync-volume
${BUCK} comp-hsync-bucket
Comment on lines +27 to +28
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
${VOL} comp-hsync-volume
${BUCK} comp-hsync-bucket



*** Test Cases ***
Expand All @@ -40,3 +44,13 @@ FSO Bucket Can Be Created and Used
Execute ozone sh bucket create --layout FILE_SYSTEM_OPTIMIZED /vol1/fso-bucket-${SUFFIX}
Execute ozone fs -mkdir -p ofs://om/vol1/fso-bucket-${SUFFIX}/dir/subdir
Execute ozone fs -put ${TESTFILE} ofs://om/vol1/fso-bucket-${SUFFIX}/dir/subdir/file

HSync Can Be Used To Create Keys
Pass Execution If '${CLIENT_VERSION}' < '${HSYNC_VERSION}' Client does not support HSYNC
Pass Execution If '${CLUSTER_VERSION}' < '${HSYNC_VERSION}' Cluster does not support HSYNC
Execute ozone sh volume create /${VOL}
Execute ozone sh bucket create /${VOL}/${BUCK}
${o3fspath} = Format FS URL o3fs ${VOL} ${BUCK}
Freon DFSG sync=HSYNC path=${o3fspath}
${pfspath} = Format FS URL ofs ${VOL} ${BUCK}
Freon DFSG sync=HSYNC path=${pfspath}
Comment on lines +51 to +56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use vol1 and bucket1 here.

Rationale:

  • write.robot is executed against the same cluster by multiple clients (with various versions)
  • ${VOL} and ${BUCK} always have the same values
  • creation will fail as soon as we have two versions that both support hsync
  • new volume and bucket are not necessary for this specific test case
  • vol1 and bucket1 are pre-created:

    ozone/hadoop-ozone/dist/src/main/compose/xcompat/test.sh

    Line 91 in 0415c0b

    execute_command_in_container scm ozone freon ockg -n1 -t1 -p warmup

For unique items to be created per client version, please use ${SUFFIX}, which is set by test.sh. It is up to each test case how they use that suffix. E.g. a test that creates buckets would append it to the bucket name. That's why there are no variables for volume/bucket name.

Suggested change
Execute ozone sh volume create /${VOL}
Execute ozone sh bucket create /${VOL}/${BUCK}
${o3fspath} = Format FS URL o3fs ${VOL} ${BUCK}
Freon DFSG sync=HSYNC path=${o3fspath}
${pfspath} = Format FS URL ofs ${VOL} ${BUCK}
Freon DFSG sync=HSYNC path=${pfspath}
${o3fspath} = Format FS URL o3fs vol1 bucket1
Freon DFSG sync=HSYNC path=${o3fspath}
${ofspath} = Format FS URL ofs vol1 bucket1
Freon DFSG sync=HSYNC path=${ofspath}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I used the vol1 and bucket1 hemantk-12@7e9eed7 but Dir Can Be Listed Using Shell test was failing in the workflow run. So rather than fixing Dir Can Be Listed Using Shell test, I created new volume and bucket for hsync test case.
I guess I can change Dir Can Be Listed Using Shell test to list only dir-${SUFFIX} dir rather than all the dirs in the bucket. I'll test it out and submit a follow-up PR.