-
Notifications
You must be signed in to change notification settings - Fork 10
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[#334] devide edi tests to separate suites
- Loading branch information
Showing
10 changed files
with
466 additions
and
338 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| *** Settings *** | ||
| Documentation Legion's EDI check for one model undeploy command | ||
| Test Timeout 5 minutes | ||
| Resource resources/keywords.robot | ||
| Resource resources/variables.robot | ||
| Variables load_variables_from_profiles.py ${PATH_TO_PROFILES_DIR} | ||
| Library legion_test.robot.Utils | ||
| Library Collections | ||
| Library Process | ||
| Suite Setup Run Keywords | ||
| ... Check EDI availability in all enclaves AND | ||
| ... Choose cluster context ${CLUSTER_NAME} | ||
| Test Setup Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} | ||
| Test Teardown Run EDI undeploy model without version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} | ||
|
|
||
| *** Test Cases *** | ||
| Check EDI undeploy procedure | ||
| [Documentation] Try to undeploy dummy valid model through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI undeploy without version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| ${resp}= Run EDI inspect ${MODEL_TEST_ENCLAVE} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Should not contain ${resp.stdout} ${model_id} | ||
|
|
||
| Check EDI undeploy procedure by invalid model id | ||
| [Setup] NONE | ||
| [Documentation] Try to undeploy model through EDI console by invalid id | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI undeploy without version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1}test | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Should contain ${resp.stdout} Cannot find any deployment - ignoring | ||
| [Teardown] NONE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| *** Settings *** | ||
| Documentation Legion's EDI check for one model inspect command | ||
| Test Timeout 5 minutes | ||
| Resource resources/keywords.robot | ||
| Resource resources/variables.robot | ||
| Variables load_variables_from_profiles.py ${PATH_TO_PROFILES_DIR} | ||
| Library legion_test.robot.Utils | ||
| Library Collections | ||
| Library Process | ||
| Suite Setup Run Keywords | ||
| ... Check EDI availability in all enclaves AND | ||
| ... Choose cluster context ${CLUSTER_NAME} | ||
| Test Setup Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} | ||
| Test Teardown Run EDI undeploy model without version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} | ||
|
|
||
| *** Test Cases *** | ||
| Check EDI enclave inspect procedure | ||
| [Documentation] Try to inspect enclave through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI inspect ${MODEL_TEST_ENCLAVE} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Should contain ${resp.stdout} ${TEST_MODEL_ID} | ||
|
|
||
| Check EDI invalid enclave name inspect procedure | ||
| [Documentation] Try to inspect invalid enclave through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI inspect ${MODEL_TEST_ENCLAVE}test | ||
| Should Be Equal As Integers ${resp.rc} 2 | ||
| Should contain ${resp.stderr} ERROR - Failed to connect | ||
|
|
||
| Check EDI enclave inspect procedure without deployed model | ||
| [Setup] NONE | ||
| [Documentation] Try inspect through EDI console on empty enclave | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI inspect ${MODEL_TEST_ENCLAVE} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Should Not Contain ${resp.stdout} ${model_id} | ||
| [Teardown] NONE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| *** Settings *** | ||
| Documentation Legion's EDI check for one model scale command | ||
| Test Timeout 5 minutes | ||
| Resource resources/keywords.robot | ||
| Resource resources/variables.robot | ||
| Variables load_variables_from_profiles.py ${PATH_TO_PROFILES_DIR} | ||
| Library legion_test.robot.Utils | ||
| Library Collections | ||
| Library Process | ||
| Suite Setup Run Keywords | ||
| ... Check EDI availability in all enclaves AND | ||
| ... Choose cluster context ${CLUSTER_NAME} | ||
| Test Setup Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} | ||
| Test Teardown Run EDI undeploy model without version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} | ||
|
|
||
| *** Test Cases *** | ||
| Check EDI scale up procedure | ||
| [Documentation] Try to scale up model through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI scale ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Sleep 10 # because no way to control explicitly scaling the model inside | ||
| # TODO remove sleep | ||
| ${resp}= Run EDI inspect with parse ${MODEL_TEST_ENCLAVE} | ||
| ${model}= Find model information in edi ${resp} ${TEST_MODEL_ID} | ||
| Log ${model} | ||
| Verify model info from edi ${model} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 2 | ||
|
|
||
| Check EDI scale down procedure | ||
| [Documentation] Try to scale down model through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI scale ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Sleep 10 | ||
| # TODO remove sleep | ||
| ${resp}= Run EDI inspect with parse ${MODEL_TEST_ENCLAVE} | ||
| ${model}= Find model information in edi ${resp} ${TEST_MODEL_ID} | ||
| Log ${model} | ||
| Verify model info from edi ${model} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 2 | ||
|
|
||
| ${resp}= Run EDI scale ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 1 | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| Sleep 10 | ||
| # TODO remove sleep | ||
| ${resp}= Run EDI inspect with parse ${MODEL_TEST_ENCLAVE} | ||
| ${model}= Find model information in edi ${resp} ${TEST_MODEL_ID} | ||
| Log ${model} | ||
| Verify model info from edi ${model} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 1 | ||
|
|
||
| Check EDI scale to 0 procedure | ||
| [Documentation] Try to scale to 0 model through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI scale ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 0 | ||
| Should Be Equal As Integers ${resp.rc} 2 | ||
| Should contain ${resp.stderr} Invalid scale parameter: should be greater then 0 | ||
|
|
||
| Check EDI invalid model id scale up procedure | ||
| [Documentation] Try to scale up dummy model with invalid name through EDI console | ||
| [Tags] edi cli enclave one_version | ||
| ${resp}= Run EDI scale ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}test 2 | ||
| Should Be Equal As Integers ${resp.rc} 2 | ||
| Should contain ${resp.stderr} No one model can be found |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| *** Settings *** | ||
| Documentation Legion's EDI check for one model_id but different model versions deploy command | ||
| Test Timeout 5 minutes | ||
| Resource resources/keywords.robot | ||
| Resource resources/variables.robot | ||
| Variables load_variables_from_profiles.py ${PATH_TO_PROFILES_DIR} | ||
| Library legion_test.robot.Utils | ||
| Library Collections | ||
| Library Process | ||
| Suite Setup Run Keywords | ||
| ... Check EDI availability in all enclaves AND | ||
| ... Choose cluster context ${CLUSTER_NAME} | ||
| Test Setup Run Keywords | ||
| ... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} AND | ||
| ... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} | ||
| Test Teardown Run Keywords | ||
| ... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} ${TEST_MODEL_IMAGE_1} AND | ||
| ... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} ${TEST_MODEL_IMAGE_2} | ||
|
|
||
| *** Test Cases *** | ||
| Check EDI deploy 2 models with different versions but the same id | ||
| [Setup] NONE | ||
| [Documentation] Try to deploy 2 dummy models with different versions but the same id through EDI console | ||
| [Tags] edi cli enclave multi_versions | ||
| ${resp}= Run EDI deploy ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| ${resp}= Check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} | ||
| Should contain ${resp} "model_version": "${TEST_MODEL_1_VERSION}" | ||
| ${resp}= Run EDI deploy ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_2} | ||
| Should Be Equal As Integers ${resp.rc} 0 | ||
| ${resp}= Check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} | ||
| Should contain ${resp} "model_version": "${TEST_MODEL_2_VERSION}" |
Oops, something went wrong.