Skip to content

Commit

Permalink
[#133] add the third test bare model
Browse files Browse the repository at this point in the history
  • Loading branch information
@ablatov
ablatov committed Sep 11, 2018
1 parent dca0e30 commit bb42c1a
Show file tree
Hide file tree
Showing 7 changed files with 108 additions and 47 deletions.
7 changes: 6 additions & 1 deletion Jenkinsfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,12 @@ node {
cd k8s/test-bare-model-api/model-2
docker build --build-arg version="${Globals.buildVersion}" -t legion/test-bare-model-api-model-2:${Globals.buildVersion} ${Globals.dockerLabels} .
"""
}, 'Build Edi Docker image': {
}, 'Build Bare model 3': {
sh """
cd k8s/test-bare-model-api/model-3
docker build --build-arg version="${Globals.buildVersion}" -t legion/test-bare-model-api-model-3:${Globals.buildVersion} ${Globals.dockerLabels} .
"""
},'Build Edi Docker image': {
sh """
cd k8s/edi
docker build --build-arg version="${Globals.buildVersion}" --build-arg pip_extra_index_params="--extra-index-url ${params.PyPiRepository}" --build-arg pip_legion_version_string="==${Globals.buildVersion}" -t legion/k8s-edi:${Globals.buildVersion} ${Globals.dockerLabels} .
Expand Down
13 changes: 13 additions & 0 deletions k8s/test-bare-model-api/model-3/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM python:3.6-alpine3.7

COPY . /app
WORKDIR /app

RUN pip install -r requirements.txt

LABEL com.epam.legion.model.id="demo-abc-model" \
com.epam.legion.model.version="1.2" \
com.epam.legion.container_type="model"

ENTRYPOINT ["python"]
CMD ["app.py"]
40 changes: 40 additions & 0 deletions k8s/test-bare-model-api/model-3/app.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
from flask import Flask, jsonify

app = Flask(__name__)


@app.route('/')
def root():
return jsonify(demo=True)


@app.route('/healthcheck')
def healthcheck():
return jsonify(demo=True)


@app.route('/api/model/<model>/<version>/info', methods=['GET', 'POST'])
def model_info(model, version):
return jsonify(
model_version=version,
model_id=model,
endpoints={
'default': {
'name': 'default',
'use_df': False,
'input_params': {'b': {'numpy_type': 'int64', 'type': 'Integer'},
'a': {'numpy_type': 'int64', 'type': 'Integer'}}

}
}
)


@app.route('/api/model/<model>/<version>/invoke', methods=['GET', 'POST'])
@app.route('/api/model/<model>/<version>/invoke/<endpoint>', methods=['GET', 'POST'])
def model_invoke(model, version, endpoint=None):
return jsonify(result=42.0)


if __name__ == '__main__':
app.run(host='0.0.0.0')
1 change: 1 addition & 0 deletions k8s/test-bare-model-api/model-3/requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Flask==0.10.1
1 change: 1 addition & 0 deletions legion_test/requirements/base.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ python-jenkins==0.4.15
Jinja2==2.10
robotframework==3.0.2
robotframework-seleniumlibrary==3.1.1
robotframework-pabot==0.44
requests==2.9.1
PyYAML==3.12
kubernetes==5.0.0
Expand Down
92 changes: 46 additions & 46 deletions tests/robot/5_check_edi_multi_versioned_models.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ Library Collections
Library Process
Suite Setup Choose cluster context ${CLUSTER_NAME}
Test Setup Run Keywords
... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} AND
... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} AND
... Run EDI deploy and check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_3} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Test Teardown Run Keywords
... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION} ${TEST_MODEL_IMAGE_1} AND
... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} ${TEST_MODEL_IMAGE_2}
... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION} ${TEST_MODEL_IMAGE_2} AND
... Run EDI undeploy by model version and check ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION} ${TEST_MODEL_IMAGE_3}

*** Test Cases ***
Check EDI availability in all enclaves
Expand All @@ -29,42 +29,42 @@ Check EDI availability in all enclaves
Check EDI deploy 2 models with different versions but the same id
[Setup] NONE
[Tags] edi cli enclave multi_versions
${resp}= Run EDI deploy ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_1}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
Should contain ${resp} "model_version": "${TEST_MODEL_1_VERSION}"
${resp}= Run EDI deploy ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_2}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Should contain ${resp} "model_version": "${TEST_MODEL_2_VERSION}"
${resp}= Run EDI deploy ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_IMAGE_3}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Check model started ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Should contain ${resp} "model_version": "${TEST_MODEL_3_VERSION}"

Check EDI undeploy 1 of 2 models with different versions but the same id
[Tags] edi cli enclave multi_versions
${resp}= Run EDI undeploy with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${resp}= Run EDI undeploy with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
Should Be Equal As Integers ${resp.rc} 0
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_1}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_3}

Check EDI undeploy all model instances by version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_1_VERSION}
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect with parse by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Log ${model_1}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Log ${model_2}
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 1
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_3} ${TEST_MODEL_3_VERSION} 1

${resp}= Run EDI undeploy with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${resp}= Run EDI undeploy with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
Should Be Equal As Integers ${resp.rc} 0
Should not contain ${resp.stdout} |${TEST_MODEL_1_VERSION}
Should contain ${resp.stdout} |${TEST_MODEL_2_VERSION}
Should not contain ${resp.stdout} |${TEST_MODEL_2_VERSION}
Should contain ${resp.stdout} |${TEST_MODEL_3_VERSION}

Check EDI undeploy 1 of 2 models without version
[Tags] edi cli enclave multi_versions
Expand All @@ -74,41 +74,41 @@ Check EDI undeploy 1 of 2 models without version

Check EDI scale up 1 of 2 models with different versions but the same id
[Tags] edi cli enclave multi_versions
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_1_VERSION}
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect with parse by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Log ${model_1}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Log ${model_2}
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 1
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_3} ${TEST_MODEL_3_VERSION} 1

Check EDI scale down 1 of 2 models with different versions but the same id
[Tags] edi cli enclave multi_versions
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_1_VERSION}
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect with parse by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Log ${model_1}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Log ${model_2}
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 1
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 2
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_3} ${TEST_MODEL_3_VERSION} 1

${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 1 ${TEST_MODEL_1_VERSION}
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 1 ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
${resp}= Run EDI inspect with parse by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${model_1}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Log ${model_1}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
${model_2}= Find model information in edi ${resp} ${TEST_MODEL_ID} ${TEST_MODEL_3_VERSION}
Log ${model_2}
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_1} ${TEST_MODEL_1_VERSION} 1
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 1
Verify model info from edi ${model_1} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_2} ${TEST_MODEL_2_VERSION} 1
Verify model info from edi ${model_2} ${TEST_MODEL_ID} ${TEST_MODEL_IMAGE_3} ${TEST_MODEL_3_VERSION} 1

Check EDI scale up 1 of 2 models by invalid version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_1_VERSION}121
${resp}= Run EDI scale with version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} 2 ${TEST_MODEL_2_VERSION}121
Should Be Equal As Integers ${resp.rc} 2
Should contain ${resp.stderr} No one model can be found

Expand All @@ -122,22 +122,22 @@ Check EDI model inspect by model id return 2 models
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model id ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}
Should Be Equal As Integers ${resp.rc} 0
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_1}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_3}

Check EDI model inspect by model version return 1 model
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_1_VERSION}
${resp}= Run EDI inspect by model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_1}
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_3}

Check EDI model inspect by model id and version return 1 model
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_1}
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should contain ${resp.stdout} ${TEST_MODEL_IMAGE_2}
Should not contain ${resp.stdout} ${TEST_MODEL_IMAGE_3}

Check EDI model inspect by invalid model id
[Tags] edi cli enclave multi_versions
Expand All @@ -147,24 +147,24 @@ Check EDI model inspect by invalid model id

Check EDI model inspect by invalid model version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_1_VERSION}test
${resp}= Run EDI inspect by model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_2_VERSION}test
Should Be Equal As Integers ${resp.rc} 0
Should be empty ${resp.stdout}

Check EDI model inspect by invalid model id and invalid version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}test ${TEST_MODEL_1_VERSION}test
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}test ${TEST_MODEL_2_VERSION}test
Should Be Equal As Integers ${resp.rc} 0
Should be empty ${resp.stdout}

Check EDI model inspect by invalid model id and valid version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}test ${TEST_MODEL_1_VERSION}
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID}test ${TEST_MODEL_2_VERSION}
Should Be Equal As Integers ${resp.rc} 0
Should be empty ${resp.stdout}

Check EDI model inspect by valid model id and invalid version
[Tags] edi cli enclave multi_versions
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_1_VERSION}test
${resp}= Run EDI inspect by model id and model version ${MODEL_TEST_ENCLAVE} ${TEST_MODEL_ID} ${TEST_MODEL_2_VERSION}test
Should Be Equal As Integers ${resp.rc} 0
Should be empty ${resp.stdout}
1 change: 1 addition & 0 deletions tests/robot/load_variables_from_profiles.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,6 @@ def get_variables(arg):
variables = legion_test.profiler_loader.get_variables(arg)
variables['TEST_MODEL_IMAGE_1'] = legion_test.test_assets.get_test_bare_model_api_image(variables, 1)
variables['TEST_MODEL_IMAGE_2'] = legion_test.test_assets.get_test_bare_model_api_image(variables, 2)
variables['TEST_MODEL_IMAGE_3'] = legion_test.test_assets.get_test_bare_model_api_image(variables, 3)

return variables

0 comments on commit bb42c1a

Please sign in to comment.