Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PRT Enhancements. #945

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 0 additions & 100 deletions test/regression_tests.sh

This file was deleted.

Binary file removed test/resources/kitten.jpg
Binary file not shown.
30 changes: 19 additions & 11 deletions test/README.md → tests/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This folder contains regression tests executed against MMS master.These tests use [POSTMAN](https://www.postman.com/downloads/) for exercising all the Management & Inference APIs.

### Running the test manually.
### Running the test manually using docker.

Pull multi-model-server pre build docker image
```
Expand All @@ -12,7 +12,7 @@ docker pull awsdeeplearningteam/multi-model-server
This would build a docker Image with a awsdeeplearningteam/multi-model-server:latest in which we would run our Regression Tests.

```
docker run -it --user root awsdeeplearningteam/multi-model-server:latest /bin/bash
docker run -it --user root -v /tmp:/tmp awsdeeplearningteam/multi-model-server:latest /bin/bash
```

In the Docker CLI execute the following cmds.
Expand All @@ -21,23 +21,31 @@ In the Docker CLI execute the following cmds.
apt-get update
apt-get install -y git wget sudo
git clone https://github.com/awslabs/multi-model-server.git
cd multi-model-server
cd multi-model-server/test/api
```
To execute tests on master run:

`./test/regression_tests.sh `
`./regression_tests.sh `

To execute tests on different run:

`./test/regression_tests.sh <branch_name>`


You can view the logs for Test execution & the Multi-model-server in the /tmp dir.
`./regression_tests.sh <branch_name>`

### Running the test manually local environment.
```
cat /tmp/test_exec.log
cat /tmp/mms.log
git clone https://github.com/awslabs/multi-model-server.git
cd multi-model-server/test
```
To execute tests on master run:

`./regression_tests.sh `

To execute tests on different run:

`./regression_tests.sh <branch_name>`

You can view the logs for Test execution & the Multi-model-server in the /tmp/MMS_regression folder.


### Adding tests

Expand All @@ -49,4 +57,4 @@ Specifically to test for inference against a new model
![POSTMAN UI](screenshot/postman.png)

Afterwards, export the collection as a v2.1 collection and replace the existing exported collection.
To add a new suite of tests, add a new collection to /postman and update regression_tests.sh to run the new collection and buldsepc.yml to keep track of the report.
To add a new suite of tests, add a new collection to /postman and update regression_tests.sh to run the new collection and buldsepc.yml to keep track of the report.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@
"body": {
"mode": "file",
"file": {
"src": "../examples/image_classifier/kitten.jpg"
"src": "resources/kitten.jpg"
},
"options": {
"raw": {
Expand Down
Loading