Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix integration test #22

Open
natalian98 opened this issue Sep 1, 2022 · 0 comments
Open

Fix integration test #22

natalian98 opened this issue Sep 1, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@natalian98
Copy link
Contributor

The test_using_charm integration test seems to send empty metadata and receive an empty response, giving a false-pass result. This can be observed in CI runs and locally:

$ tox -e integration -- --model=mlmd-test --keep-models
integration installed: asttokens==2.0.8,attrs==22.1.0,backcall==0.2.0,bcrypt==4.0.0,cachetools==5.2.0,certifi==2022.6.15,cffi==1.15.1,chardet==3.0.4,cryptography==37.0.4,decorator==5.1.1,executing==1.0.0,google-auth==2.11.0,idna==2.10,iniconfig==1.1.1,ipdb==0.13.9,ipython==8.4.0,jedi==0.18.1,Jinja2==3.1.2,jsonschema==3.2.0,juju==3.0.1,jujubundlelib==0.5.7,kubernetes==24.2.0,macaroonbakery==1.3.1,MarkupSafe==2.1.1,matplotlib-inline==0.1.6,mypy-extensions==0.4.3,oauthlib==3.2.0,oci-image==1.0.0,ops==1.2.0,packaging==21.3,paramiko==2.11.0,parso==0.8.3,pexpect==4.8.0,pickleshare==0.7.5,pluggy==1.0.0,prompt-toolkit==3.0.30,protobuf==3.20.1,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pycparser==2.21,Pygments==2.13.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyparsing==3.0.9,pyRFC3339==1.1,pyrsistent==0.18.1,pytest==7.1.2,pytest-asyncio==0.19.0,pytest-operator==0.22.0,python-dateutil==2.8.2,pytz==2022.2.1,PyYAML==6.0,requests==2.25.0,requests-oauthlib==1.3.1,rsa==4.9,serialized-data-interface==0.3.5,six==1.16.0,stack-data==0.5.0,theblues==0.5.2,toml==0.10.2,tomli==2.0.1,toposort==1.7,traitlets==5.3.0,typing-extensions==4.3.0,typing-inspect==0.8.0,urllib3==1.26.12,wcwidth==0.2.5,websocket-client==1.4.0,websockets==7.0
integration run-test-pre: PYTHONHASHSEED='3683306377'
integration run-test: commands[0] | pytest -vv --tb native --show-capture=no --log-cli-level=INFO -s --model=mlmd-test --keep-models /home/ubuntu/mlmd-operator/tests/integration
============================================================================ test session starts ============================================================================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 -- /home/ubuntu/mlmd-operator/.tox/integration/bin/python
cachedir: .tox/integration/.pytest_cache
rootdir: /home/ubuntu/mlmd-operator
plugins: asyncio-0.19.0, operator-0.22.0
asyncio: mode=strict
collected 2 items                                                                                                                                                           

tests/integration/test_charm.py::test_build_and_deploy SKIPPED (unconditional skip)
tests/integration/test_charm.py::test_using_charm 
------------------------------------------------------------------------------ live log setup -------------------------------------------------------------------------------
INFO     pytest_operator.plugin:plugin.py:653 Connecting to existing model my-controller:mlmd-test on unspecified cloud
------------------------------------------------------------------------------- live log call -------------------------------------------------------------------------------
INFO     root:test_charm.py:39 Using temporary directory /home/ubuntu/mlmd-operator/.tox/integration/tmp/pytest/test_using_charm0
INFO     root:test_charm.py:40 cwd = /home/ubuntu/mlmd-operator
INFO     root:test_charm.py:41 script_abs_path = /home/ubuntu/mlmd-operator/tests/integration/data/interact_with_mlmd.sh
+ MODEL=mlmd-test
+ echo MODEL=mlmd-test
MODEL=mlmd-test
+ wget https://raw.githubusercontent.com/google/ml-metadata/master/ml_metadata/proto/metadata_store.proto
--2022-08-31 09:33:53--  https://raw.githubusercontent.com/google/ml-metadata/master/ml_metadata/proto/metadata_store.proto
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 36972 (36K) [text/plain]
Saving to: ‘metadata_store.proto’

metadata_store.proto                        100%[========================================================================================>]  36.11K  --.-KB/s    in 0s      

2022-08-31 09:33:53 (110 MB/s) - ‘metadata_store.proto’ saved [36972/36972]

+ wget https://raw.githubusercontent.com/google/ml-metadata/master/ml_metadata/proto/metadata_store_service.proto
--2022-08-31 09:33:53--  https://raw.githubusercontent.com/google/ml-metadata/master/ml_metadata/proto/metadata_store_service.proto
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 39192 (38K) [text/plain]
Saving to: ‘metadata_store_service.proto’

metadata_store_service.proto                100%[========================================================================================>]  38.27K  --.-KB/s    in 0s      

2022-08-31 09:33:53 (89.3 MB/s) - ‘metadata_store_service.proto’ saved [39192/39192]

+ wget -O- https://github.com/fullstorydev/grpcurl/releases/download/v1.8.0/grpcurl_1.8.0_linux_x86_64.tar.gz
+ tar -xzv
--2022-08-31 09:33:53--  https://github.com/fullstorydev/grpcurl/releases/download/v1.8.0/grpcurl_1.8.0_linux_x86_64.tar.gz
Resolving github.com (github.com)... 140.82.113.4
Connecting to github.com (github.com)|140.82.113.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/111431261/88be8800-4f53-11eb-94fd-6a7ef1143069?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220831%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220831T093354Z&X-Amz-Expires=300&X-Amz-Signature=34b32b5c3850e69f87bbdeb08d6ef42aa04ac13e68afcb2337fe13692efddfc6&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=111431261&response-content-disposition=attachment%3B%20filename%3Dgrpcurl_1.8.0_linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2022-08-31 09:33:54--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/111431261/88be8800-4f53-11eb-94fd-6a7ef1143069?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220831%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220831T093354Z&X-Amz-Expires=300&X-Amz-Signature=34b32b5c3850e69f87bbdeb08d6ef42aa04ac13e68afcb2337fe13692efddfc6&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=111431261&response-content-disposition=attachment%3B%20filename%3Dgrpcurl_1.8.0_linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4986341 (4.8M) [application/octet-stream]
Saving to: ‘STDOUT’

-                                             0%[                                                                                         ]       0  --.-KB/s               LICENSE
grpcurl
-                                           100%[========================================================================================>]   4.75M  --.-KB/s    in 0.1s    

2022-08-31 09:33:54 (35.2 MB/s) - written to stdout [4986341/4986341]

+ mkdir -p ml_metadata/proto/
+ mv metadata_store.proto ml_metadata/proto/
++ kubectl get services/mlmd -n mlmd-test -oyaml
++ yq e .spec.clusterIP -
+ SERVICE=10.152.183.202
+ ./grpcurl -v --proto=metadata_store_service.proto --plaintext 10.152.183.202:8080 ml_metadata.MetadataStoreService/GetArtifacts

Resolved method descriptor:
// Gets all the artifacts.
rpc GetArtifacts ( .ml_metadata.GetArtifactsRequest ) returns ( .ml_metadata.GetArtifactsResponse );

Request metadata to send:
(empty)

Response headers received:
accept-encoding: identity,gzip
content-type: application/grpc
grpc-accept-encoding: identity,deflate,gzip

Response contents:
{
  
}

Response trailers received:
(empty)
Sent 0 requests and received 1 response
PASSED
----------------------------------------------------------------------------- live log teardown -----------------------------------------------------------------------------
INFO     pytest_operator.plugin:plugin.py:768 Model status:

Model      Controller     Cloud/Region     Version  SLA          Timestamp
mlmd-test  my-controller  myk8s/localhost  2.9.33   unsupported  09:33:54Z

App   Version                Status  Scale  Charm  Channel  Rev  Address         Exposed  Message
mlmd  res:oci-image@e2cb9ce  active      1  mlmd   stable    10  10.152.183.114  no       

Unit     Workload  Agent  Address     Ports     Message
mlmd/0*  active    idle   10.1.17.61  8080/TCP  
@natalian98 natalian98 added the enhancement New feature or request label Sep 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant