You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Helm unittesting doesn't work if values section is present in *_test.yaml instead of set.
Another point is that if snapshotting testing is used it seems that snapshots in __snapshot__/ directory are ignored and any changes to tests or values in set section doesn't affect the tests run - it's still successful.
Pants version
2.15.1
OS
both
Additional info
Consider the following setup
# deployment_test.yaml
tests:
- it: checks correct number of replicas
asserts:
- equal:
path: spec.replicas
value: 2
values:
- general_values.yaml
# IF COMMENT `values` section and uncomment `set` - it works fine.
# set:
# replicaCount: 2
# BUILD
helm_unittest_tests(
sources=[
"*_test.yaml",
"general_values.yaml"
]
)
# general_values.yaml
replicaCount: 2
serviceAccount:
name: opa
The error occurs:
pants test ::
13:07:38.10 [INFO] Completed: Running Helm unittest suite awesome/tests/deployment_test.yaml
13:07:38.11 [ERROR] Completed: Run Helm Unittest - awesome/tests/deployment_test.yaml - failed (exit code 1).
### Chart [ awesome ] awesome
FAIL test deployment awesome/tests/deployment_test.yaml
- checks correct number of replicas
Error: open awesome/tests/general_values.yaml: no such file or directory
Snapshot testing is also behaves not as expected. So, according to the documentation you need to generate .snap file and put it into __snapshot__ directory. When test runs it compares the output value with the snapshot. While the manifest is generated during the run it doesn't seem to actually compare the result with the snapshot, because any changes to either to snapshot or to values in set section doesn't doesn't impact test results - it's always successful.
I believe both cases are might be related to the same issue and it's related to missing logic on how unittests are integrated into pants. Though I might missing some piece of documentation as well, in that case would appreciate any help :)
The text was updated successfully, but these errors were encountered:
Thanks for the report, as you have already found out, Helm snapshot testing is not yet supported.
Regarding the usage of additional value files in tests: this is already possible but the elements used in the BUILD files are slightly different.
The helm_unittest_tests target is meant to be used only to identify test suites or, in other words, _test.yml files only. If you change your BUILD file to be look like the following one, it should work:
Describe the bug
Helm unittesting doesn't work if
values
section is present in *_test.yaml instead ofset
.Another point is that if snapshotting testing is used it seems that snapshots in
__snapshot__/
directory are ignored and any changes to tests or values inset
section doesn't affect the tests run - it's still successful.Pants version
2.15.1
OS
both
Additional info
The error occurs:
.snap
file and put it into__snapshot__
directory. When test runs it compares the output value with the snapshot. While the manifest is generated during the run it doesn't seem to actually compare the result with the snapshot, because any changes to either to snapshot or to values inset
section doesn't doesn't impact test results - it's always successful.Might be related - #16532
I believe both cases are might be related to the same issue and it's related to missing logic on how unittests are integrated into pants. Though I might missing some piece of documentation as well, in that case would appreciate any help :)
The text was updated successfully, but these errors were encountered: