Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests missing diff #239

Open
burlen opened this issue May 21, 2019 · 4 comments
Open

tests missing diff #239

burlen opened this issue May 21, 2019 · 4 comments
Assignees
Labels
1_high_priority an issue that should be fixed prior to the next release testing

Comments

@burlen
Copy link
Collaborator

burlen commented May 21, 2019

a number of tests run the pipeline without checking the result. these can be fixed now that we have mesh based diff.

  1. test_temporal_average.cpp
  2. test_cf_reader.cpp
  3. test_programmable_algorithm.py
  4. test_derived_quantity.py
  5. test_connected_components.py
  6. test_cf_reader.py
  7. test_vector_ops.py
  8. test_binary_segmentation.cpp

use dataset_diff in these tests to validate the output

@burlen burlen added this to the Bayesian AR Detector Release milestone May 21, 2019
elbashandy added a commit that referenced this issue Jun 11, 2019
elbashandy added a commit that referenced this issue Jul 26, 2019
elbashandy added a commit that referenced this issue Dec 11, 2019
elbashandy added a commit that referenced this issue Dec 12, 2019
elbashandy added a commit that referenced this issue Dec 12, 2019
test diff has beed added to all mentioned alg tests. cf_reader cxx & python tests are failing (cam5 test in cxx)
@elbashandy
Copy link
Contributor

This is done but will not close it because the cf_reader is giving an error with the cam5 dataset.

The error:

test 6
    Start 6: test_cf_reader_cam5

6: Test command: /project/TECA_build/bin/test/test_cf_reader "-i" "/user_host/TECA_data/cam5_1_amip_run2\.cam2\.h2\.1991-10-[0-9][0-9]-10800\.nc" "-o" "/user_host/TECA_data/test_cf_reader_cam5.bin" "-s" "1,2" "-x" "lon" "-y" "lat" "-t" "time" "U850" "V850"
6: Test timeout computed to be: 1500
6: STATUS: [0:139878790192064] [/user_host/TECA/io/teca_cf_reader.cxx:389 2.2.1-225-g1ee302e]
6: STATUS: Found metadata cache "/user_host/TECA_data/4ba9d904ca81b960467deed857eca3214cabf3ad.md"
6: ERROR: [0:139878790192064] [/user_host/TECA/alg/teca_dataset_diff.cxx:366 2.2.1-225-g1ee302e]
6: ERROR: relative difference 0.177221 exceeds tolerance 1e-06 in element 0. ref value "-2.86862" is not equal to test value "-2.36024"
6: ERROR: [0:139878790192064] [/user_host/TECA/alg/teca_dataset_diff.cxx:425 2.2.1-225-g1ee302e]
6: ERROR: difference in array 0 "U850"
6: ERROR: [0:139878790192064] [/user_host/TECA/alg/teca_dataset_diff.cxx:471 2.2.1-225-g1ee302e]
6: ERROR: difference in point arrays
6: ERROR: [0:139878790192064] [/user_host/TECA/alg/teca_dataset_diff.cxx:247 2.2.1-225-g1ee302e]
6: ERROR: Failed to compare cartesian meshes.
1/1 Test #6: test_cf_reader_cam5 ..............***Failed  Error regular expression found in output. Regex=[ERROR]  0.27 sec

@burlen
Copy link
Collaborator Author

burlen commented Dec 16, 2019

@elbashandy It is not done if there is a failing test.

is there a MR related to the above commit?

@elbashandy
Copy link
Contributor

No I didn't do a merge request till we fix that issue.

elbashandy added a commit that referenced this issue Dec 18, 2019
test diff has beed added to all mentioned alg tests. cf_reader cxx & python tests were failing with cam5 as it was diffing one of 2 time step overwritten outputs with one of the time step inputs
@burlen burlen added the testing label May 12, 2020
@burlen
Copy link
Collaborator Author

burlen commented May 12, 2020

see also #256

@taobrienlbl taobrienlbl added the 3_low_priority documents an issue or new feature that is not time critical label Jun 19, 2020
@elbashandy elbashandy added 1_high_priority an issue that should be fixed prior to the next release and removed 3_low_priority documents an issue or new feature that is not time critical labels Jun 22, 2020
@burlen burlen removed this from the someday milestone Jul 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1_high_priority an issue that should be fixed prior to the next release testing
Projects
None yet
Development

No branches or pull requests

3 participants