Skip to content

Commit

Permalink
[ENH] informative failure message in `test_get_package_metadata_retur…
Browse files Browse the repository at this point in the history
…ns_expected_results` (#239)

So far, `test_get_package_metadata_returns_expected_results` has failed
in every attempt to make a proper PR to `scikit-base`, due to the design
decision of using the package itself as a test case for `all_objects` -
so adding any function or class anywhere without making a corresponding
change in a register will fail the test.

Apart from that this needs to change, as a short term measure a more
helpful error message should allow developers to find the place to add
the expected functions or classes.

This PR adds these error messages.
  • Loading branch information
fkiraly authored Oct 23, 2023
1 parent 8a517d7 commit f08e8ac
Showing 1 changed file with 31 additions and 2 deletions.
33 changes: 31 additions & 2 deletions skbase/lookup/tests/test_lookup.py
Original file line number Diff line number Diff line change
Expand Up @@ -743,14 +743,43 @@ def test_get_package_metadata_returns_expected_results(
if exclude_non_public_items:
module_funcs = SKBASE_PUBLIC_FUNCTIONS_BY_MODULE.get(module, ())
module_classes = SKBASE_PUBLIC_CLASSES_BY_MODULE.get(module, ())
which_str = "public"
fun_str = "SKBASE_PUBLIC_FUNCTIONS_BY_MODULE"
cls_str = "SKBASE_PUBLIC_CLASSES_BY_MODULE"
else:
module_funcs = SKBASE_FUNCTIONS_BY_MODULE.get(module, ())
module_classes = SKBASE_CLASSES_BY_MODULE.get(module, ())
which_str = "all"
fun_str = "SKBASE_FUNCTIONS_BY_MODULE"
cls_str = "SKBASE_CLASSES_BY_MODULE"

# Verify expected functions are returned
assert set(results[module]["functions"].keys()) == set(module_funcs)
retrieved_funcs = set(results[module]["functions"].keys())
expected_funcs = set(module_funcs)

if retrieved_funcs != expected_funcs:
msg = (
"When using get_package_metadata utility, retrieved objects "
f"for {which_str} functions in module {module} do not match expected. "
f"Expected: {expected_funcs}; "
f"retrieved: {retrieved_funcs}. "
f"Expected functions are stored in {fun_str}, in test_lookup."
)
raise AssertionError(msg)

# Verify expected classes are returned
assert set(results[module]["classes"].keys()) == set(module_classes)
retrieved_cls = set(results[module]["classes"].keys())
expected_cls = set(module_classes)

if retrieved_cls != expected_cls:
msg = (
"When using get_package_metadata utility, retrieved objects "
f"for {which_str} classes in module {module} do not match expected. "
f"Expected: {expected_cls}; "
f"retrieved: {retrieved_cls}. "
f"Expected functions are stored in {cls_str}, in test_lookup."
)
raise AssertionError(msg)

# Verify class metadata attributes correct
for klass, klass_metadata in results[module]["classes"].items():
Expand Down

0 comments on commit f08e8ac

Please sign in to comment.