-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
resolve problems with unit test workflow #317
Conversation
prjemian
commented
Dec 19, 2023
•
edited
Loading
edited
- close BUG: problems with unit test workflow #316
- close BUG: KeyError with databroker 2.0.0b31 #318
Caught up with #314 |
The remaining problems do not happen with databroker 1.2. The CI uses databroker v2.0.0b31 (at this time). All of these problems are related to storing the UB matrix (and other information) in the run as a detector. FAILED hkl/tests/test_save_restore_UB.py::test_fourc_orientation_save - assert 0 == 1
FAILED hkl/tests/test_save_restore_UB.py::test_fourc_run_orientation_info - AssertionError: assert 'fourc' in {}
FAILED hkl/tests/test_save_restore_UB.py::test_list_orientation_runs - AttributeError: 'DataFrame' object has no attribute 'scan_id'
FAILED hkl/tests/test_save_restore_UB.py::test_no_primary_stream - AttributeError: 'DataFrame' object has no attribute 'scan_id'
FAILED hkl/tests/test_save_restore_UB.py::test_missing_energy_key - assert 0 == 1
FAILED hkl/tests/test_save_restore_UB.py::test_restore_orientation - KeyError: 'fourc'
FAILED hkl/tests/test_save_restore_UB.py::test_restore_sample - KeyError: 'fourc'
FAILED hkl/tests/test_util.py::test_issue215 - assert 0 == 1 |
It's clear that the |
Still: 2023-12-23T21:48:26.2424512Z [autodoc] from hkl.configuration import DCConfiguration.canonical_axes
2023-12-23T21:48:26.2427010Z [autodoc] import hkl.configuration => <module 'hkl.configuration' from '/home/runner/work/hklpy/hklpy/hkl/configuration.py'>
2023-12-23T21:48:26.2428908Z [autodoc] getattr(_, 'DCConfiguration')
2023-12-23T21:48:26.2430174Z [autodoc] => <class 'hkl.configuration.DCConfiguration'>
2023-12-23T21:48:26.2431422Z [autodoc] getattr(_, 'canonical_axes')
2023-12-23T21:48:26.2433439Z [app] emitting event: 'build-finished'(TypeError("traceback_exception_init() got an unexpected keyword argument 'max_group_width'"),) Switch to Py311 for this build |
Locally, docs fail 30 times on lines such as these examples: autodoc: failed to import attribute 'DCConfiguration.canonical_axes' from module 'hkl.configuration'; the following exception was raised:
autodoc: failed to import attribute 'DCConfiguration.constraints' from module 'hkl.configuration'; the following exception was raised:
autodoc: failed to import attribute 'DCConfiguration.energy_keV' from module 'hkl.configuration'; the following exception was raised:
autodoc: failed to import attribute 'DCConfiguration.engine' from module 'hkl.configuration'; the following exception was raised: All are classes using the |
This SO post is helpful. When using type annotations and |
Issue in 2023-12-23T22:42:31.9288016Z [autodoc] => <class 'hkl.configuration.DCConfiguration'>
2023-12-23T22:42:31.9289251Z [autodoc] getattr(_, 'geometry')
2023-12-23T22:42:31.9298270Z [app] emitting event: 'build-finished'(TypeError("traceback_exception_init() got an unexpected keyword argument 'max_group_width'"),)
2023-12-23T22:42:31.9300360Z
2023-12-23T22:42:31.9338927Z Error in sys.excepthook:
2023-12-23T22:42:31.9339203Z Traceback (most recent call last):
2023-12-23T22:42:31.9340648Z File "/home/runner/micromamba/envs/documentation/lib/python3.11/site-packages/dask/base.py", line 71, in wrapper
2023-12-23T22:42:31.9341217Z return func(exc_type, exc.with_traceback(tb), tb)
2023-12-23T22:42:31.9342102Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-23T22:42:31.9343323Z File "/home/runner/micromamba/envs/documentation/lib/python3.11/site-packages/trio/_core/_multierror.py", line 447, in trio_excepthook
2023-12-23T22:42:31.9344046Z for chunk in traceback.format_exception(etype, value, tb):
2023-12-23T22:42:31.9346975Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-23T22:42:31.9348181Z File "/home/runner/micromamba/envs/documentation/lib/python3.11/traceback.py", line 139, in format_exception
2023-12-23T22:42:31.9348894Z te = TracebackException(type(value), value, tb, limit=limit, compact=True)
2023-12-23T22:42:31.9351416Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-23T22:42:31.9353171Z File "/home/runner/micromamba/envs/documentation/lib/python3.11/site-packages/trio/_core/_multierror.py", line 393, in traceback_exception_init |
Locally, trio 0.23.2, which is latest release. |
@padraic-shafer @mrakitin This is ready now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I have a clarification question about the logic in the run_orientation_info
.
I like this. Co-authored-by: Padraic Shafer <76011594+padraic-shafer@users.noreply.github.com>
@mrakitin @padraic-shafer - Anything else? |
Absent additional comments to be addressed, I plan to merge this PR by the end of 2024-01-10. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me now!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Some suggestions were added in the comments.
# TODO: Once py38 is dropped, re-enable the default value setting | ||
U: List[List[float]] # = field(default_factory=list[list[float]]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is py38 the issue, or is it because DCReflection is not a dataclass
?
I believe the default_factory
field should be a zero-argument callable, such as...
default_factory=lambda: [[]] # list containing an empty list
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I can check that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, a change to
U: field(default_factory=lambda: [[]]) # list containing an empty list
produces many versions of this exception. This example pytest -vvv --lf hkl/tests/test_configuration.py::test_restore
tests the first such CI failure:
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0 -- /home/prjemian/.conda/envs/bluesky_2024_1/bin/python3.11
cachedir: .pytest_cache
rootdir: /home/prjemian/Documents/projects/Bluesky/hklpy
configfile: pytest.ini
plugins: pytest_notebook-0.9.0, anyio-4.0.0
collected 1 item
run-last-failure: 4 known failures not in selected tests
hkl/tests/test_configuration.py::test_restore FAILED [100%]
====================================================================== FAILURES ======================================================================
____________________________________________________________________ test_restore ____________________________________________________________________
e4cv_renamed = CustomFourCircle(prefix='', name='e4cv_renamed', settle_time=0.0, timeout=None, egu='', limits=(0, 0), source='compute... '_hklpy_version', '_pseudos', '_reals', '_constraints', '_mode', 'orientation_attrs', 'h', 'k', 'l'], concurrent=True)
k4cv = SimulatedK4CV(prefix='', name='k4cv', settle_time=0.0, timeout=None, egu='', limits=(0, 0), source='computed', read_at... '_hklpy_version', '_pseudos', '_reals', '_constraints', '_mode', 'orientation_attrs', 'h', 'k', 'l'], concurrent=True)
def test_restore(e4cv_renamed, k4cv):
config = DiffractometerConfiguration(e4cv_renamed)
> before = config.export("dict")
hkl/tests/test_configuration.py:37:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hkl/configuration.py:540: in export
data = getattr(self, f"to_{fmt}")()
hkl/configuration.py:804: in to_dict
return serialize(DCConfiguration, self.model)
hkl/configuration.py:745: in model
obj = deserialize(DCConfiguration, data) # also validates structure
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/apischema/deserialization/__init__.py:887: in deserialize
return deserialization_method(
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/apischema/deserialization/__init__.py:817: in deserialization_method
deserialization_method_factory(
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/apischema/deserialization/__init__.py:753: in deserialization_method_factory
).visit_with_conv(tp, conversion)
...
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/apischema/objects/visitor.py:129: in unsupported
return super().unsupported(tp) if fields is dummy else self._object(tp, fields)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apischema.deserialization.DeserializationMethodVisitor object at 0x7f47556da810>
tp = Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object at 0x7f47d0d2fd50>,default_factory=<function DCSam...e,compare=True,metadata=mappingproxy({}),kw_only=<dataclasses._MISSING_TYPE object at 0x7f47d0d2fd50>,_field_type=None)
def unsupported(self, tp: AnyType) -> Result:
> raise Unsupported(tp)
E apischema.visitor.Unsupported: Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object at 0x7f47d0d2fd50>,default_factory=<function DCSample.<lambda> at 0x7f476f358860>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=<dataclasses._MISSING_TYPE object at 0x7f47d0d2fd50>,_field_type=None)
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/apischema/visitor.py:151: Unsupported
================================================================== warnings summary ==================================================================
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
../../../../.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
/home/prjemian/.conda/envs/bluesky_2024_1/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================== short test summary info ===============================================================
FAILED hkl/tests/test_configuration.py::test_restore - apischema.visitor.Unsupported: Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object at 0x7f47d0d2fd50>,default_factory=<function D...
=========================================================== 1 failed, 2 warnings in 0.23s ============================================================
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note this example used Py3.11
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created new issue for this.
Co-authored-by: Padraic Shafer <76011594+padraic-shafer@users.noreply.github.com>
Co-authored-by: Padraic Shafer <76011594+padraic-shafer@users.noreply.github.com>
@padraic-shafer @mrakitin : added the pipefail to the CI -- all tests pass at this time |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me!