Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not so auto update #1959

Merged
merged 21 commits into from
Dec 6, 2024
Merged

Not so auto update #1959

merged 21 commits into from
Dec 6, 2024

Conversation

visr
Copy link
Member

@visr visr commented Dec 3, 2024

The monthly updates failed. This does them manually and fixes the reasons they failed.

@visr
Copy link
Member Author

visr commented Dec 3, 2024

@evetion do we need to use another module?

15:25:43   ERROR: Unable to locate a modulefile for 'gcc/11.3.0'

And have you seen such a PicklingError before?

15:25:45   ✨ Pixi task (generate-testmodels in dev): python utils/generate-testmodels.py
15:25:54   Traceback (most recent call last):
15:25:54     File "/opt/teamcityagent/work/ecd2b8f9b25b1609/ribasim/utils/generate-testmodels.py", line 44, in <module>
15:25:54       for model_name in p.imap_unordered(generate_model_partial, models):
15:25:54                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:25:54     File "/opt/teamcityagent/work/ecd2b8f9b25b1609/ribasim/.pixi/envs/dev/lib/python3.12/multiprocessing/pool.py", line 873, in next
15:25:54       raise value
15:25:54   multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fc8414cf3e0>'. Reason: 'PicklingError("Can't pickle <class 'ribasim.input_base.TableModel[BasinProfileSchema]'>: attribute lookup TableModel[BasinProfileSchema] on ribasim.input_base failed")'
15:25:54   Process exited with code 1

@visr
Copy link
Member Author

visr commented Dec 3, 2024

I updated the module load command to the currently available module, and updated to GCC 12. Also fixed some mypy unused ignores. But there seems to be several more issues that need to be fixed.

This looks like an installation failure of typeguard that I see locally. Explicitly adding with pixi didn't seem to help. A pixi clean cache, pixi clean and pixi self-update did work (probably not all are needed).

pixi run generate-testmodels

  File "A:\repo\ribasim\Ribasim\python\ribasim\ribasim\geometry\area.py", line 1, in <module>
    import pandera as pa
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\__init__.py", line 10, in <module>
    import pandera.backends
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\backends\__init__.py", line 4, in <module>
    import pandera.backends.base.builtin_checks
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\backends\base\builtin_checks.py", line 14, in <module>
    from pandera.api.checks import Check
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\api\checks.py", line 17, in <module>
    from pandera.strategies.base_strategies import SearchStrategy
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\strategies\__init__.py", line 3, in <module>
    from pandera.strategies.pandas_strategies import *
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\strategies\pandas_strategies.py", line 42, in <module>
    from pandera.engines import numpy_engine, pandas_engine
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pandera\engines\pandas_engine.py", line 33, in <module>
    import typeguard
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\typeguard\__init__.py", line 21, in <module>
    from ._importhook import ImportHookManager as ImportHookManager
  File "A:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\typeguard\_importhook.py", line 35, in <module>
    OPTIMIZATION = "typeguard" + "".join(version("typeguard").split(".")[:3])
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'

On CI this fails with a pickling error

✨ Pixi task (generate-testmodels in dev): python utils/generate-testmodels.py
Traceback (most recent call last):
  File "/home/runner/work/Ribasim/Ribasim/utils/generate-testmodels.py", line 44, in <module>
    for model_name in p.imap_unordered(generate_model_partial, models):
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/Ribasim/Ribasim/.pixi/envs/dev/lib/python3.12/multiprocessing/pool.py", line [8](https://github.com/Deltares/Ribasim/actions/runs/12147545818/job/33874020840?pr=1959#step:7:9)73, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7ff[10](https://github.com/Deltares/Ribasim/actions/runs/12147545818/job/33874020840?pr=1959#step:7:11)76f0f80>'. Reason: 'PicklingError("Can't pickle <class 'ribasim.input_base.TableModel[BasinProfileSchema]'>: attribute lookup TableModel[BasinProfileSchema] on ribasim.input_base failed")'

I tried rolling back from pandera 0.21 to 0.20 but it didn't seem to help.

@evetion
Copy link
Member

evetion commented Dec 5, 2024

I can reproduce the CI error, not your typeguard one.

@visr
Copy link
Member Author

visr commented Dec 5, 2024

Yeah neither can I, that is just pixi environment installation sometimes going wrong leading to this kind of error.

visr added 5 commits December 5, 2024 13:20
```
ValidationError                           Traceback (most recent call last)
File a:\repo\ribasim\Ribasim\models\see.py:1
----> 1 ribasim_testmodels.trivial_model()

File A:\repo\ribasim\Ribasim\python\ribasim_testmodels\ribasim_testmodels\trivial.py:9, in trivial_model()
      7 def trivial_model() -> Model:
      8     """Trivial model with just a basin, tabulated rating curve and terminal node."""
----> 9     model = Model(
     10         starttime="2020-01-01",
     11         endtime="2021-01-01",
     12         crs="EPSG:28992",
     13         results=Results(subgrid=True, compression=False),
     14         use_validation=True,
     15     )
     17     # Convert steady forcing to m/s
     18     # 2 mm/d precipitation, 1 mm/d evaporation
     19     precipitation = 0.002 / 86400

File a:\repo\ribasim\Ribasim\.pixi\envs\dev\Lib\site-packages\pydantic\main.py:214, in BaseModel.__init__(self, **data)
    212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215 if self is not validated_self:
    216     warnings.warn(
    217         'A custom validator is returning a value other than `self`.\n'
    218         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
    219         'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
    220         stacklevel=2,
    221     )

ValidationError: 1 validation error for Model
  Value error, Can't do inplace setting when converting from DataFrame to GeoDataFrame [type=value_error, input_value={'starttime': '2020-01-01... 'use_validation': True}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/value_error
```
pixi.toml Show resolved Hide resolved
@evetion
Copy link
Member

evetion commented Dec 5, 2024

🎉

@visr visr requested a review from evetion December 5, 2024 21:15
Copy link
Member

@evetion evetion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make an issue (if not already done so) to remove these conda workarounds in all three places?

@visr visr merged commit f1c7904 into main Dec 6, 2024
28 checks passed
@visr visr deleted the not-so-auto-update branch December 6, 2024 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants