Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: pre-commit hook dataset-config-validation #269

Open
Bento007 opened this issue Sep 13, 2024 · 0 comments
Open

Fix: pre-commit hook dataset-config-validation #269

Bento007 opened this issue Sep 13, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@Bento007
Copy link
Contributor

Bento007 commented Sep 13, 2024

The hook makes assumptions that dependencies are install in the local python environment. This causes it to fail if those dependencies are missing. The pre-commit hook should be as independent from the local environment as possible to avoid running into issues running on different machines. The dependencies should be included in the pre-commit hook or installed as part of the make recipe it uses.

Error Message

dataset-config-validation................................................Failed
- hook id: dataset-config-validation
- exit code: 2

python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'
make: *** [validate-configs] Error 1
python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'
make: *** [validate-configs] Error 1
python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'

Definition of Done

  • only run this pre-commit on ingest config files that have changes
  • in a GHA run the same validation against all ingest configs
  • run this hook in an isolated environment with the needed dependencies installed.
@Bento007 Bento007 added the bug Something isn't working label Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant