Test/save_dataset

[siligam]

Thank you for contributing! 🎉

Summary of the most important changes

• tests driven approach to refine saving dataset process
• Allows saving dataset with single timestep Save Dataset does not handle 1-timestep files well #61
• time_range used in file names conforms to CMIP guide lines.
• Avoids unnecessary data resampling.

Usage example in the PR

Please put a small usage example in the PR body so that we can see what you want to do.

These sorts of examples should also be in the docstrings of your code!

Checklist

• I have tested the changes in this PR.
• I have updated the documentation.
• If I have made a new step which requires new arguments, these are added to the validate.py file
  so that the user can see documentation for the arguments.

Copilot Summary

If you have Github Co-Pilot, please include an automatically generated summary.

[pgierz]

The failing tests look like segfaults, and should be handled when #66 is done. I would recommend merging that branch into this one and seeing if it works then.

[pgierz]

This looks good, but I have some questions and suggestions that you could have a look at.

[pgierz]

I'd also like to have @mandresm read through these changes to see that he understands them as well.

[pgierz]

Some last cleanup, see suggested changes.

[pgierz]

Copilot Summary

This pull request includes several important changes to the pymorize package, focusing on improving configuration handling, enhancing dataset processing, and updating testing workflows. The most significant changes are grouped into configuration handling improvements, dataset processing enhancements, and testing workflow updates.

Configuration Handling Improvements:

• src/pymorize/cmorizer.py: Added copy import and modified _post_init_attach_pymorize_config_rules to use copy.deepcopy for configuration to ensure rules get independent copies of the configuration. [1] [2]
• src/pymorize/config.py: Updated example code to use yaml.dump for writing configuration files, ensuring proper YAML formatting.

Dataset Processing Enhancements:

• src/pymorize/cmorizer.py: Modified _post_init_populate_rules_with_tables to use tbl.table_id instead of tbl for adding tables to rules.
• src/pymorize/cmorizer.py: Updated _parallel_process_dask to use deep copies of rules before processing to avoid unintended modifications.
• src/pymorize/files.py: Added functions to handle time label precision in file names and updated file path creation to ensure directories are created if they do not exist. [1] [2] [3] [4]

Testing Workflow Updates:

• .github/workflows/CI-test.yaml: Added PREFECT_SERVER_EPHEMERAL_STARTUP_TIMEOUT_SECONDS environment variable to ensure sufficient startup time for Prefect server during tests.
• tests/integration/test_fesom_2p6_pimesh_esm_tools.py: Marked a test as skipped to investigate a segmentation fault issue.

These changes collectively improve the robustness and reliability of the pymorize package, particularly in handling configurations and processing datasets.