Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor clcalipso.py with handlers.yaml #218

Closed
tomvothecoder opened this issue Sep 26, 2023 · 0 comments · Fixed by #213
Closed

Refactor clcalipso.py with handlers.yaml #218

tomvothecoder opened this issue Sep 26, 2023 · 0 comments · Fixed by #213
Labels
refactor Clean up or optimize code

Comments

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Sep 26, 2023 •
edited
Loading

clcalipso.py can be added directly to handlers.py without any additional code in formulas.py.
@tomvothecoder tomvothecoder added the refactor Clean up or optimize code label Sep 26, 2023
@tomvothecoder tomvothecoder mentioned this issue Sep 26, 2023
Merged
@tomvothecoder tomvothecoder changed the title Move clcalipso.py and clisccp.py to handlers.yaml Refactor clcalipso.py with handlers.yaml Oct 31, 2023
tomvothecoder added a commit that referenced this issue Nov 8, 2023
@tomvothecoder
Refactor significant portions of the codebase and some land-atm handlers
8d3ab3a 
1. Refactor `lib.py` (now deleted)
   - Closes #190
   - Replace `run_parallel()` and `run_serial()` with `__main__.py` -> `E3SMtoCMIP._run_parallel()` and
`E3SMtoCMIP._run_serial()`
     - Updated var name `filepaths` to `vars_to_filepaths` for clarity (its a dict mapping var key to list of string paths)
     - Update args passed to handler method, remove unnecessary args
     - Add `try` and `except` statement for submitting `pool` jobs to maintain compatibility with MPAS variable handlers, which use different handler method arguments
   - Replace `handle_variables()`, `get_dimension_data()` and `load_axis()` with `VarHandler.cmorize()`
   - Delete `handle_simple()` -- _will be re-implemented from scratch in #130_
3. Update `handler.py`
   - Update `VarHandler.cmorize()`
      - Refactored significantly -- extracted logic into smaller, maintainable private methods
      - Replaced `data` dictionary storing `xr.DataArrays`  with `ds` (`xr.Dataset` object)
	  - Distinguish CMOR usage with "cmor" string in function names and python variables
	  - Add support for hybrid sigma variables: `pfull`, `phalf`
   - Add `VarHandler._cmor_write()`
     - Add support for CMORizing fixed-time variables
       - Separate PR for #217
     - If time dimension exists, CMORize the variable with all time and time bound values with a single call to `cmor.write()` instead of looping over each time value index and CMORizing each slice -- **this should improve performance and removes the `tqdm` progress bar.**
4. Update `handlers.yaml`
    - Add `phalf` and `pfull` entries
      - Closes #115
      - Delete `phalf.py` and `pfull.py`
    - Add `clcalispo` entry
      - Closes #218
      - Delete `clcalipso.py`
5. Update `_formulas.py`
   - Update all function argument types and return types from `np.ndarray` to `xr.DataArray`
   - Add formula functions for `pfull` and `phalf`
   - Add `convert_units()` function, which handles 1-to-1 unit conversions -- replaces `default.default_handlers.write_data()`
6. Refactor `default.py` (now deleted)
   - Replaced by `VarHandler.cmorize()` and `_formulas.py.convert_units()`

7. Remove `cdms2` and `cdutil` from dependencies in `dev.yml` and `ci.yml`7
8.  Clean up legacy code in `clisccp.py`
    - Separate PR for #218
9. Add `Makefile` for easy access to commonly used commands (e.g., building and installing package)
tomvothecoder added a commit that referenced this issue Nov 10, 2023
@tomvothecoder
Refactor significant portions of the codebase and refactor some land-…
12bb1b3 
…atm handlers (#213)

1. Refactor `lib.py` (now deleted)
   - Closes #190
   - Replace `run_parallel()` and `run_serial()` with `__main__.py` -> `E3SMtoCMIP._run_parallel()` and
`E3SMtoCMIP._run_serial()`
     - Updated var name `filepaths` to `vars_to_filepaths` for clarity (its a dict mapping var key to list of string paths)
     - Update args passed to handler method, remove unnecessary args
     - Add `try` and `except` statement for submitting `pool` jobs to maintain compatibility with MPAS variable handlers, which use different handler method arguments
   - Replace `handle_variables()`, `get_dimension_data()` and `load_axis()` with `VarHandler.cmorize()`
   - Delete `handle_simple()` -- _will be re-implemented from scratch in #130_
3. Update `handler.py`
   - Update `VarHandler.cmorize()`
      - Refactored significantly -- extracted logic into smaller, maintainable private methods
      - Replaced `data` dictionary storing `xr.DataArrays`  with `ds` (`xr.Dataset` object)
	  - Distinguish CMOR usage with "cmor" string in function names and python variables
	  - Add support for hybrid sigma variables: `pfull`, `phalf`
   - Add `VarHandler._cmor_write()`
     - Add support for CMORizing fixed-time variables
       - Separate PR for #217
     - If time dimension exists, CMORize the variable with all time and time bound values with a single call to `cmor.write()` instead of looping over each time value index and CMORizing each slice -- **this should improve performance and removes the `tqdm` progress bar.**
4. Update `handlers.yaml`
    - Add `phalf` and `pfull` entries
      - Closes #115
      - Delete `phalf.py` and `pfull.py`
    - Add `clcalispo` entry
      - Closes #218
      - Delete `clcalipso.py`
5. Update `_formulas.py`
   - Update all function argument types and return types from `np.ndarray` to `xr.DataArray`
   - Add formula functions for `pfull` and `phalf`
   - Add `convert_units()` function, which handles 1-to-1 unit conversions -- replaces `default.default_handlers.write_data()`
6. Refactor `default.py` (now deleted)
   - Replaced by `VarHandler.cmorize()` and `_formulas.py.convert_units()`
7. Remove `cdms2` and `cdutil` from dependencies in `dev.yml` and `ci.yml`7
8.  Clean up legacy code in `clisccp.py`
    - Separate PR for #218
9. Add `Makefile` for easy access to commonly used commands (e.g., building and installing package)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
refactor Clean up or optimize code
Projects
None yet
Milestone
No milestone
1 participant
@tomvothecoder