Omdao #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

akey7 merged 17 commits into develop from omdao

Jan 7, 2021

.github/ISSUE_TEMPLATE/bug_report.md

-Original file line number
+Diff line change
@@ -0,0 +1,30 @@
+    ---
+    name: Bug report
+    about: Create a report to help us improve
+    title: ''
+    labels: ''
+    assignees: ''
+    ---
+    ## Description
+    _Describe the bug here_
+    ### Steps to reproduce issue
+    _Please provide a minimum working example (MWE) if possible_
+. …
+. …
+. …
+    ### Current behavior
+    …
+    ### Expected behavior
+    …
+    ### Code versions
+    _List versions only if relevant_
+    - Python
+    - …

.github/ISSUE_TEMPLATE/feature_request.md

-Original file line number
+Diff line change
@@ -0,0 +1,14 @@
+    ---
+    name: Feature request
+    about: Suggest an idea for this project
+    title: ''
+    labels: ''
+    assignees: ''
+    ---
+    # Description of feature
+    Describe the feature here and provide some context. Under what scenario would this be useful?
+    # Potential solution
+    Can you think of ways to implement this?

.github/PULL_REQUEST_TEMPLATE.md

-Original file line number
+Diff line change
@@ -0,0 +1,28 @@
+    Delete the text explanations below these headers and replace them with information about your PR.
+    Please first consult the [developer guide](https://weis.readthedocs.io/en/latest/how_to_contribute_code.html) to make sure your PR follows all code, testing, and documentation conventions.
+    ## Purpose
+    Explain the goal of this pull request. If it addresses an existing issue be sure to link to it. Describe the big picture of your changes here, perhaps using a bullet list if multiple changes are done to accomplish a single goal. If it accomplishes multiple goals, it may be best to create separate PR's for each.
+    ## Type of change
+    What types of change is it?
+    _Select the appropriate type(s) that describe this PR_
+    - [ ] Bugfix (non-breaking change which fixes an issue)
+    - [ ] New feature (non-breaking change which adds functionality)
+    - [ ] Breaking change (non-backwards-compatible fix or feature)
+    - [ ] Code style update (formatting, renaming)
+    - [ ] Refactoring (no functional changes, no API changes)
+    - [ ] Documentation update
+    - [ ] Maintenance update
+    - [ ] Other (please describe)
+    ## Testing
+    Explain the steps needed to test the new code to verify that it does indeed address the issue and produce the expected behavior.
+    ## Checklist
+    _Put an `x` in the boxes that apply._
+    - [ ] I have run existing tests which pass locally with my changes
+    - [ ] I have added new tests or examples that prove my fix is effective or that my feature works
+    - [ ] I have added necessary documentation

.github/workflows/CI_LandBOSSE.yml

-Original file line number
+Diff line change
@@ -0,0 +1,83 @@
+    name: CI_LandBOSSE
+    # We run CI on push commits and pull requests on all branches
+    on: [push, pull_request]
+    # A workflow run is made up of one or more jobs that can run sequentially or in parallel
+    jobs:
+        build_pip:
+            name: Pip Build (${{ matrix.os }})
+            runs-on: ${{ matrix.os }}
+            strategy:
+                fail-fast: False
+                matrix:
+                    os: ["ubuntu-latest", "windows-latest"]
+                    python-version: [3.6, 3.7, 3.8]
+            steps:
+                - uses: actions/checkout@v2
+                - name: Set up Python ${{ matrix.python-version }}
+                  uses: actions/setup-python@v2
+                  with:
+                      python-version: ${{ matrix.python-version }}
+                - name: Pip Install Dependencies
+                  shell: pwsh
+                  run: |
+                      python -m pip install --upgrade pip install pytest pandas numpy scipy xlsxwriter openpyxl openmdao
+                - name: Pip Install LandBOSSE
+                  shell: pwsh
+                  run: |
+                      pip install -e .
+                # Validate
+                - name: Pip Validation
+                  shell: pwsh
+                  run: |
+                      python main.py --input project_input_template --output project_input_template --validate
+                # Run tests
+                - name: Pip Run pytest
+                  shell: pwsh
+                  run: |
+                      pytest landbosse/tests
+        build_conda:
+            name: Conda Build (${{ matrix.os }})
+            runs-on: ${{ matrix.os }}
+            strategy:
+                fail-fast: False
+                matrix:
+                    os: ["ubuntu-latest", "windows-latest"]
+                    python-version: [3.8]
+            steps:
+                - uses: actions/checkout@v2
+                - uses: conda-incubator/setup-miniconda@v2
+                  # https://github.com/marketplace/actions/setup-miniconda
+                  with:
+                      miniconda-version: "latest"
+                      auto-update-conda: true
+                      python-version: 3.8
+                      environment-file: environment.yml
+                # Install
+                - name: Conda Install LandBOSSE
+                  shell: pwsh
+                  run: |
+                      python setup.py develop
+                # Validate
+                - name: Conda Validation
+                  shell: pwsh
+                  run: |
+                      python main.py --input project_input_template --output project_input_template --validate
+                # Run tests
+                - name: Conda Run pytest
+                  shell: pwsh
+                  run: |
+                      pytest landbosse/tests

.gitignore

-Original file line number
+Diff line change
@@ Expand Up / @@ -131,6 +131,9 @@ dmypy.json @@
     # VSCode
     .vscode/
+    # Emacs
+    *~
     # Ignore Pandas _libs files
     pandas/_libs/
@@ Expand Down @@

environment.yml

-Original file line number
+Diff line change
@@ -0,0 +1,18 @@
+    name: test
+    channels:
+      - conda-forge
+      - defaults
+    dependencies:
+      - python
+      - setuptools
+      - et-xmlfile
+      - pytest
+      - openpyxl
+      - xlsxwriter
+      - pandas
+      - numpy
+      - scipy
+      - openmdao
+      - pip

landbosse/excelio/XlsxDataframeCache.py

-Original file line number
+Diff line change
@@ Expand Up @@
             else:
                 xlsx_filename = os.path.join(xlsx_path, f'{xlsx_basename}.xlsx')
-            xlsx = pd.ExcelFile(xlsx_filename)
+            xlsx = pd.ExcelFile(xlsx_filename, engine='openpyxl')
             sheets_dict = {sheet_name: xlsx.parse(sheet_name) for sheet_name in xlsx.sheet_names}
+            for sheet_name in xlsx.sheet_names:
+                sheets_dict[sheet_name].dropna(inplace=True, how='all')
             cls._cache[xlsx_basename] = sheets_dict
             return cls.copy_dataframes(sheets_dict)
@@ Expand Down @@

landbosse/excelio/XlsxValidator.py

-Original file line number
+Diff line change
@@ Expand Up @@
             # the raw_cost and raw_cost_total_or_per_turbine columns.
             actual_df = pd.DataFrame(actual_module_type_operation_list)
             actual_df.drop(['raw_cost', 'raw_cost_total_or_per_turbine'], axis=1, inplace=True)
-            expected_df = pd.read_excel(expected_xlsx, 'costs_by_module_type_operation')
+            expected_df = pd.read_excel(expected_xlsx, 'costs_by_module_type_operation', engine='openpyxl')
+            #expected_df = expected_df.dropna(inplace=True, how='all')
             expected_df.rename(columns={
                 'Project ID with serial': 'project_id_with_serial',
                 'Number of turbines': 'num_turbines',
@@ Expand Down @@

landbosse/landbosse_omdao/CsvGenerator.py

-Original file line number
+Diff line change
@@ -0,0 +1,113 @@
+    import pandas as pd
+    class CsvGenerator:
+        """
+        This class generates CSV files.
+        """
+        def __init__(self, file_ops):
+            """
+            Parameters
+            ----------
+            file_ops : XlsxFileOperations
+                An instance of XlsxFileOperations to manage file names.
+            """
+            self.file_ops = file_ops
+        def create_details_dataframe(self, details):
+            """
+            This writes the details .csv.
+            Parameters
+            ----------
+            details : list[dict]
+                A list of dictionaries to be converted into a Pandas dataframe
+            Returns
+            -------
+            pd.DataFrame
+                The dataframe that can be written to a .csv file.
+            """
+            # This the list of details to write to the .csv
+            details_to_write_to_csv = []
+            for row in details:
+                new_row = {}
+                new_row["Project ID with serial"] = row["project_id_with_serial"]
+                new_row["Module"] = row["module"]
+                new_row["Variable name"] = row["variable_df_key_col_name"]
+                new_row["Unit"] = row["unit"]
+                value = row["value"]
+                value_is_number = self._is_numeric(value)
+                if value_is_number:
+                    new_row["Numeric value"] = value
+                else:
+                    new_row["Non-numeric value"] = value
+                # If there is a last_number, which means this is a dataframe row that has a number
+                # at the end, write this into the numeric value column. This overrides automatic
+                # type detection.
+                if "last_number" in row:
+                    new_row["Numeric value"] = row["last_number"]
+                details_to_write_to_csv.append(new_row)
+            details = pd.DataFrame(details_to_write_to_csv)
+            return details
+        def create_costs_dataframe(self, costs):
+            """
+            Parameters
+            ----------
+            costs : list[dict]
+                The list of dictionaries of costs.
+            Returns
+            -------
+            pd.DataFrame
+                A dataframe to be written as a .csv
+            """
+            new_rows = []
+            for row in costs:
+                new_row = {
+                    "Project ID with serial": row["project_id_with_serial"],
+                    "Number of turbines": row["num_turbines"],
+                    "Turbine rating MW": row["turbine_rating_MW"],
+                    "Rotor diameter m": row["rotor_diameter_m"],
+                    "Module": row["module"],
+                    "Type of cost": row["type_of_cost"],
+                    "Cost per turbine": row["cost_per_turbine"],
+                    "Cost per project": row["cost_per_project"],
+                    "Cost per kW": row["usd_per_kw_per_project"],
+                }
+                new_rows.append(new_row)
+            costs_df = pd.DataFrame(new_rows)
+            return costs_df
+        def _is_numeric(self, value):
+            """
+            This method tests if a value is a numeric (that is, can be parsed
+            by float()) or non numeric (which cannot be parsed).
+            The decision from this method determines whether values go into
+            the numeric or non-numeric columns.
+            Parameters
+            ----------
+            value
+                The value to be tested.
+            Returns
+            -------
+            bool
+                True if the value is numeric, False otherwise.
+            """
+            try:
+                float(value)
+            except ValueError:
+                return False
+            return True

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Omdao #165

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!