Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Xarray DataObjects #183

Closed
wants to merge 35 commits into from

Conversation

PaulTalbot-INL
Copy link
Collaborator


Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

First step in addressing #182.

What are the significant changes in functionality due to this change request?

This merge request wraps the powerful XArray.Dataset high-density field data container with a caching system to alleviate the cost of "appending" to the data object.

To do this, the new libraries xarray and pandas are added as RAVEN dependencies, which leads to creating a script for the moose_build regression test recipe that will set up the appropriate RAVEN conda environments, rather than force the existing conda package to expand.

Lastly, a small change is added that will allow conda to exist both in a home directory as well as /opt.


For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

  • 1. Review all computer code.
  • 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
  • 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
  • 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
  • 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
  • 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
  • 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
  • 8. If an analytic test is changed/added is the the analytic documentation updated/added?

PaulTalbot-INL and others added 27 commits April 6, 2017 11:16
This can be used if the user does not have access to change /opt
@PaulTalbot-INL PaulTalbot-INL changed the title [WIP]: Cached xarray datasets Cached xarray datasets May 18, 2017
@moosebuild
Copy link

Job Mingw Test:win-mingw on b1afabc : invalidated by @bobkinl

Windows test environment adjusted to use updated build and xarray added.

@PaulTalbot-INL
Copy link
Collaborator Author

It was pointed out, we should wait on this merge request until the DataObject is close to needing it, since it forces the users to update their libraries.

@moosebuild
Copy link

Job Mingw Test:win-mingw on b1afabc : invalidated by @bobkinl

Giving the switch to new Windows config for testing another try.

@moosebuild
Copy link

Job Mingw Test:win-mingw on b1afabc : invalidated by @PaulTalbot-INL

New Windows install procedure

@moosebuild
Copy link

moosebuild commented May 22, 2017

Job Mingw Test:win-mingw on b1afabc : invalidated by @bobkinl

New build environment worked, but xarray not installed yet. It is now.

test comment edit.

@maljovec
Copy link
Contributor

maljovec commented Aug 31, 2017

Test checklist:

  • A
  • B
  • C

@PaulTalbot-INL PaulTalbot-INL changed the title Cached xarray datasets WIP: Xarray DataObjects Oct 20, 2017
@PaulTalbot-INL
Copy link
Collaborator Author

We determined that cached XArray was not sufficiently fast for our needs, and instead will use one structure for collecting data, then convert that data into an XArray. This is no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants