Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TASK] HPC Inferfaces #1415

Closed
5 of 10 tasks
PaulTalbot-INL opened this issue Jan 19, 2021 · 1 comment
Closed
5 of 10 tasks

[TASK] HPC Inferfaces #1415

PaulTalbot-INL opened this issue Jan 19, 2021 · 1 comment
Assignees
@alfoa
Labels
archive To archive the feature requests or PRs when there are no further developments priority_minor task This tag should be used for any new capability, improvement or enanchment

Comments

@PaulTalbot-INL
Copy link
Collaborator

Issue Description

Currently RAVEN is defaulted/hard-wired to run on a particular HPC machine (currently FALCON at INL). An interface structure for other HPCs would make it more portable and modular.

For Change Control Board: Issue Review

This review should occur before any development is performed as a response to this issue.

  • 1. Is it tagged with a type: defect or task?
  • 2. Is it tagged with a priority: critical, normal or minor?
  • 3. If it will impact requirements or requirements tests, is it tagged with requirements?
  • 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
  • 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure

This review should occur when the issue is imminently going to be closed.

  • 1. If the issue is a defect, is the defect fixed?
  • 2. If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
  • 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
  • 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
  • 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?
@PaulTalbot-INL PaulTalbot-INL added priority_minor task This tag should be used for any new capability, improvement or enanchment labels Jan 19, 2021
@PaulTalbot-INL PaulTalbot-INL mentioned this issue Jan 19, 2021
Merged
20 tasks
alfoa added a commit that referenced this issue Jan 25, 2021
@PaulTalbot-INL @dylanjm
@alfoa @joshua-cogliati-inl
2020-12 HERON milestone changes (#1409)
8ad7ca8 
Closes #1082 Closes #368 Closes #1212 (detecting RAVEN code interface failures)
Closes #1413 (memory debugging tool)
Closes #1414 (expected meta keys from SVL)
Closes #1416 (maxCycles in Interpolated ROMCollection)
Closes #1417 (grad point perturbation distance scaling)
Closes #1418 (nyquist length adjusting for ARMA in ROMCollection)
Introduces and partially addresses #1412 (gradient points can sample outside constraints)
Introduces and partially addresses #1415 (HPC interfaces)
What are the significant changes in functionality due to this change request?
Major Changes

    Allows CodeInterfaces to set whether they print failed runs to screen or not via CodeInterface.printFailedRuns, which defaults to True
    Added "successful run" file creation for RAVEN for failure checking in RAVEN CodeInterface; this replace the "read the last 500 lines of screen output" approach that was used previously
    Memory profiler decorator added for RAVEN run debugging
    Extension of SupervisedLearning to allow "expected meta keys" to be declared
    Constraint handling added to GradientApproximation methods within GradientDescent optimization, assuring samples points are not violating boundary, explicit, and implicit constraints. This has been placed in the FiniteDifference class, and may need reworking or inclusion in other sibling classes to be suitably general.
    Sawtooth qsub command script added. This needs to become a sort of "HPC interface" tool instead of an ad-hoc script manipulation.
    Debugging utility in utils.Debugging added to allow analyzing the size of Python objects.

Minor Changes

    Disables screen printing of failed runs for RAVEN code interface, for clarity (nested inner RAVEN failed runs within the outer RAVEN screen output is very difficult to follow)
    Minor adjustments for RAVEN code interface to set up head and remote nodes for use with RAY
    Many tweaks to JobHandler for working with RAY
    Improved traceback reporting for failures in EnsembleModel configuration, allowing for additional clarity on model failures (especially ExternalModels)
    maxCycles added as option to ROMCollection objects to limit the number of generated "years", mostly for debugging purposes
    Improved modularity for boundary checking in the RavenSampled optimizer
    New gradDistanceScalar option for GradientApproximater class allowing fine control over how far a gradient evaluation point should be from an optimal point.
    initialStepScale added to StepManipulator children to allow user manipulation of the initial step size in relative terms.
    nyquistScalar added to ARMA to allow finite control over which Fourier bases are included in the segments and which are considered global
    Added an error when segmenting a ROMCollection results in unequal segments. This needs some design considerations to decide how to handle successfully.
    Retention of all ROMCollection.Clustered segment ROMs has been disabled, for performance considerations; this also disables random sampling from the clusters to generate new signals. The right solution is probably to generate distributions of characteristics that can then be sampled from to generate representative ROMs on demand.
    CashFlow updated.

Tests Added

    Test for gradient descent optimization attempting to sample outside of boundary/explicit/implicit constraints
    ARMA test added for Interpolated ROMCollection sampling between multiple setpoint years.


Co-authored-by: Dylan McDowell <dylanjm@users.noreply.github.com>
Co-authored-by: alfoa <andrea.alfonsi@inl.gov>
Co-authored-by: Joshua J. Cogliati <joshua.cogliati@inl.gov>
@wangcj05 wangcj05 added the archive To archive the feature requests or PRs when there are no further developments label Feb 16, 2024
@wangcj05
Copy link
Collaborator

archived.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alfoa alfoa

Labels
archive To archive the feature requests or PRs when there are no further developments priority_minor task This tag should be used for any new capability, improvement or enanchment
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@PaulTalbot-INL @wangcj05 @alfoa