Mandd/mutator changes #1968

mandd · 2022-09-15T20:29:11Z

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

Close #1953
close #1959
close #1976

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added is the the analytic documentation updated/added?
9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

…rt of the repair duplicated chromosomes in GA

into mandd/mutatorChanges

…cases

[WIP] adding changes to bitFlipMutator and randomMutator

Mutators tests updated

moosebuild · 2022-11-17T17:11:19Z

Job Mingw Test on b005a9b : invalidated by @mandd

mandd

@wangcj05 : this PR is ready to be reviewed

wangcj05 · 2022-11-21T15:48:56Z

@mandd The following test failed, I guess you need to re-gold the outputs:

FAILED:
Diff tests/framework/Optimizers/GeneticAlgorithms/GAwithEnsembleModel

wangcj05 · 2022-11-21T15:49:38Z

ravenframework/Models/EnsembleModel.py

+    '''
+    if output.name not in optionalOutputNames:
+      if output.name not in targetEvaluationNames.keys():
+        if 'batchMode' not in joinedResponse.keys():
+          output.addRealization(joinedResponse)
+        else:
+          inputVarValues = joinedResponse['batchInfo'][0]['batchRealizations'][0]['SampledVars']
+          for key in inputVarValues.keys():
+            inputVarValues[key] = np.array([inputVarValues[key]])
+          metaValues = joinedResponse['batchInfo'][0]['batchRealizations'][0]
+          for key in metaValues.keys():
+            metaValues[key] = np.array([metaValues[key]])
+          outputData = joinedResponse
+          batchData = {**inputVarValues, **metaValues, **outputData}
+          output.addRealization(batchData)
+      else:
+        output.addRealization(outcomes[targetEvaluationNames[output.name]]['response'])
+    else:
+      # collect optional output if present and not already collected
+      output.addRealization(optionalOutputs[optionalOutputNames[output.name]])
+    '''


Remove the commented lines.

mandd · 2022-11-21T18:21:23Z

@mandd The following test failed, I guess you need to re-gold the outputs:

FAILED: Diff tests/framework/Optimizers/GeneticAlgorithms/GAwithEnsembleModel

Test has been regolded

moosebuild · 2022-11-21T20:44:15Z

Job Test CentOS 8 on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:53:53Z

Job Test Ubuntu 20-2 Optional on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:54:00Z

Job Test Ubuntu 18-2 Python 3 on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:54:07Z

Job Test Fedora 31 on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:54:13Z

Job Test Fedora 32 on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:54:19Z

Job Test Ubuntu 16 on 2b3d0db : invalidated by @joshua-cogliati-inl

mystery failure

wangcj05

@mandd I have several comments for you to consider.

wangcj05 · 2022-11-21T22:50:11Z

ravenframework/Models/EnsembleModel.py

@@ -476,6 +476,7 @@ def collectOutput(self,finishedJob,output):
      if output.name not in targetEvaluationNames.keys():
        # in the event a batch is run, the evaluations will be a dict as {'RAVEN_isBatch':True, 'realizations': [...]}
        if isinstance(evaluation,dict) and evaluation.get('RAVEN_isBatch',False):
+


no need to have an empty space here.

wangcj05 · 2022-11-21T22:55:07Z

ravenframework/Optimizers/mutators/mutators.py

+  if kwargs['locs'] == None:
+    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
+    locs.sort()
+  else:
+    locs = [kwargs['locs'][0], kwargs['locs'][1]]
+    locs.sort()


Similar lines have been repeated three times in this file, I would to recommend to create a function to perform this.

function locationsGenerator has been added

wangcj05 · 2022-11-21T22:55:49Z

ravenframework/Optimizers/mutators/mutators.py

  if kwargs['locs'] == None:
    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
-    loc1 = locs[0]
-    loc2 = locs[1]
+    loc1 = np.minimum(locs[0], locs[1])
+    loc2 = np.maximum(locs[0], locs[1])
  else:
-    loc1 = kwargs['locs'][0]
-    loc2 = kwargs['locs'][1]
+    loc1 = np.minimum(kwargs['locs'][0], kwargs['locs'][1])
+    loc2 = np.maximum(kwargs['locs'][0], kwargs['locs'][1])


These have been repeated several times.

fixed with new method

wangcj05 · 2022-11-21T22:56:40Z

ravenframework/Optimizers/mutators/mutators.py

+    This method is designed to randomly mutate a single gene in each chromosome with probability = mutationProb.
+    @ In, offSprings, xr.DataArray, children resulting from the crossover process
+    @ In, kwargs, dict, dictionary of parameters for this mutation method:
+          mutationProb, float, probability that governs the mutation process, i.e., if prob < random number, then the mutation will occur


docstring for "distDict"

wangcj05 · 2022-11-21T22:57:42Z

ravenframework/Optimizers/mutators/mutators.py

+  if kwargs['locs'] == None:
+    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
+    locL = np.minimum(locs[0], locs[1])
+    locU = np.maximum(locs[0], locs[1])
+  else:
+    locL = np.minimum(kwargs['locs'][0], kwargs['locs'][1])
+    locU = np.maximum(kwargs['locs'][0], kwargs['locs'][1])


These lines have been repeated several times.

new function has been created

wangcj05 · 2022-11-21T22:58:32Z

ravenframework/Optimizers/mutators/mutators.py

@@ -186,6 +198,7 @@ def inversionMutator(offSprings, distDict, **kwargs):
 __mutators['scrambleMutator']   = scrambleMutator
 __mutators['bitFlipMutator']    = bitFlipMutator
 __mutators['inversionMutator']  = inversionMutator
+__mutators['randomMutator']     = randomMutator


Please update the manual for the randomMutator

good catch, added

wangcj05 · 2022-11-21T23:04:49Z

tests/framework/unit_tests/Optimizers/testSwapMutator.py

+def checkSameDataArrays(comment, resultedDA, expectedDA, update=True):
+  """
+    This method compares two identical things
+    @ In, comment, string, a comment printed out if it fails
+    @ In, resultedDA, xr.DataArray, the resulted DataArray to be tested
+    @ In, expectedDA, xr.DataArray, the expected DataArray
+    @ In, update, bool, optional, if False then don't update results counter
+    @ Out, res, bool, True if same
+  """
+  res = resultedDA.identical(expectedDA)
+  if update:
+    if res:
+      results["pass"] += 1
+    else:
+      print("checking string", comment, '|', resultedDA, "!=", expectedDA)
+      results["fail"] += 1
+  return res
+
+results = {'pass': 0, 'fail': 0}
+
+def createElement(tag,attrib={},text={}):
+  """
+    Method to create a dummy xml element readable by the distribution classes
+    @ In, tag, string, the node tag
+    @ In, attrib, dict, optional, the attribute of the xml node
+    @ In, text, dict, optional, the dict containing what should be in the xml text
+  """
+  element = ET.Element(tag,attrib)
+  element.text = text
+  return element


Not related to this PR. I think we should start to standardize our unit tests. The first thing to do is to create utils functions for unit tests. For examples, the above functions have been existed in a lot of unit test files.

I agree, it looks like it's a Wednesday morning topics

mandd

Addressing Wang's comments

mandd · 2022-11-22T16:35:36Z

ravenframework/Models/EnsembleModel.py

@@ -476,6 +476,7 @@ def collectOutput(self,finishedJob,output):
      if output.name not in targetEvaluationNames.keys():
        # in the event a batch is run, the evaluations will be a dict as {'RAVEN_isBatch':True, 'realizations': [...]}
        if isinstance(evaluation,dict) and evaluation.get('RAVEN_isBatch',False):
+


mandd · 2022-11-22T16:48:52Z

ravenframework/Optimizers/mutators/mutators.py

+    This method is designed to randomly mutate a single gene in each chromosome with probability = mutationProb.
+    @ In, offSprings, xr.DataArray, children resulting from the crossover process
+    @ In, kwargs, dict, dictionary of parameters for this mutation method:
+          mutationProb, float, probability that governs the mutation process, i.e., if prob < random number, then the mutation will occur


mandd · 2022-11-22T17:48:04Z

tests/framework/unit_tests/Optimizers/testSwapMutator.py

+def checkSameDataArrays(comment, resultedDA, expectedDA, update=True):
+  """
+    This method compares two identical things
+    @ In, comment, string, a comment printed out if it fails
+    @ In, resultedDA, xr.DataArray, the resulted DataArray to be tested
+    @ In, expectedDA, xr.DataArray, the expected DataArray
+    @ In, update, bool, optional, if False then don't update results counter
+    @ Out, res, bool, True if same
+  """
+  res = resultedDA.identical(expectedDA)
+  if update:
+    if res:
+      results["pass"] += 1
+    else:
+      print("checking string", comment, '|', resultedDA, "!=", expectedDA)
+      results["fail"] += 1
+  return res
+
+results = {'pass': 0, 'fail': 0}
+
+def createElement(tag,attrib={},text={}):
+  """
+    Method to create a dummy xml element readable by the distribution classes
+    @ In, tag, string, the node tag
+    @ In, attrib, dict, optional, the attribute of the xml node
+    @ In, text, dict, optional, the dict containing what should be in the xml text
+  """
+  element = ET.Element(tag,attrib)
+  element.text = text
+  return element


I agree, it looks like it's a Wednesday morning topics

mandd · 2022-11-22T18:01:17Z

ravenframework/Optimizers/mutators/mutators.py

@@ -186,6 +198,7 @@ def inversionMutator(offSprings, distDict, **kwargs):
 __mutators['scrambleMutator']   = scrambleMutator
 __mutators['bitFlipMutator']    = bitFlipMutator
 __mutators['inversionMutator']  = inversionMutator
+__mutators['randomMutator']     = randomMutator


good catch, added

mandd · 2022-11-22T18:14:18Z

ravenframework/Optimizers/mutators/mutators.py

+  if kwargs['locs'] == None:
+    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
+    locL = np.minimum(locs[0], locs[1])
+    locU = np.maximum(locs[0], locs[1])
+  else:
+    locL = np.minimum(kwargs['locs'][0], kwargs['locs'][1])
+    locU = np.maximum(kwargs['locs'][0], kwargs['locs'][1])


new function has been created

mandd · 2022-11-22T18:19:10Z

ravenframework/Optimizers/mutators/mutators.py

+  if kwargs['locs'] == None:
+    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
+    locs.sort()
+  else:
+    locs = [kwargs['locs'][0], kwargs['locs'][1]]
+    locs.sort()


function locationsGenerator has been added

mandd · 2022-11-22T18:21:29Z

ravenframework/Optimizers/mutators/mutators.py

  if kwargs['locs'] == None:
    locs = list(set(randomUtils.randomChoice(list(np.arange(offSprings.data.shape[1])),size=2,replace=False)))
-    loc1 = locs[0]
-    loc2 = locs[1]
+    loc1 = np.minimum(locs[0], locs[1])
+    loc2 = np.maximum(locs[0], locs[1])
  else:
-    loc1 = kwargs['locs'][0]
-    loc2 = kwargs['locs'][1]
+    loc1 = np.minimum(kwargs['locs'][0], kwargs['locs'][1])
+    loc2 = np.maximum(kwargs['locs'][0], kwargs['locs'][1])


fixed with new method

wangcj05

changes look good.

moosebuild · 2022-11-22T20:20:27Z

Job Test qsubs sawtooth on cf931e9 : invalidated by @mandd

wangcj05 · 2022-11-23T16:32:32Z

Test are green, checklist is good. PR can be merged.

mandd and others added 30 commits July 26, 2022 17:26

fixes for parallel coordinate plot

bca3a82

added test

7add05e

added check for grid sampler

debdea3

fixed test

de819cb

delete whitespaces

0a511d3

addressing comments

22c8019

edits

41ef939

submit method edits

54aa4be

plot edits

b2c33ef

edits

310e792

Merge branch 'devel' into mandd/mutatorChanges

baa4f9a

added registration

9cb8092

raise errors

b7daef1

final edits

320125f

Adding changes to random mutators and bitflip and trying to remove pa…

9078d2a

…rt of the repair duplicated chromosomes in GA

Merge branch 'mandd/mutatorChanges' of https://github.com/idaholab/raven

7f6a0ae

into mandd/mutatorChanges

this error msg does not allow to use the random mutator for discrete …

2624661

…cases

creating BitFlip Mutator unitTest

fa3380b

Regolded files

74e9307

Merge branch 'mandd/mutatorChanges' into mutatorChanges

56639ee

test

86504f4

edits

d7567a6

Merge branch 'idaholab:devel' into mutatorChanges

62d7bc3

removing commented lines

eb8961a

Merge pull request #1977 from JunyungKim/mutatorChanges

ff98f39

[WIP] adding changes to bitFlipMutator and randomMutator

merge mutator file

7325db3

Merge remote-tracking branch 'origin' into mandd/mutatorChanges

e617e33

Merge branch 'mandd/mutatorChangesNew' into mandd/mutatorChanges

ab33b07

adding unit tests and inversion mustators fixed to test branch

f7536e0

This is about mutators and unit tests of GA.

aac2d6b

Merge pull request #1995 from JunyungKim/mutators_tests_updated

b005a9b

Mutators tests updated

Jimmy-INL mentioned this pull request Nov 15, 2022

Junyung/mutator changes #1993

Closed

9 tasks

Jimmy-INL changed the title ~~[WIP] Mandd/mutator changes~~ Mandd/mutator changes Nov 15, 2022

Jimmy-INL requested review from aalfonsi and wangcj05 November 17, 2022 20:07

mandd added 2 commits November 18, 2022 11:54

edits

93ca3f3

merge with devel

47bf4ea

mandd commented Nov 19, 2022

View reviewed changes

cleaned file

4c6307a

wangcj05 reviewed Nov 21, 2022

View reviewed changes

mandd added 2 commits November 21, 2022 11:20

addressing comments

a82f354

delete whitespaces

2b3d0db

wangcj05 requested changes Nov 21, 2022

View reviewed changes

mandd commented Nov 22, 2022

View reviewed changes

mandd added 2 commits November 22, 2022 11:22

addressing comments

0758103

fixed typo

cf931e9

wangcj05 approved these changes Nov 22, 2022

View reviewed changes

wangcj05 merged commit 668e9b8 into devel Nov 23, 2022

wangcj05 deleted the mandd/mutatorChanges branch November 23, 2022 16:32

Mandd/mutator changes #1968

Mandd/mutator changes #1968

Conversation

mandd commented Sep 15, 2022 • edited by wangcj05 Loading

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

moosebuild commented Nov 17, 2022

mandd left a comment

Choose a reason for hiding this comment

wangcj05 commented Nov 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mandd commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

wangcj05 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mandd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangcj05 left a comment

Choose a reason for hiding this comment

moosebuild commented Nov 22, 2022

wangcj05 commented Nov 23, 2022

mandd commented Sep 15, 2022 •

edited by wangcj05

Loading