Framework as package #1784

joshua-cogliati-inl · 2022-03-07T21:23:35Z

Pull Request Description

What issue does this change request address?

What are the significant changes in functionality due to this change request?

This changes framework to a package. This allows creating a pip package that just has a framework directory in the root directory, instead of multiple directories and files.

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added is the the analytic documentation updated/added?
9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

moosebuild · 2022-03-07T22:32:34Z

Job Test qsubs sawtooth on c104753 : invalidated by @joshua-cogliati-inl

Failed tests/cluster_tests/InternalParallel/test_hybrid_model_code

moosebuild · 2022-03-07T22:40:47Z

Job Mingw Test on c104753 : invalidated by @joshua-cogliati-inl

failed in Set Python Environment

moosebuild · 2022-03-08T15:30:57Z

Job Test qsubs sawtooth on a3f36b8 : invalidated by @joshua-cogliati-inl

Timeout failures.

moosebuild · 2022-03-15T21:37:28Z

Job Test qsubs sawtooth on 366ae3b : invalidated by @joshua-cogliati-inl

Failed tests/cluster_tests/InternalParallel/test_hybrid_model_code

joshua-cogliati-inl · 2022-03-16T15:14:30Z

Suggested email:

In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework.

This will require changes in three things.

Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned.
Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead.
Pickle files have the package information embedded inside, so they will need to be regenerated.

This is pull request: #1784

The following are example modifications for framework as a package in the source code.

Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth.

Example for something that is in a directory inside of ravenframework:

Original:

import Files

New:

from .. import Files

Original:

from utils import mathUtils

New:

from ..utils import mathUtils

Original:

import utils.TreeStructure

New:

from ..utils import TreeStructure

(and modify all uses of utils.TreeStructure to TreeStructure)

Original:

import utils.TreeStructure as TS

New:

from ..utils import TreeStructure as TS

Ravenframework imports:

Original:

import Files

New:

from ravenframework import Files

Original:

from utils import mathUtils

New:

from ravenframework.utils import mathUtils

Original:

import utils.TreeStructure

New:

from ravenframework.utils import TreeStructure

(and modify all uses of utils.TreeStructure to TreeStructure)

Original:

import utils.TreeStructure as TS

New:

from ravenframework.utils import TreeStructure as TS

Importlib (Example from ravenframework/Models/Code.py )

Original:

importlib.import_module("CodeInterfaces")

New:

importlib.import_module("..CodeInterfaces", "ravenframework.Models")

(Note the second parameter package anchor for finding relative packages.)

Driver.py can no longer be used from the python executable.

Original:

python framework/Driver.py "${ARGS[@]}"

New:

python raven_framework.py "${ARGS[@]}"

or

raven_framework "${ARGS[@]}"

joshua-cogliati-inl · 2022-03-16T15:19:34Z

Before this is merged, we should send an email warning people about the upcoming change.

PaulTalbot-INL · 2022-03-16T15:25:29Z

I see there's a TEAL update in there; that's intentional? Just checking for completeness.

ravenframework/InputTemplates/TemplateBaseClass.py

joshua-cogliati-inl · 2022-03-16T15:58:01Z

I see there's a TEAL update in there; that's intentional? Just checking for completeness.

No, that should wait until after this is merged.

PaulTalbot-INL

I recommend we remove the TEAL update and the two commented lines; otherwise the changes pass code check.

PaulTalbot-INL · 2022-03-16T16:03:42Z

I think this PR will need @wangcj05 control board approval.

PaulTalbot-INL · 2022-03-16T16:07:17Z

I don't think these changes will impact users beyond the email Joshua prepared. However, I think this significantly impacts our approaches as developers, so I wonder if a Monday or Wednesday meeting should cover what the impact of these changes are on:

Installing RAVEN (and plugins)
1. Installation via pip
2. Installing and running tests (separate?)
Pathing within RAVEN (and plugins)
? Anything else I missed

wangcj05

@joshua-cogliati-inl I have a comment about the use of "ravenframework.requestedModules" or "...." to import the modules. In addition, I think the user manual need to be updated:

the places using Driver.py to rrun the input files.
the code snippets that are using "import" to import raven modules

In addition, the user guide may also need some update.

wangcj05 · 2022-03-17T15:35:19Z

ravenframework/Metrics/metrics/DSS.py

@@ -23,7 +23,7 @@

 #Internal Modules------------------------------------------------------------------------------------
 from .MetricInterface import MetricInterface
-from utils import InputData, InputTypes
+from ...utils import InputData, InputTypes


@joshua-cogliati-inl which is a better way to import the module, "ravenframework.utils" or "...utils"? I think I prefer to using "ravenframework.utils"

I agree ravenframework.utils looks nicer, but ...utils paths correctly internal to the package as a library. I'm not sure of all the reasons why, but I think importing a package as though it was outside itself it a bad practice.

Hm, well, one reason I used ...utils was so I didn't have to change it when framework got renamed. Right now ravenframework.whatever is a good warning that that section of code has something strange happening to the python path (like in CodeInterfaces or in CustomModes). For this pull request I recommend keeping the relative paths, but I have no strong opinions on this for future additions.

wangcj05 · 2022-03-17T16:01:00Z

I don't think these changes will impact users beyond the email Joshua prepared. However, I think this significantly impacts our approaches as developers, so I wonder if a Monday or Wednesday meeting should cover what the impact of these changes are on:

Installing RAVEN (and plugins)

Installation via pip

Installing and running tests (separate?)

Pathing within RAVEN (and plugins)

? Anything else I missed

I agree with @PaulTalbot-INL, I think next Monday is good for the discussion. @joshua-cogliati-inl Can you prepare some slides for next Monday discussion?

joshua-cogliati-inl · 2022-03-17T21:57:34Z

I agree with @PaulTalbot-INL, I think next Monday is good for the discussion. @joshua-cogliati-inl Can you prepare some slides for next Monday discussion?

Okay, I can make some slides for next Monday's discussion.

P.S. I will look thru the manual for imports and use of Driver.

joshua-cogliati-inl · 2022-03-23T20:55:33Z

I updated the user manual fixing places that used import and Driver.py.

moosebuild · 2022-03-24T14:30:58Z

Job Test qsubs sawtooth on 6d79372 : invalidated by @joshua-cogliati-inl

Timeout tests/framework/CodeInterfaceTests/RAVEN/Code Timeout tests/framework/CodeInterfaceTests/RAVEN/ROM Timeout tests/framework/CodeInterfaceTests/RAVEN/ReturnDatabase Diff tests/cluster_tests/RavenRunsRaven/Code Diff tests/cluster_tests/test_mpiqsub_parameters Diff tests/cluster_tests/AdaptiveSobol/test_parallel_adaptive_sobol

moosebuild · 2022-03-24T14:31:09Z

Job Test qsubs falcon on 6d79372 : invalidated by @joshua-cogliati-inl

Timeout tests/framework/CodeInterfaceTests/RAVEN/Code Timeout tests/framework/CodeInterfaceTests/RAVEN/ROM Timeout tests/framework/CodeInterfaceTests/RAVEN/ReturnDatabase Diff tests/cluster_tests/RavenRunsRaven/Code Diff tests/cluster_tests/test_mpiqsub_parameters Diff tests/cluster_tests/AdaptiveSobol/test_parallel_adaptive_sobol

wangcj05

PR Looks good.

I think comments from the review have been resolved.

wangcj05 · 2022-03-31T16:56:51Z

PR is good and checklist is satisfied.

wangcj05 · 2022-03-31T16:57:56Z

@joshua-cogliati-inl Could you send out an email to the users and plugin developers (LOGOS, TEAL, HERON, FARM, etc.)?

wangcj05 · 2022-05-12T18:09:12Z

Hi @wanghy-anl, this PR causes the failure of FARM on RAVEN regression test machines. Basically, we changed the way to import raven modules. Do you have plan to update the module import in FARM? The following is the suggestion how to change the import from Josh:

Suggested email:

In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework.

This will require changes in three things.

Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned.
Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead.
Pickle files have the package information embedded inside, so they will need to be regenerated.
This is pull request: https://github.com/idaholab/raven/pull/1784

The following are example modifications for framework as a package in the source code.

Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth.

Example for something that is in a directory inside of ravenframework:

Original:

import Files

New:

from .. import Files

Original:

from utils import mathUtils

New:

from ..utils import mathUtils

Original:

import utils.TreeStructure

New:

from ..utils import TreeStructure

(and modify all uses of utils.TreeStructure to TreeStructure)

Original:

import utils.TreeStructure as TS

New:

from ..utils import TreeStructure as TS

Ravenframework imports:

Original:

import Files

New:

from ravenframework import Files

Original:

from utils import mathUtils

New:

from ravenframework.utils import mathUtils

Original:

import utils.TreeStructure

New:

from ravenframework.utils import TreeStructure

(and modify all uses of utils.TreeStructure to TreeStructure)

Original:

import utils.TreeStructure as TS

New:

from ravenframework.utils import TreeStructure as TS

Importlib (Example from ravenframework/Models/Code.py )

Original:

importlib.import_module("CodeInterfaces")

New:

importlib.import_module("..CodeInterfaces", "ravenframework.Models")

(Note the second parameter package anchor for finding relative packages.)

Driver.py can no longer be used from the python executable.

Original:

python framework/Driver.py "${ARGS[@]}"

New:

python raven_framework.py "${ARGS[@]}"

or

raven_framework "${ARGS[@]}"

wanghy-anl · 2022-05-12T18:11:52Z

Hello Congjian, Thank you for letting me know. I can fix it. Paul gave me the instructions on how to update the submodule versions a moment ago. I will get it done no later than next Monday. Thanks, Haoyu

…

On Thu, May 12, 2022 at 1:09 PM Congjian Wang - INL < ***@***.***> wrote: Hi @wanghy-anl <https://github.com/wanghy-anl>, this PR causes the failure of FARM on RAVEN regression test machines. Basically, we changed the way to import raven modules. Do you have plan to update the module import in FARM? The following is the suggestion how to change the import from Josh: Suggested email: In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework. This will require changes in three things. Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned. Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead. Pickle files have the package information embedded inside, so they will need to be regenerated. This is pull request: #1784 The following are example modifications for framework as a package in the source code. Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth. Example for something that is in a directory inside of ravenframework: Original: import Files New: from .. import Files Original: from utils import mathUtils New: from ..utils import mathUtils Original: import utils.TreeStructure New: from ..utils import TreeStructure (and modify all uses of utils.TreeStructure to TreeStructure) Original: import utils.TreeStructure as TS New: from ..utils import TreeStructure as TS Ravenframework imports: Original: import Files New: from ravenframework import Files Original: from utils import mathUtils New: from ravenframework.utils import mathUtils Original: import utils.TreeStructure New: from ravenframework.utils import TreeStructure (and modify all uses of utils.TreeStructure to TreeStructure) Original: import utils.TreeStructure as TS New: from ravenframework.utils import TreeStructure as TS Importlib (Example from ravenframework/Models/Code.py ) Original: importlib.import_module("CodeInterfaces") New: importlib.import_module("..CodeInterfaces", "ravenframework.Models") (Note the second parameter package anchor for finding relative packages.) Driver.py can no longer be used from the python executable. Original: python framework/Driver.py "${ARGS[@]}" New: python raven_framework.py "${ARGS[@]}" or raven_framework "${ARGS[@]}" — Reply to this email directly, view it on GitHub <#1784 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APD4NWKD35KGR4VRUOXKZODVJVCNJANCNFSM5QEPTSVQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

wangcj05 · 2022-05-12T18:39:04Z

Wonderful! Thanks a lot. Congjian

________________________________ From: Haoyu Wang ***@***.***> Sent: Thursday, May 12, 2022 12:12 PM To: idaholab/raven ***@***.***> Cc: Congjian Wang ***@***.***>; State change ***@***.***> Subject: [EXTERNAL] Re: [idaholab/raven] Framework as package (PR #1784) Hello Congjian, Thank you for letting me know. I can fix it. Paul gave me the instructions on how to update the submodule versions a moment ago. I will get it done no later than next Monday. Thanks, Haoyu

On Thu, May 12, 2022 at 1:09 PM Congjian Wang - INL < ***@***.***> wrote: Hi @wanghy-anl <https://github.com/wanghy-anl>, this PR causes the failure of FARM on RAVEN regression test machines. Basically, we changed the way to import raven modules. Do you have plan to update the module import in FARM? The following is the suggestion how to change the import from Josh: Suggested email: In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework. This will require changes in three things. Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned. Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead. Pickle files have the package information embedded inside, so they will need to be regenerated. This is pull request: #1784 The following are example modifications for framework as a package in the source code. Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth. Example for something that is in a directory inside of ravenframework: Original: import Files New: from .. import Files Original: from utils import mathUtils New: from ..utils import mathUtils Original: import utils.TreeStructure New: from ..utils import TreeStructure (and modify all uses of utils.TreeStructure to TreeStructure) Original: import utils.TreeStructure as TS New: from ..utils import TreeStructure as TS Ravenframework imports: Original: import Files New: from ravenframework import Files Original: from utils import mathUtils New: from ravenframework.utils import mathUtils Original: import utils.TreeStructure New: from ravenframework.utils import TreeStructure (and modify all uses of utils.TreeStructure to TreeStructure) Original: import utils.TreeStructure as TS New: from ravenframework.utils import TreeStructure as TS Importlib (Example from ravenframework/Models/Code.py ) Original: importlib.import_module("CodeInterfaces") New: importlib.import_module("..CodeInterfaces", "ravenframework.Models") (Note the second parameter package anchor for finding relative packages.) Driver.py can no longer be used from the python executable. Original: python framework/Driver.py "${ARGS[@]}" New: python raven_framework.py "${ARGS[@]}" or raven_framework "${ARGS[@]}" — Reply to this email directly, view it on GitHub <#1784 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APD4NWKD35KGR4VRUOXKZODVJVCNJANCNFSM5QEPTSVQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fidaholab%2Fraven%2Fpull%2F1784%23issuecomment-1125286090&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cXNZa3ix25uY2ZRsXo7rMXsh8ifKl9EmEA2U%2B3CRL1w%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABX3L36O2ZTMRPYX637PXEDVJVCXHANCNFSM5QEPTSVQ&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KDKcPY0zKDXNADA78gJmRDZS%2BhIto836rT8wh2WgWlM%3D&reserved=0>. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

wanghy-anl · 2022-05-13T03:28:18Z

Hello Congjian, I opened a PR #1826 called "Farm submodule update". Please have a check and let me know if more changes are needed. Thanks, Haoyu On Thu, May 12, 2022 at 1:39 PM Congjian Wang - INL < ***@***.***> wrote:

…

Wonderful! Thanks a lot. Congjian ________________________________ From: Haoyu Wang ***@***.***> Sent: Thursday, May 12, 2022 12:12 PM To: idaholab/raven ***@***.***> Cc: Congjian Wang ***@***.***>; State change ***@***.***> Subject: [EXTERNAL] Re: [idaholab/raven] Framework as package (PR #1784) Hello Congjian, Thank you for letting me know. I can fix it. Paul gave me the instructions on how to update the submodule versions a moment ago. I will get it done no later than next Monday. Thanks, Haoyu On Thu, May 12, 2022 at 1:09 PM Congjian Wang - INL < ***@***.***> wrote: > Hi @wanghy-anl <https://github.com/wanghy-anl>, this PR causes the > failure of FARM on RAVEN regression test machines. Basically, we changed > the way to import raven modules. Do you have plan to update the module > import in FARM? The following is the suggestion how to change the import > from Josh: > > Suggested email: > > In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework. > > This will require changes in three things. > > Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned. > Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead. > Pickle files have the package information embedded inside, so they will need to be regenerated. > This is pull request: #1784 > > The following are example modifications for framework as a package in the source code. > > Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth. > > Example for something that is in a directory inside of ravenframework: > > Original: > > import Files > > New: > > from .. import Files > > Original: > > from utils import mathUtils > > New: > > from ..utils import mathUtils > > Original: > > import utils.TreeStructure > > New: > > from ..utils import TreeStructure > > (and modify all uses of utils.TreeStructure to TreeStructure) > > Original: > > import utils.TreeStructure as TS > > New: > > from ..utils import TreeStructure as TS > > Ravenframework imports: > > Original: > > import Files > > New: > > from ravenframework import Files > > Original: > > from utils import mathUtils > > New: > > from ravenframework.utils import mathUtils > > Original: > > import utils.TreeStructure > > New: > > from ravenframework.utils import TreeStructure > > (and modify all uses of utils.TreeStructure to TreeStructure) > > Original: > > import utils.TreeStructure as TS > > New: > > from ravenframework.utils import TreeStructure as TS > > Importlib (Example from ravenframework/Models/Code.py ) > > Original: > > importlib.import_module("CodeInterfaces") > > New: > > importlib.import_module("..CodeInterfaces", "ravenframework.Models") > > (Note the second parameter package anchor for finding relative packages.) > > Driver.py can no longer be used from the python executable. > > Original: > > python framework/Driver.py "${ARGS[@]}" > > New: > > python raven_framework.py "${ARGS[@]}" > > or > > raven_framework "${ARGS[@]}" > > — > Reply to this email directly, view it on GitHub > <#1784 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/APD4NWKD35KGR4VRUOXKZODVJVCNJANCNFSM5QEPTSVQ > > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fidaholab%2Fraven%2Fpull%2F1784%23issuecomment-1125286090&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cXNZa3ix25uY2ZRsXo7rMXsh8ifKl9EmEA2U%2B3CRL1w%3D&reserved=0>, or unsubscribe< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABX3L36O2ZTMRPYX637PXEDVJVCXHANCNFSM5QEPTSVQ&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KDKcPY0zKDXNADA78gJmRDZS%2BhIto836rT8wh2WgWlM%3D&reserved=0 >. You are receiving this because you modified the open/close state.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#1784 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APD4NWKIONRKV6DQYC4LTELVJVF5LANCNFSM5QEPTSVQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

wangcj05 · 2022-05-13T14:46:53Z

Hi Haoyu, Thanks. Once FARM pass the regression tests, I will merge it. Best, Congjian

________________________________ From: Haoyu Wang ***@***.***> Sent: Thursday, May 12, 2022 9:28 PM To: idaholab/raven ***@***.***> Cc: Congjian Wang ***@***.***>; State change ***@***.***> Subject: [EXTERNAL] Re: [idaholab/raven] Framework as package (PR #1784) Hello Congjian, I opened a PR #1826 called "Farm submodule update". Please have a check and let me know if more changes are needed. Thanks, Haoyu On Thu, May 12, 2022 at 1:39 PM Congjian Wang - INL < ***@***.***> wrote:

Wonderful! Thanks a lot. Congjian ________________________________ From: Haoyu Wang ***@***.***> Sent: Thursday, May 12, 2022 12:12 PM To: idaholab/raven ***@***.***> Cc: Congjian Wang ***@***.***>; State change ***@***.***> Subject: [EXTERNAL] Re: [idaholab/raven] Framework as package (PR #1784) Hello Congjian, Thank you for letting me know. I can fix it. Paul gave me the instructions on how to update the submodule versions a moment ago. I will get it done no later than next Monday. Thanks, Haoyu On Thu, May 12, 2022 at 1:09 PM Congjian Wang - INL < ***@***.***> wrote: > Hi @wanghy-anl <https://github.com/wanghy-anl>, this PR causes the > failure of FARM on RAVEN regression test machines. Basically, we changed > the way to import raven modules. Do you have plan to update the module > import in FARM? The following is the suggestion how to change the import > from Josh: > > Suggested email: > > In order to make a raven pip package, RAVEN's code needs to be inside a single package. This requires changing all the imports, since now they are inside ravenframework, instead of just at the top level. In addition, the directory framework has been renamed to ravenframework. > > This will require changes in three things. > > Any imports of things that used to be in framework, now need to have ravenframework explicitly mentioned. > Direct use of Driver.py can no longer be done, since the imports will not be setup correctly. Use raven_framework.py instead. > Pickle files have the package information embedded inside, so they will need to be regenerated. > This is pull request: #1784 > > The following are example modifications for framework as a package in the source code. > > Relative (use . if in ravenframework/, .. if in ravenframework/someDir/, ... if in ravenframework/someDir/anotherDir/ and so forth. > > Example for something that is in a directory inside of ravenframework: > > Original: > > import Files > > New: > > from .. import Files > > Original: > > from utils import mathUtils > > New: > > from ..utils import mathUtils > > Original: > > import utils.TreeStructure > > New: > > from ..utils import TreeStructure > > (and modify all uses of utils.TreeStructure to TreeStructure) > > Original: > > import utils.TreeStructure as TS > > New: > > from ..utils import TreeStructure as TS > > Ravenframework imports: > > Original: > > import Files > > New: > > from ravenframework import Files > > Original: > > from utils import mathUtils > > New: > > from ravenframework.utils import mathUtils > > Original: > > import utils.TreeStructure > > New: > > from ravenframework.utils import TreeStructure > > (and modify all uses of utils.TreeStructure to TreeStructure) > > Original: > > import utils.TreeStructure as TS > > New: > > from ravenframework.utils import TreeStructure as TS > > Importlib (Example from ravenframework/Models/Code.py ) > > Original: > > importlib.import_module("CodeInterfaces") > > New: > > importlib.import_module("..CodeInterfaces", "ravenframework.Models") > > (Note the second parameter package anchor for finding relative packages.) > > Driver.py can no longer be used from the python executable. > > Original: > > python framework/Driver.py "${ARGS[@]}" > > New: > > python raven_framework.py "${ARGS[@]}" > > or > > raven_framework "${ARGS[@]}" > > — > Reply to this email directly, view it on GitHub > <#1784 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/APD4NWKD35KGR4VRUOXKZODVJVCNJANCNFSM5QEPTSVQ > > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fidaholab%2Fraven%2Fpull%2F1784%23issuecomment-1125286090&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cXNZa3ix25uY2ZRsXo7rMXsh8ifKl9EmEA2U%2B3CRL1w%3D&reserved=0>, or unsubscribe< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABX3L36O2ZTMRPYX637PXEDVJVCXHANCNFSM5QEPTSVQ&data=05%7C01%7Ccongjian.wang%40inl.gov%7C3f73ffce0bdd4d77f19208da3442eb4e%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637879759274720786%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KDKcPY0zKDXNADA78gJmRDZS%2BhIto836rT8wh2WgWlM%3D&reserved=0 >. You are receiving this because you modified the open/close state.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#1784 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APD4NWKIONRKV6DQYC4LTELVJVF5LANCNFSM5QEPTSVQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fidaholab%2Fraven%2Fpull%2F1784%23issuecomment-1125622084&data=05%7C01%7Ccongjian.wang%40inl.gov%7C5e75f7b522684ed05a6208da3490a6ff%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637880093135584226%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PiUZ1F53hdvFBmEqNuC15b7cr2%2B7BNjrdeUrV0Gwk6A%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABX3L3ZMGQ6FY3GFTJMKOTDVJXD55ANCNFSM5QEPTSVQ&data=05%7C01%7Ccongjian.wang%40inl.gov%7C5e75f7b522684ed05a6208da3490a6ff%7C4cf464b7869a42368da2a98566485554%7C0%7C0%7C637880093135584226%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dbDs1Id3I8Xy0wrWuTRlKGJWmQ%2F22bEFnWJsrd2akEU%3D&reserved=0>. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

joshua-cogliati-inl added 8 commits March 14, 2022 10:35

Converting main raven framework to using framework as a package.

f34da94

Switching test harness to framework as a package.

10a8ef5

Switching tests to using framework as a package.

c7fd7d2

Switching ExamplePlugin to support framework as a package.

94d3344

Print better debugging information when Testers or Differs fail.

29b4c6c

Adding setuptools to dependency list.

893c75d

Switching to framework as a package.

9e6c793

Adding license for raven_framework.py

9f37680

joshua-cogliati-inl force-pushed the framework_as_package branch from a3f36b8 to 9f37680 Compare March 14, 2022 16:36

Moving framework to ravenframework

c80c840

joshua-cogliati-inl force-pushed the framework_as_package branch from cded721 to 366ae3b Compare March 15, 2022 21:28

joshua-cogliati-inl added Do Not Merge Ready To Review labels Mar 16, 2022

PaulTalbot-INL reviewed Mar 16, 2022

View reviewed changes

ravenframework/InputTemplates/TemplateBaseClass.py Outdated Show resolved Hide resolved

PaulTalbot-INL previously requested changes Mar 16, 2022

View reviewed changes

Converting to ravenframework

0b570f5

joshua-cogliati-inl added 4 commits March 16, 2022 10:10

Removing unneeded commented out lines.

1522da8

Converting to ravenframework

046457e

Updating pickle to ravenframework.

03d3739

Fixing crow tests.

137cf9a

joshua-cogliati-inl force-pushed the framework_as_package branch from 641e4a2 to 6f224ad Compare March 16, 2022 16:41

joshua-cogliati-inl added 2 commits March 16, 2022 10:45

Fix whitespace.

5475d00

Adding missing __init__.py files.

b3751db

wangcj05 requested changes Mar 17, 2022

View reviewed changes

Updating documents for new framework directory.

6d79372

joshua-cogliati-inl force-pushed the framework_as_package branch from 2ec3b0e to 6d79372 Compare March 23, 2022 22:43

joshua-cogliati-inl mentioned this pull request Mar 28, 2022

[DEFECT] import (discovery) of RAVEN contrib seems to be compromised #1793

Closed

13 tasks

joshua-cogliati-inl removed the Do Not Merge label Mar 28, 2022

wangcj05 approved these changes Mar 31, 2022

View reviewed changes

wangcj05 mentioned this pull request Mar 31, 2022

Consistent Module Imports #42

Closed

10 tasks

wangcj05 merged commit cb6fb8f into idaholab:devel Mar 31, 2022

This was referenced Mar 31, 2022

updated to latest TEAL devel #1777

Closed

Alfoa/feature selection #1301

Merged

joshua-cogliati-inl mentioned this pull request Apr 15, 2022

[WIP] Jimmy testing representativity based on changes in basic stat #1790

Closed

9 tasks

wanghy-anl mentioned this pull request May 13, 2022

Farm submodule update #1826

Merged

9 tasks

wangcj05 mentioned this pull request May 19, 2022

Fix dymola interface #1642

Closed

9 tasks

wangcj05 added the RAVENv2.2 for RAVENv2.2 Release label Jun 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Framework as package #1784

Framework as package #1784

joshua-cogliati-inl commented Mar 7, 2022 •

edited by wangcj05

Loading

moosebuild commented Mar 7, 2022

moosebuild commented Mar 7, 2022

moosebuild commented Mar 8, 2022

moosebuild commented Mar 15, 2022

joshua-cogliati-inl commented Mar 16, 2022 •

edited

Loading

joshua-cogliati-inl commented Mar 16, 2022

PaulTalbot-INL commented Mar 16, 2022

joshua-cogliati-inl commented Mar 16, 2022

PaulTalbot-INL left a comment

PaulTalbot-INL commented Mar 16, 2022

PaulTalbot-INL commented Mar 16, 2022

wangcj05 left a comment

wangcj05 Mar 17, 2022

PaulTalbot-INL Mar 17, 2022

joshua-cogliati-inl Mar 17, 2022

wangcj05 commented Mar 17, 2022

joshua-cogliati-inl commented Mar 17, 2022

joshua-cogliati-inl commented Mar 23, 2022

moosebuild commented Mar 24, 2022

moosebuild commented Mar 24, 2022

wangcj05 left a comment

wangcj05 commented Mar 31, 2022

wangcj05 commented Mar 31, 2022

wangcj05 commented May 12, 2022

wanghy-anl commented May 12, 2022 via email

wangcj05 commented May 12, 2022 via email

wanghy-anl commented May 13, 2022 via email

wangcj05 commented May 13, 2022 via email

Framework as package #1784

Framework as package #1784

Conversation

joshua-cogliati-inl commented Mar 7, 2022 • edited by wangcj05 Loading

Pull Request Description

What issue does this change request address?

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

moosebuild commented Mar 7, 2022

moosebuild commented Mar 7, 2022

moosebuild commented Mar 8, 2022

moosebuild commented Mar 15, 2022

joshua-cogliati-inl commented Mar 16, 2022 • edited Loading

joshua-cogliati-inl commented Mar 16, 2022

PaulTalbot-INL commented Mar 16, 2022

joshua-cogliati-inl commented Mar 16, 2022

PaulTalbot-INL left a comment

Choose a reason for hiding this comment

PaulTalbot-INL commented Mar 16, 2022

PaulTalbot-INL commented Mar 16, 2022

wangcj05 left a comment

Choose a reason for hiding this comment

wangcj05 Mar 17, 2022

Choose a reason for hiding this comment

PaulTalbot-INL Mar 17, 2022

Choose a reason for hiding this comment

joshua-cogliati-inl Mar 17, 2022

Choose a reason for hiding this comment

wangcj05 commented Mar 17, 2022

joshua-cogliati-inl commented Mar 17, 2022

joshua-cogliati-inl commented Mar 23, 2022

moosebuild commented Mar 24, 2022

moosebuild commented Mar 24, 2022

wangcj05 left a comment

Choose a reason for hiding this comment

wangcj05 commented Mar 31, 2022

wangcj05 commented Mar 31, 2022

wangcj05 commented May 12, 2022

wanghy-anl commented May 12, 2022 via email

wangcj05 commented May 12, 2022 via email

wanghy-anl commented May 13, 2022 via email

wangcj05 commented May 13, 2022 via email

joshua-cogliati-inl commented Mar 7, 2022 •

edited by wangcj05

Loading

joshua-cogliati-inl commented Mar 16, 2022 •

edited

Loading