Skip to content

Commit 9c60a2f

Browse files
jan-janssenjan-janssen
authored
Move examples to separate folder (#73)
* Move examples to separate folder * update environment path * fix import * update python module path * update postbuild script * move mybinder related parts * rename notebooks * rename python modules
1 parent 67a983b commit 9c60a2f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+4355
-97
lines changed

‎.github/workflows/pipeline.yml

Lines changed: 26 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
- uses: psf/black@stable
1818
with:
1919
options: "--check --diff"
20-
src: ./qe_xml_parser/src/qe_xml_parser
20+
src: ./example_workflows/quantum_espresso/qe_xml_parser/src/qe_xml_parser
2121

2222
nfdi4ing:
2323
runs-on: ubuntu-22.04
@@ -27,57 +27,60 @@ jobs:
2727
with:
2828
auto-update-conda: true
2929
python-version: "3.12"
30-
environment-file: environment.yml
30+
environment-file: binder/environment.yml
3131
auto-activate-base: false
3232
- name: Installation and setup
3333
shell: bash -l {0}
3434
run: |
3535
pip install -e python_workflow_definition
3636
conda install -c conda-forge jupyter papermill
3737
verdi presto --profile-name pwd
38-
sudo apt-get install -y $(cat apt.txt)
38+
sudo apt-get install -y $(cat binder/apt.txt)
3939
- name: Create Additional Conda Environments
4040
shell: bash -l {0}
4141
run: |
42-
conda env create -n preprocessing -f source/envs/preprocessing.yaml -y
43-
conda env create -n processing -f source/envs/processing.yaml -y
44-
conda env create -n postprocessing -f source/envs/postprocessing.yaml -y
42+
conda env create -n preprocessing -f example_workflows/nfdi/source/envs/preprocessing.yaml -y
43+
conda env create -n processing -f example_workflows/nfdi/source/envs/processing.yaml -y
44+
conda env create -n postprocessing -f example_workflows/nfdi/source/envs/postprocessing.yaml -y
4545
- name: Tests
4646
shell: bash -l {0}
4747
run: |
48-
papermill aiida_nfdi.ipynb aiida_nfdi_out.ipynb -k "python3"
49-
papermill jobflow_nfdi.ipynb jobflow_nfdi_out.ipynb -k "python3"
50-
papermill pyiron_base_nfdi.ipynb pyiron_base_nfdi_out.ipynb -k "python3"
51-
papermill universal_workflow_nfdi.ipynb universal_workflow_nfdi_out.ipynb -k "python3"
48+
cd example_workflows/nfdi/
49+
papermill aiida.ipynb aiida_out.ipynb -k "python3"
50+
papermill jobflow.ipynb jobflow_out.ipynb -k "python3"
51+
papermill pyiron_base.ipynb pyiron_base_out.ipynb -k "python3"
52+
papermill universal_workflow.ipynb universal_workflow_out.ipynb -k "python3"
5253
5354
quantumespresso:
5455
runs-on: ubuntu-latest
5556
steps:
5657
- uses: actions/checkout@v4
5758
- name: Setup environment
5859
shell: bash -l {0}
59-
run: tail --lines=+4 environment_qe.yml >> environment.yml
60+
run: tail --lines=+4 example_workflows/quantum_espresso/environment.yml >> binder/environment.yml
6061
- uses: conda-incubator/setup-miniconda@v3
6162
with:
6263
auto-update-conda: true
6364
python-version: "3.12"
64-
environment-file: environment.yml
65+
environment-file: binder/environment.yml
6566
auto-activate-base: false
6667
- name: Installation and setup
6768
shell: bash -l {0}
6869
run: |
69-
pip install -e qe_xml_parser
70+
pip install -e example_workflows/quantum_espresso/qe_xml_parser
7071
pip install -e python_workflow_definition
7172
conda install -c conda-forge jupyter papermill
7273
verdi presto --profile-name pwd
7374
- name: Tests
7475
shell: bash -l {0}
7576
run: |
7677
export ESPRESSO_PSEUDO=$(pwd)/espresso/pseudo
77-
papermill aiida_qe.ipynb aiida_qe_out.ipynb -k "python3"
78-
papermill jobflow_qe.ipynb jobflow_qe_out.ipynb -k "python3"
79-
papermill pyiron_base_qe.ipynb pyiron_base_qe_out.ipynb -k "python3"
80-
papermill universal_workflow_qe.ipynb universal_workflow_qe_out.ipynb -k "python3"
78+
cp -r example_workflows/quantum_espresso/espresso .
79+
cd example_workflows/quantum_espresso
80+
papermill aiida.ipynb aiida_out.ipynb -k "python3"
81+
papermill jobflow.ipynb jobflow_out.ipynb -k "python3"
82+
papermill pyiron_base.ipynb pyiron_base_out.ipynb -k "python3"
83+
papermill universal_workflow.ipynb universal_workflow_out.ipynb -k "python3"
8184
8285
arithmetic:
8386
runs-on: ubuntu-latest
@@ -87,7 +90,7 @@ jobs:
8790
with:
8891
auto-update-conda: true
8992
python-version: "3.12"
90-
environment-file: environment.yml
93+
environment-file: binder/environment.yml
9194
auto-activate-base: false
9295
- name: Installation and setup
9396
shell: bash -l {0}
@@ -98,10 +101,11 @@ jobs:
98101
- name: Tests
99102
shell: bash -l {0}
100103
run: |
101-
papermill aiida_arithmetic.ipynb aiida_arithmetic_out.ipynb -k "python3"
102-
papermill jobflow_arithmetic.ipynb jobflow_arithmetic_out.ipynb -k "python3"
103-
papermill pyiron_base_arithmetic.ipynb pyiron_base_arithmetic_out.ipynb -k "python3"
104-
papermill universal_workflow_arithmetic.ipynb universal_workflow_arithmetic_out.ipynb -k "python3"
104+
cd example_workflows/arithmetic
105+
papermill aiida.ipynb aiida_out.ipynb -k "python3"
106+
papermill jobflow.ipynb jobflow_out.ipynb -k "python3"
107+
papermill pyiron_base.ipynb pyiron_base_out.ipynb -k "python3"
108+
papermill universal_workflow.ipynb universal_workflow_out.ipynb -k "python3"
105109
106110
book:
107111
runs-on: ubuntu-latest

‎README.md

Lines changed: 28 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@ These two Python functions are combined in the following example workflow:
2727
tmp_dict = get_prod_and_div(x=1, y=2)
2828
result = get_sum(x=tmp_dict["prod"], y=tmp_dict["div"])
2929
```
30-
For the workflow representation of these Python functions the Python functions are stored in the [arithmetic_workflow.py](arithmetic_workflow.py)
31-
Python module. The connection of the Python functions are stored in the [workflow_arithmetic.json](workflow_arithmetic.json)
30+
For the workflow representation of these Python functions the Python functions are stored in the [arithmetic_workflow.py](example_workflows/arithmetic/workflow.py)
31+
Python module. The connection of the Python functions are stored in the [workflow_arithmetic.json](example_workflows/arithmetic/workflow.json)
3232
JSON file:
3333
```
3434
{
@@ -59,45 +59,45 @@ The corresponding Jupyter notebooks demonstrate this functionality:
5959

6060
| Example | Explanation |
6161
|----------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
62-
| [aiida_arithmetic.ipynb](aiida_arithmetic.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
63-
| [jobflow_arithmetic.ipynb](jobflow_arithmetic.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
64-
| [pyiron_base_arithmetic.ipynb](pyiron_base_arithmetic.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
65-
| [universal_workflow_arithmetic.ipynb](universal_workflow_arithmetic.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |
62+
| [aiida_arithmetic.ipynb](example_workflows/arithmetic/aiida.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
63+
| [jobflow_arithmetic.ipynb](example_workflows/arithmetic/jobflow.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
64+
| [pyiron_base_arithmetic.ipynb](example_workflows/arithmetic/pyiron_base.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
65+
| [universal_workflow_arithmetic.ipynb](example_workflows/arithmetic/universal_workflow.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |
6666

6767
### Quantum Espresso Workflow
6868
The second workflow example is the calculation of an energy volume curve with Quantum Espresso. In the first step the
6969
initial structure is relaxed, afterward it is strained and the total energy is calculated.
70-
* [quantum_espresso_workflow.py](quantum_espresso_workflow.py) Python functions
71-
* [workflow_qe.json](workflow_qe.json) Workflow definition in the Python Workflow Definition.
72-
* [environment_qe.yml](environment_qe.yml) Conda environment
70+
* [quantum_espresso_workflow.py](example_workflows/quantum_espresso/workflow.py) Python functions
71+
* [workflow_qe.json](example_workflows/quantum_espresso/workflow.json) Workflow definition in the Python Workflow Definition.
72+
* [environment_qe.yml](example_workflows/quantum_espresso/environment.yml) Conda environment
7373

7474
| Example | Explanation |
7575
|------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
76-
| [aiida_qe.ipynb](aiida_qe.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
77-
| [jobflow_qe.ipynb](jobflow_qe.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
78-
| [pyiron_base_qe.ipynb](pyiron_base_qe.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
79-
| [universal_workflow_qe.ipynb](universal_workflow_qe.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |
76+
| [aiida_qe.ipynb](example_workflows/quantum_espresso/aiida.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
77+
| [jobflow_qe.ipynb](example_workflows/quantum_espresso/jobflow.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
78+
| [pyiron_base_qe.ipynb](example_workflows/quantum_espresso/pyiron_base.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
79+
| [universal_workflow_qe.ipynb](example_workflows/quantum_espresso/universal_workflow.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |
8080

8181
### NFDI4Ing Scientific Workflow Requirements
8282
To demonstrate the compatibility of the Python Workflow Definition to file based workflows, the workflow benchmark developed as part of [NFDI4Ing](https://github.com/BAMresearch/NFDI4IngScientificWorkflowRequirements)
8383
is implemented for all three simulation codes based on a shared workflow definition.
84-
* [nfdi_ing_workflow.py](nfdi_ing_workflow.py) Python functions
85-
* [workflow_nfdi.json](workflow_nfdi.json) Workflow definition in the Python Workflow Definition.
84+
* [nfdi_ing_workflow.py](example_workflows/nfdi/workflow.py) Python functions
85+
* [workflow_nfdi.json](example_workflows/nfdi/workflow.json) Workflow definition in the Python Workflow Definition.
8686

8787
Additional source files provided with the workflow benchmark:
88-
* [source/envs/preprocessing.yaml](source/envs/preprocessing.yaml) Conda environment for preprocessing
89-
* [source/envs/processing.yaml](source/envs/processing.yaml) Conda environment for processing
90-
* [source/envs/postprocessing.yaml](source/envs/postprocessing.yaml) Conda environment for postprocessing
91-
* [source/macros.tex.template](source/macros.tex.template) LaTeX module template
92-
* [source/paper.tex](source/paper.tex) LaTeX paper template
93-
* [source/poisson.py](source/poisson.py) Poisson Python script
94-
* [source/postprocessing.py](source/postprocessing.py) Postprocessing Python script
95-
* [source/prepare_paper_macros.py](source/prepare_paper_macros.py) LaTeX preprocessing Python script
96-
* [source/unit_square.geo](source/unit_square.geo) Input structure
88+
* [source/envs/preprocessing.yaml](example_workflows/nfdi/source/envs/preprocessing.yaml) Conda environment for preprocessing
89+
* [source/envs/processing.yaml](example_workflows/nfdi/source/envs/processing.yaml) Conda environment for processing
90+
* [source/envs/postprocessing.yaml](example_workflows/nfdi/source/envs/postprocessing.yaml) Conda environment for postprocessing
91+
* [source/macros.tex.template](example_workflows/nfdi/source/macros.tex.template) LaTeX module template
92+
* [source/paper.tex](example_workflows/nfdi/source/paper.tex) LaTeX paper template
93+
* [source/poisson.py](example_workflows/nfdi/source/poisson.py) Poisson Python script
94+
* [source/postprocessing.py](example_workflows/nfdi/source/postprocessing.py) Postprocessing Python script
95+
* [source/prepare_paper_macros.py](example_workflows/nfdi/source/prepare_paper_macros.py) LaTeX preprocessing Python script
96+
* [source/unit_square.geo](example_workflows/nfdi/source/unit_square.geo) Input structure
9797

9898
| Example | Explanation |
9999
|----------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
100-
| [aiida_nfdi.ipynb](aiida_nfdi.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
101-
| [jobflow_nfdi.ipynb](jobflow_nfdi.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
102-
| [pyiron_base_nfdi.ipynb](pyiron_base_nfdi.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
103-
| [universal_workflow_nfdi.ipynb](universal_workflow_nfdi.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |
100+
| [aiida_nfdi.ipynb](example_workflows/nfdi/aiida.ipynb) | Define Workflow with aiida and execute it with jobflow and pyiron_base. |
101+
| [jobflow_nfdi.ipynb](example_workflows/nfdi/jobflow.ipynb) | Define Workflow with jobflow and execute it with aiida and pyiron_base. |
102+
| [pyiron_base_nfdi.ipynb](example_workflows/nfdi/pyiron_base.ipynb) | Define Workflow with pyiron_base and execute it with aiida and jobflow. |
103+
| [universal_workflow_nfdi.ipynb](example_workflows/nfdi/universal_workflow.ipynb) | Execute workflow defined in the Python Workflow Definition with aiida, executorlib, jobflow, pyiron_base and pure Python. |

0 commit comments

Comments
 (0)