Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ with SlurmClusterExecutor() as exe:
```
In this case the [Python simple queuing system adapter (pysqa)](https://pysqa.readthedocs.io) is used to submit the
`calc()` function to the [SLURM](https://slurm.schedmd.com) job scheduler and request an allocation with two CPU cores
for the execution of the function - [HPC Cluster Executor](https://executorlib.readthedocs.io/en/latest/2-hpc-submission.html). In the background the [sbatch](https://slurm.schedmd.com/sbatch.html)
for the execution of the function - [HPC Cluster Executor](https://executorlib.readthedocs.io/en/latest/2-hpc-cluster.html). In the background the [sbatch](https://slurm.schedmd.com/sbatch.html)
command is used to request the allocation to execute the Python function.

Within a given [SLURM](https://slurm.schedmd.com) job executorlib can also be used to assign a subset of the
Expand Down
4 changes: 2 additions & 2 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,8 @@ documentation covers the [installation of pysqa](https://pysqa.readthedocs.io/en
detail.

## HPC Job Executor
For optimal performance in [HPC Allocation Mode](https://executorlib.readthedocs.io/en/latest/3-hpc-job.html) the
[flux framework](https://flux-framework.org) is recommended as job scheduler. Even when the [Simple Linux Utility for Resource Management (SLURM)](https://slurm.schedmd.com)
For optimal performance the [HPC Job Executor](https://executorlib.readthedocs.io/en/latest/3-hpc-job.html) leverages the
[flux framework](https://flux-framework.org) as its recommended job scheduler. Even when the [Simple Linux Utility for Resource Management (SLURM)](https://slurm.schedmd.com)
or any other job scheduler is already installed on the HPC cluster. [flux framework](https://flux-framework.org) can be
installed as a secondary job scheduler to leverage [flux framework](https://flux-framework.org) for the distribution of
resources within a given allocation of the primary scheduler.
Expand Down
7 changes: 4 additions & 3 deletions docs/trouble_shooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@ dependency. The installation of this and other optional dependencies is covered

## Missing Dependencies
The default installation of executorlib only comes with a limited number of dependencies, especially the [zero message queue](https://zeromq.org)
and [cloudpickle](https://github.com/cloudpipe/cloudpickle). Additional features like [caching](https://executorlib.readthedocs.io/en/latest/installation.html#caching), [HPC submission mode](https://executorlib.readthedocs.io/en/latest/installation.html#hpc-cluster-executor)
and [HPC allocation mode](https://executorlib.readthedocs.io/en/latest/installation.html#hpc-job-executor) require additional dependencies. The dependencies are explained in more detail in the
[installation section](https://executorlib.readthedocs.io/en/latest/installation.html#).
and [cloudpickle](https://github.com/cloudpipe/cloudpickle). Additional features like [caching](https://executorlib.readthedocs.io/en/latest/installation.html#caching), the [HPC Cluster Executors](https://executorlib.readthedocs.io/en/latest/installation.html#hpc-cluster-executor)
and the [HPC Job Executors](https://executorlib.readthedocs.io/en/latest/installation.html#hpc-job-executor) require
additional dependencies. The dependencies are explained in more detail in the
[installation section](https://executorlib.readthedocs.io/en/latest/installation.html).

## Python Version
Executorlib supports all current Python version ranging from 3.9 to 3.13. Still some of the dependencies and especially
Expand Down
2 changes: 1 addition & 1 deletion notebooks/3-hpc-job.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@
"metadata": {},
"source": [
"### Block Allocation\n",
"The block allocation for the HPC allocation mode follows the same implementation as the [block allocation for the local mode](https://executorlib.readthedocs.io/en/latest/1-local.html#block-allocation). It starts by defining the initialization function `init_function()` which returns a dictionary which is internally used to look up input parameters for Python functions submitted to the `FluxJobExecutor` class. Commonly this functionality is used to store large data objects inside the Python process created for the block allocation, rather than reloading these Python objects for each submitted function."
"The block allocation for the HPC allocation mode follows the same implementation as the [block allocation for the Single Node Executor](https://executorlib.readthedocs.io/en/latest/1-single-node.html#block-allocation). It starts by defining the initialization function `init_function()` which returns a dictionary which is internally used to look up input parameters for Python functions submitted to the `FluxJobExecutor` class. Commonly this functionality is used to store large data objects inside the Python process created for the block allocation, rather than reloading these Python objects for each submitted function."
]
},
{
Expand Down
Loading