Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 21 additions & 107 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,23 @@
# PySlurm
# <img src="logo.png" alt="PySlurm Logo">

[![PySlurm](https://github.com/PySlurm/pyslurm/actions/workflows/pyslurm.yml/badge.svg?branch=main)](https://github.com/PySlurm/pyslurm/actions/workflows/pyslurm.yml)
pyslurm is the Python client library for the [Slurm Workload Manager](https://slurm.schedmd.com)

## Overview

PySlurm is the Python client library for the [Slurm](https://slurm.schedmd.com) HPC Scheduler.

## Prerequisites
## Requirements

* [Slurm](https://slurm.schedmd.com) - Slurm shared library and header files
* [Python](https://www.python.org) - >= 3.6
* [Cython](https://cython.org) - >= 0.29.30 but < 3.0

This PySlurm branch is for the Slurm Major-Release 23.02
This Version is for Slurm 23.02.x

## Versioning

In pyslurm, the versioning scheme follows the official Slurm versioning. The
first two numbers (`MAJOR.MINOR`) always correspond to Slurms Major-Release,
for example `23.02`.
The last number (`MICRO`) is however not tied in any way to Slurms `MICRO`
version, but is instead PySlurm's internal Patch-Level. For example, any
pyslurm 23.02.X version should work with any Slurm 23.02.X release.

## Installation

Expand All @@ -21,14 +26,14 @@ By default, it is searched inside `/usr/include` for the Header files and in
For Slurm installations in different locations, you will need to provide
the corresponding paths to the necessary files.

You can specify these Paths with environment variables (recommended), for example:
You can specify those with environment variables (recommended), for example:

```shell
export SLURM_INCLUDE_DIR=/opt/slurm/23.02/include
export SLURM_LIB_DIR=/opt/slurm/23.02/lib
```

Then you can proceed to install PySlurm, for example by cloning the Repository:
Then you can proceed to install pyslurm, for example by cloning the Repository:

```shell
git clone https://github.com/PySlurm/pyslurm.git && cd pyslurm
Expand All @@ -40,105 +45,14 @@ pip install .

Also see `python setup.py --help`

## Release Versioning

PySlurm's versioning scheme follows the official Slurm versioning. The first
two numbers (MAJOR.MINOR) always correspond to Slurms Major-Release, for example
`23.02`. The last number (MICRO) is however not tied in any way to Slurms
MICRO version. For example, any PySlurm 23.02.X version should work with any
Slurm 23.02.X release.

## Documentation

The API documentation is hosted at <https://pyslurm.github.io>.

To build the docs locally, use [Sphinx](http://www.sphinx-doc.org) to generate
the documentation from the reStructuredText based docstrings found in the
pyslurm module once it is built:

```shell
cd doc
make clean
make html
```

## Testing

PySlurm requires an installation of Slurm.

### Using a Test Container

To run tests locally without an existing Slurm cluster, `docker` and
`docker-compose` is required.

Clone the project:

```shell
git clone https://github.com/PySlurm/pyslurm.git
cd pyslurm
```

Start the Slurm container in the background:

```shell
docker-compose up -d
```

The cluster takes a few seconds to start all the required Slurm services. Tail
the logs:

```shell
docker-compose logs -f
```

When the cluster is ready, you will see the following log message:

```text
Cluster is now available
```

Press CTRL+C to stop tailing the logs. Slurm is now running in a container in
detached mode. `docker-compose` also bind mounds the git directory inside the
container at `/pyslurm` so that the container has access to the test cases.

Install test dependencies:

```shell
pipenv sync --dev
```

Execute the tests inside the container:

```shell
pipenv run pytest -sv scripts/run_tests_in_container.py
```

When testing is complete, stop the running Slurm container:

```shell
docker-compose down
```

### Testing on an Existing Slurm Cluster

You may also choose to clone the project and run tests on a node where Slurm is
already compiled and installed:

```shell
git clone https://github.com/PySlurm/pyslurm.git
cd pyslurm
pip install .
./scripts/configure.sh
pipenv sync --dev
pipenv run pytest -sv
```

## Contributors

PySlurm is made by [contributors like
pyslurm is made by [contributors like
you](https://github.com/PySlurm/pyslurm/graphs/contributors).

## Help
## Support

Feel free to ask questions in the [GitHub
Discussions](https://github.com/orgs/PySlurm/discussions)

Ask questions on the [PySlurm Google
Group](https://groups.google.com/forum/#!forum/pyslurm)
Found a bug or you are missing a feature? Feel free to [open an Issue!](https://github.com/PySlurm/pyslurm/issues/new)
1 change: 1 addition & 0 deletions doc_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ setuptools
mkdocstrings[python]
mike
mkdocs-material
mkdocs-awesome-pages-plugin
53 changes: 1 addition & 52 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,52 +1 @@
---
hide:
- navigation
---
# PySlurm: Slurm Interface to python

This module provides a low-level Python wrapper around the Slurm C-API using Cython.

::: pyslurm.config
handler: python

::: pyslurm.front_end
handler: python

::: pyslurm.hostlist
handler: python

::: pyslurm.job
handler: python

::: pyslurm.jobstep
handler: python

::: pyslurm.node
handler: python

::: pyslurm.partition
handler: python

::: pyslurm.reservation
handler: python

::: pyslurm.slurmdb_events
handler: python

::: pyslurm.slurmdb_reservations
handler: python

::: pyslurm.slurmdb_clusters
handler: python

::: pyslurm.slurmdb_jobs
handler: python

::: pyslurm.statistics
handler: python

::: pyslurm.topology
handler: python

::: pyslurm.trigger
handler: python
--8<-- "README.md"
1 change: 1 addition & 0 deletions docs/logo.png
3 changes: 3 additions & 0 deletions docs/reference/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
title: API Reference
nav:
- ...
10 changes: 10 additions & 0 deletions docs/reference/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Config
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.config
handler: python
3 changes: 3 additions & 0 deletions docs/reference/db/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
title: Database
nav:
- ...
10 changes: 10 additions & 0 deletions docs/reference/db/cluster.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Cluster
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_clusters
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/connection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Connection
---

::: pyslurm.db.Connection
handler: python
10 changes: 10 additions & 0 deletions docs/reference/db/event.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Event
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_events
handler: python
4 changes: 4 additions & 0 deletions docs/reference/db/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# pyslurm.db

The `pyslurm.db` package contains all functionality to interact with the Slurm
Database Daemon (slurmdbd)
13 changes: 13 additions & 0 deletions docs/reference/db/job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
title: Job
---

!!! note
This supersedes the [pyslurm.slurmdb_job](../old/db/job.md) class, which
will be removed in a future release

::: pyslurm.db.Job
handler: python

::: pyslurm.db.Jobs
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/jobsearchfilter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: JobSearchFilter
---

::: pyslurm.db.JobSearchFilter
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/jobstats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: JobStatistics
---

::: pyslurm.db.JobStatistics
handler: python
9 changes: 9 additions & 0 deletions docs/reference/db/jobstep.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: JobStep
---

::: pyslurm.db.JobStep
handler: python

::: pyslurm.db.JobSteps
handler: python
10 changes: 10 additions & 0 deletions docs/reference/db/reservation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Reservation
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_reservations
handler: python
9 changes: 9 additions & 0 deletions docs/reference/exceptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: Exceptions
---

::: pyslurm.PyslurmError
handler: python

::: pyslurm.RPCError
handler: python
10 changes: 10 additions & 0 deletions docs/reference/frontend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Frontend
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.front_end
handler: python
10 changes: 10 additions & 0 deletions docs/reference/hostlist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Hostlist
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.hostlist
handler: python
Loading