Skip to content

Commit

Permalink
Merge pull request #214 from rcc-uchicago/trung
Browse files Browse the repository at this point in the history
Improving the accounts usage command mention on midway2 and midway3
  • Loading branch information
ndtrung81 authored Sep 3, 2024
2 parents 2c1f58c + 4994ef4 commit 23eeaa0
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 14 deletions.
82 changes: 70 additions & 12 deletions docs/101/allocations.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,82 @@ The `accounts` tool offers a convenient means to check your account balance. Sim
accounts balance
```

You can also log into your [MyRCC](https://my.rcc.uchicago.edu/overview) profile and check the service units of the accounts they are a member of in a user-friendly web interface.

## How do I review the usage of my allocation?
The `accounts` tool provides several options for summarizing allocation usage. To obtain a summary, execute the command:

```
accounts usage
```
The `accounts` tool provides several options for summarizing the usage of the allocations to which your RCC account has access to. To obtain a summary, execute the command `accounts usage` with the account name(s):

To view usage per job, utilize the following command:

```
accounts usage --byjob
```
=== "Midway2, DaLI"
```
accounts usage
accounts usage --account [pi-cnetid]
accounts usage --accounts [pi-cnetid1],[pi-cnetid2]
```

===+ "Midway3, Beagle3"
```
accounts usage
accounts usage -a [pi-cnetid]
```

If you are the PI, you may employ the `--byuser` option to view individual usage by group members:
where `[pi-cnetid]` is the name of the account, often your PI's.

```
accounts usage --byuser
```
You may use the `--byuser` option to view individual usage by group members:

=== "Midway2, DaLI"
```
accounts usage
accounts usage --account [pi-cnetid] --byuser
```

===+ "Midway3, Beagle3"
```
accounts usage
accounts usage -a [pi-cnetid] -byuser
```

You can also use the `rcchelp` command (a wrapper of system-provided tools including `accounts`), which requires slightly different arguments (note only one character '-')

=== "Midway2, DaLI"
```
rcchelp usage
rcchelp usage -account [pi-cnetid] -byuser
```

=== "Midway3, Beagle3"
```
rcchelp usage
rcchelp usage -a [pi-cnetid] -byuser
```

---

To view the resource usage per job, you need to run on Midway2/DaLI using either `accounts`:

=== "Midway2, DaLI"
```
accounts usage --byjob
accounts usage --account [pi-cnetid] --byjob
```

or `rcchelp`

=== "Midway2, DaLI"
```
rcchelp usage -byjob
rcchelp usage -account [pi-cnetid] -byjob
```

both of which include the jobs on both Midway2 and Midway3 partitions (e.g. `caslake` and `gpu`).

!!!note
These commands, although executed on Midway2 ecosystem, provide the necessary information about jobs run on both Midway2 and Midway3.

The output is a table that lists the job IDs, partitions, number of CPU cores and SU consumptions. To further check the resource consumption of a job on Midway3, you need to log in to Midway3 and run `sacct -j` with the job ID.

If the job is still running, you can use `scontrol show job` with the job ID to check the job information.

---

Expand Down
17 changes: 15 additions & 2 deletions docs/slurm/sbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -862,12 +862,25 @@ module load python
for idx in {0..31}
do
taskset -c $i python script.py input-$idx.txt > output-$idx.txt &
taskset -c $idx python script.py input-$idx.txt > output-$idx.txt &
done
wait
```

The `taskset` command binds each Python process to a CPU core ID. Each process reads in the corresponding input file and writes the output to an output file, both indexed by `idx`.
Breakdown:

1. `for idx in {0..31} do ... done`: This loop iterates 32 times, with idx taking values from 0 to 31.

2. `taskset -c $idx`: This command binds the execution of the following command (`python script.py ...`) to a specific CPU core, where `$idx` specifies the core number.

3. `python script.py input-$idx.txt > output-$idx.txt &`: This runs the Python script with a specific input file (`input-$idx.txt`) and writes the output to a corresponding output file (`output-$idx.txt`). The `&` at the end puts each command in the background, allowing them to run in parallel.

4. `wait`: This command makes the script wait until all background processes have finished before it exits

Summary:

The job script parallelizes the execution of script.py across 32 CPU cores, each processing a different input file and saving the results in corresponding output files.


### Dependency jobs

Expand Down

0 comments on commit 23eeaa0

Please sign in to comment.