Improving the accounts usage command mention on midway2 and midway3 #214

ndtrung81 · 2024-08-22T15:47:01Z

The PR improves the description of the accounts usage command and their diffs on midway2 vs midway3.

…d midway3

ndtrung81 · 2024-08-22T16:17:37Z

Currently the command rcchelp usage -account [pi-cnetid] -byjob and the corresponding accounts usage command don't function on midway3, because the cached yaml file fetched from the database does not have the job info. However, users can still get the jobs on midway3 partitions (caslake and gpu) by running these commands on midway2.

ndtrung81 · 2024-08-22T16:19:30Z

@aolykhin this PR has relevant info mentioned in the issue you raised earlier #130 regarding byjob info

od101

thank you, @ndtrung81 .
left minor comments, suggestions, etc.
thank you!

od101 · 2024-08-23T22:31:11Z

docs/101/allocations.md

@@ -26,23 +26,73 @@ accounts balance
 ```

 ## How do I review the usage of my allocation?
-The `accounts` tool provides several options for summarizing allocation usage. To obtain a summary, execute the command:
+The `accounts` tool provides several options for summarizing the usage of the allocations to which your RCC account is accessible. To obtain a summary, execute the command `accounts usage` with the account name(s):
+


"... RCC account is accessible." => "... RCC account has access to."?

Just in case, users can also log into MyRCC profile and see all the service unites of the accounts they are a member of in a user-friendly web interface.

od101 · 2024-08-23T22:34:52Z

docs/101/allocations.md

+      accounts usage
+      accounts usage --account [pi-cnetid]
+      accounts usage --accounts [pi-cnetid1],[pi-cnetid2]
+      ```


all work as shown.

something related:

help shows "-account -a ACCOUNT[,...]", i.e., the option "account" with single dash, i.e., "-", and doesn't show any option of "accounts".

"-a" as shown in help doesn't work.

the example of "accounts" above will work just fine with "account", and otherwise too.

accounts usage --accounts pi-1, pi-2 for me, only returns the first account (pi-1), second one doesn't show up.

Here it looks to me on midway2

[ndtrung@midway2-login1 ~]$ accounts usage --accounts pi-depablo,pi-gagalli Account Usage: Thu Aug 29 15:32:56 2024 Accounts: pi-depablo,pi-gagalli +------------+-------------+ | Account | Charge (SU) | +------------+-------------+ | pi-depablo | 2781855.77 | | pi-gagalli | 1337831.94 | +------------+-------------+

od101 · 2024-08-23T22:38:40Z

docs/101/allocations.md

+      ```
+      accounts usage
+      accounts usage --account [pi-cnetid] --byuser
+      ```


works as expected. "-byuser" works too.

... often your PI's CNetID. Execute groups command to find it, if you are not sure.

od101 · 2024-08-23T22:41:37Z

docs/101/allocations.md

+=== "Midway3, Beagle3"
+      ```
+      rcchelp usage 
+      rcchelp usage -account [pi-cnetid] -byuser


last example needs to be:
rcchelp usage -a [pi-cnetid] -byuser

yes, I fixed that error, thanks @od101

od101 · 2024-08-23T22:42:18Z

docs/101/allocations.md


-To view usage per job, utilize the following command:
+To view the resource usage per job, you need to run on Midway2 before using either `accounts`:



"run on Midway2" => "run on Midway2/DaLI"?

od101 · 2024-08-23T22:44:33Z

docs/101/allocations.md

+      ```
+      accounts usage --byjob
+      accounts usage --account [pi-cnetid] --byjob
+      ```


works fine.

something related: help shows single dash for "byjob", but it works fine with both single/double. same for "account" option.

od101 · 2024-08-23T22:44:56Z

docs/101/allocations.md

-```
+=== "Midway2, DaLI"
+      ```
+      rcchelp usage 


byjob missing.

od101 · 2024-08-23T22:46:59Z

docs/101/allocations.md

+
+both of which include the jobs on both Midway2 and Midway3 partitions (e.g. `caslake` and `gpu`). 
+
+The output is a table that lists the job IDs, partitions, number of CPU cores and SU consumptions. To further check the resource consumption of a job on Midway3, you need to log in to Midway3 and run `sacct -j` or `scontrol show job` with the job ID. 



maybe to note that sacct will work during and after the execution. but, scontrol will work only during, and not after.

pedramesfahani

Thank you @ndtrung81 for the updates.

pedramesfahani · 2024-08-27T13:38:27Z

docs/101/allocations.md

@@ -26,23 +26,73 @@ accounts balance
 ```

 ## How do I review the usage of my allocation?
-The `accounts` tool provides several options for summarizing allocation usage. To obtain a summary, execute the command:
+The `accounts` tool provides several options for summarizing the usage of the allocations to which your RCC account is accessible. To obtain a summary, execute the command `accounts usage` with the account name(s):
+


Just in case, users can also log into MyRCC profile and see all the service unites of the accounts they are a member of in a user-friendly web interface.

pedramesfahani · 2024-08-27T13:43:52Z

docs/101/allocations.md

+      accounts usage
+      accounts usage --account [pi-cnetid]
+      accounts usage --accounts [pi-cnetid1],[pi-cnetid2]
+      ```


accounts usage --accounts pi-1, pi-2 for me, only returns the first account (pi-1), second one doesn't show up.

pedramesfahani · 2024-08-27T13:46:49Z

docs/101/allocations.md

+      ```
+      accounts usage
+      accounts usage --account [pi-cnetid] --byuser
+      ```


... often your PI's CNetID. Execute groups command to find it, if you are not sure.

pedramesfahani · 2024-08-27T13:49:07Z

docs/101/allocations.md

+=== "Midway3, Beagle3"
+      ```
+      rcchelp usage 
+      rcchelp usage -account [pi-cnetid] -byuser


pedramesfahani · 2024-08-27T13:53:48Z

docs/101/allocations.md

+      rcchelp usage -account [pi-cnetid] -byjob
+      ```
+
+both of which include the jobs on both Midway2 and Midway3 partitions (e.g. `caslake` and `gpu`). 


To a bit make the sentence be more clear and easier to follow:
Note! This command, although it is executed on Midway2 ecosystem, provides the necessary information about both Midway2 and Midway3 ecosystems.

pedramesfahani · 2024-08-27T13:57:02Z

docs/slurm/sbatch.md

@@ -862,7 +862,7 @@ module load python

 for idx in {0..31}
 do
-   taskset -c $i python script.py input-$idx.txt > output-$idx.txt &
+   taskset -c $idx python script.py input-$idx.txt > output-$idx.txt &
 done
 wait
 ```


This script is a Bash loop that runs 32 instances of a Python script (script.py) in parallel, each bound to a specific CPU core.

Breakdown:

Loop (for idx in {0..31}): This loop iterates 32 times, with idx taking values from 0 to 31.

taskset -c $idx: This command binds the execution of the following command (python script.py ...) to a specific CPU core, where $idx specifies the core number.

python script.py input-$idx.txt > output-$idx.txt &: This runs the Python script with a specific input file (input-$idx.txt) and writes the output to a corresponding output file (output-$idx.txt). The & at the end puts each command in the background, allowing them to run in parallel.

wait: This command makes the script wait until all background processes have finished before it exits.

Summary:
The script parallelizes the execution of script.py across 32 CPU cores, each processing a different input file and saving the results in corresponding output files.

ndtrung81 · 2024-08-29T20:53:55Z

thanks very much @od101 and @pedramesfahani for the careful reviews. I made the corrections and incorporated your edits.

Trung Nguyen and others added 2 commits August 22, 2024 08:39

Added that the accounts command arguments are different on midway2 an…

10923b3

…d midway3

Mentioned that accounts usage on midway2 show jobs on midway3 as well

9454e9c

ndtrung81 requested review from pedramesfahani and od101 August 22, 2024 15:47

Added that rcchelp can give similar output

66b60f5

ndtrung81 requested a review from aolykhin August 22, 2024 16:17

ndtrung81 mentioned this pull request Aug 22, 2024

accounts usage instructions incorrect for midway 3 #215

Closed

ndtrung81 requested a review from RossHyman August 22, 2024 20:01

od101 approved these changes Aug 23, 2024

View reviewed changes

pedramesfahani reviewed Aug 27, 2024

View reviewed changes

pedramesfahani approved these changes Aug 27, 2024

View reviewed changes

Incorporated edits and suggestions from Pedram and Dossay

cc29a57

fixed a misplaced word, added DaLI

4994ef4

ndtrung81 merged commit 23eeaa0 into main Sep 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improving the accounts usage command mention on midway2 and midway3 #214

Improving the accounts usage command mention on midway2 and midway3 #214

ndtrung81 commented Aug 22, 2024

ndtrung81 commented Aug 22, 2024

ndtrung81 commented Aug 22, 2024

od101 left a comment

od101 Aug 23, 2024

pedramesfahani Aug 27, 2024

od101 Aug 23, 2024

pedramesfahani Aug 27, 2024

ndtrung81 Aug 29, 2024

od101 Aug 23, 2024

pedramesfahani Aug 27, 2024

od101 Aug 23, 2024

pedramesfahani Aug 27, 2024

ndtrung81 Aug 29, 2024

od101 Aug 23, 2024

od101 Aug 23, 2024 •

edited

Loading

od101 Aug 23, 2024

ndtrung81 Aug 29, 2024

od101 Aug 23, 2024

pedramesfahani left a comment

pedramesfahani Aug 27, 2024

pedramesfahani Aug 27, 2024

pedramesfahani Aug 27, 2024

pedramesfahani Aug 27, 2024

pedramesfahani Aug 27, 2024

pedramesfahani Aug 27, 2024

ndtrung81 commented Aug 29, 2024


		To view usage per job, utilize the following command:
		To view the resource usage per job, you need to run on Midway2 before using either `accounts`:


		both of which include the jobs on both Midway2 and Midway3 partitions (e.g. `caslake` and `gpu`).

		The output is a table that lists the job IDs, partitions, number of CPU cores and SU consumptions. To further check the resource consumption of a job on Midway3, you need to log in to Midway3 and run `sacct -j` or `scontrol show job` with the job ID.

Improving the accounts usage command mention on midway2 and midway3 #214

Improving the accounts usage command mention on midway2 and midway3 #214

Conversation

ndtrung81 commented Aug 22, 2024

ndtrung81 commented Aug 22, 2024

ndtrung81 commented Aug 22, 2024

od101 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

od101 Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pedramesfahani left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ndtrung81 commented Aug 29, 2024

od101 Aug 23, 2024 •

edited

Loading