Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect display of number of jobs in bin for memory histogram (length of bar correct) #27

Open
tardigradus opened this issue Nov 8, 2023 · 1 comment

Comments

@tardigradus
Copy link

tardigradus commented Nov 8, 2023

The display of the number of jobs in a bin seems in some cases to be incorrect for the memory efficiency, although the length of the bar is correct:

--------------------------------------------------------                                                                                                                                                                                                                                                                                    
Job Information                                                                                                                                                                                                                                                                                                                             
ID: 123456                                                                                                                                                                                                                                                                                                                                
Name: abc                                                                                                                                                                                                                                                                                                                                
Cluster: acluster                                                                                                                                                                                                                                                                       
User/Group: auser/agroup                                                                                                                                                                                                                                                    
Requested CPUs: 1 cores on 1 node(s)                                                                                                                                                                                                                                                 
Requested Memory: 0.59G                                                                                                                                                                                                                                                              
Requested Time: 7-00:00:00                                                                                                                                                                                                                                                           
--------------------------------------------------------                                                                                                                                                                                                                             
Job Status                                                                                                                                                                                                                                                                           
COMPLETED: 221                                                                                                                                                                                                                                                                       
OUT_OF_MEMORY: 3                                                                                                                                                                                                                                                                     
PENDING: 1                                                                                                                                                                                                                                                                           
RUNNING: 768                                                                                                                                                                                                                                                                         
--------------------------------------------------------                                                                                                                                                                                                                             
--------------------------------------------------------                                                                                                                                                                                                                             
Finished Job Statistics                                                                                                                                                                                                                                                              
(excludes pending, running, and cancelled jobs)                                                                                                                                                                                                                                      
Average CPU Efficiency 98.91%                                                                                                                                                                                                                                                        
Average Memory Usage 0.90G                                                                                                                                                                                                                                                           
Average Run-time 2883.89s                                                                                                                                                                                                                                                            
---------------------                                                                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                                     
CPU Efficiency (%)                                                                                                                                                                                                                                                                   
---------------------                                                                                                                                                                                                                                                                
  0.00 -  10.00  [  0]                                                                                                                                                                                                                                                               
 10.00 -  20.00  [  0]                                                                                                                                                                                                                                                               
 20.00 -  30.00  [  0]                                                                                                                                                                                                                                                               
 30.00 -  40.00  [  0]                                                                                                                                                                                                                                                               
 40.00 -  50.00  [  0]                                                                                                                                                                                                                                                               
 50.00 -  60.00  [  0]                                                                                                                                                                                                                                                               
 60.00 -  70.00  [  0]                                                                                                                                                                                                                                                               
 70.00 -  80.00  [  0]                                                                                                                                                                                                                                                               
 80.00 -  90.00  [  0]                                                                                                                                                                                                                                                               
 90.00 - 100.00  [224]  ████████████████████████████████████████                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                                     
Memory Efficiency (%)                                                                                                                                                                                                                                                                
---------------------                                                                                                                                                                                                                                                                
  0.00 -  10.00  [0]                                                                                                                                                                                            
 10.00 -  20.00  [0]                                                                                                                                                                                            
 20.00 -  30.00  [0]                                                                                                                                                                                            
 30.00 -  40.00  [0]                                                                                                                                                                                            
 40.00 -  50.00  [0]                                                                                                                                                                                            
 50.00 -  60.00  [0]                                                                                                                                                                                            
 60.00 -  70.00  [0]                                                                                                                                                                                            
 70.00 -  80.00  [0]                                                                                                                                                                                            
 80.00 -  90.00  [0]                                                                                                                                                                                            
 90.00 - 100.00  [3]  ████████████████████████████████████████                                                                                                                                                  
                                                                                                                                                                                                                
Time Efficiency (%)                                                                                                                                                                                             
---------------------                                                                                                                                                                                           
  0.00 -  10.00  [224]  ████████████████████████████████████████                                                                                                                                                
 10.00 -  20.00  [  0]                                                                                                                                                                                          
 20.00 -  30.00  [  0]                                                                                                                                                                                          
 30.00 -  40.00  [  0]                                                                                                                                                                                          
 40.00 -  50.00  [  0]                                                                                                                                                                                          
 50.00 -  60.00  [  0]                                                                                                                                                                                          
 60.00 -  70.00  [  0]                                                                                                                                                                                          
 70.00 -  80.00  [  0]                                                                                                                                                                                          
 80.00 -  90.00  [  0]                                                                                                                                                                                          
 90.00 - 100.00  [  0]                                                             
--------------------------------------------------------                                                  

The problem seems to only occur when all the values are in a single bin.

Note: I have tweaked the format for the bin labels in hist.py from termplotlib, but whether I do or not does not affect the miscounting.

@tardigradus
Copy link
Author

Here is another example. The incorrect counting seems to be related to the number of jobs which failed with OUT_OF_MEMORY:

--------------------------------------------------------
Job Information
ID: 123456
Name: abc
Cluster: cluster
User/Group: auser/agroup
Requested CPUs: 1 cores on 1 node(s)
Requested Memory: 0.59G
Requested Time: 7-00:00:00
--------------------------------------------------------
Job Status
COMPLETED: 858
OUT_OF_MEMORY: 42
PENDING: 1
RUNNING: 768
--------------------------------------------------------
--------------------------------------------------------
Finished Job Statistics
(excludes pending, running, and cancelled jobs)
Average CPU Efficiency 99.38%
Average Memory Usage 0.90G
Average Run-time 13271.27s
---------------------

CPU Efficiency (%)
---------------------
  0.00 -  10.00  [  0]
 10.00 -  20.00  [  1]
 20.00 -  30.00  [  0]
 30.00 -  40.00  [  0]
 40.00 -  50.00  [  0]
 50.00 -  60.00  [  0]
 60.00 -  70.00  [  0]
 70.00 -  80.00  [  0]
 80.00 -  90.00  [  0]
 90.00 - 100.00  [899]  ████████████████████████████████████████

Memory Efficiency (%)
---------------------
  0.00 -  10.00  [ 0]
 10.00 -  20.00  [ 0]
 20.00 -  30.00  [ 0]
 30.00 -  40.00  [ 0]
 40.00 -  50.00  [ 0]
 50.00 -  60.00  [ 0]
 60.00 -  70.00  [ 0]
 70.00 -  80.00  [ 0]
 80.00 -  90.00  [ 0]
 90.00 - 100.00  [42]  ████████████████████████████████████████

Time Efficiency (%)
---------------------
  0.00 -  10.00  [900]  ████████████████████████████████████████
 10.00 -  20.00  [  0]
 20.00 -  30.00  [  0]
 30.00 -  40.00  [  0]
 40.00 -  50.00  [  0]
 50.00 -  60.00  [  0]
 60.00 -  70.00  [  0]
 70.00 -  80.00  [  0]
 80.00 -  90.00  [  0]
 90.00 - 100.00  [  0]
--------------------------------------------------------

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant