System memory functions big refactoring

[giampaolo]

From g.rodola on July 24, 2012 19:14:54

As of now psutil provides 2 functions to extract system memory stats:

- phymem_usage()
- virtmem_usage()

...providing a (total, free, used, percent) namedtuple.

There are different problems with these 2 functions:

1 - first of all they are insufficient: it is necessary to provide 
platform-specific types of memory. For example, on BSD and OSX we have active, 
inactive, wired and cached memory, which are also important (note: top shows 
them). Same for Linux, where we have cached and buffers memory (free shows 
them). These stats are currently not exposed, except for "buffers" and "cached" 
on Linux, provided by two extra functions.

2 - on both UNIX and Windows there's a concept of 'free' and 'available' 
memory; as of right now we only provide 'free' but again, that's insufficient.
What really matters is the available memory: 
http://www.sevenforums.com/performance-maintenance/114504-diff-between-available-free-physical-memory.html
 ...and that's what need to be used to monitor actual memory usage in a cross 
platform fashion.

3 - virtmem_usage() is inconsistent: on Windows and BSD it returns information 
about virtual memory while on OSX and Linux return stats about the swap memory.


=== REFACTORING PROPOSAL === 
http://blog.zabbix.com/when-alexei-isnt-looking/#vm.memory.size 
http://www.zabbix.com/documentation/2.0/manual/appendix/items/vm.memory.size_params

http://www.zabbix.com/documentation/2.0/manual/appendix/items/supported_by_platform?s[]=swap
 The links above provide useful suggestions as to how we can organize memory 
functions in psutil, and that's what I've been doing in the memory branch: 
https://code.google.com/p/psutil/source/browse/#svn%2Fbranches%2Fmemory 
Basically we'd have two functions:

= psutil.virtual_memory() =

We'll have 5 fields available on all platforms:
- total
- available
- percent
- used
- free

...plus others (e.g. buffers, cached, wired, etc...) depending on what platform we're on.
Again, monitoring apps are interested in 'available' memory.

= psutil.swap_memory() =

It'll have 5 fields:

- total
- used
- free
- percent
- sin   (no. of bytes the system has swapped in from disk (cumulative))
- sout  (no. of bytes the system has swapped out from disk (cumulative))


=== FUNCTIONS NEED TO BE DEPRECATED ===

- phymem_usage()
- virtmem_usage()
- phymem_buffers()
- cached_phymem()

These can be deprecated in 0.6.0 and scheduled for removal in 1.0.0 or later.
Also we have:

- psutil.avail_phymem()
- psutil.used_phymem()
- psutil.total_virtmem()
- psutil.avail_virtmem()
- psutil.used_virtmem()

...which were were already deprecated in 0.3.0.
They also need to go away.

Original issue: http://code.google.com/p/psutil/issues/detail?id=311

[giampaolo]

From jlo...@gmail.com on July 24, 2012 10:26:17

Couple questions that come to mind: 

1) Are we interested in providing aliases for platform specific memory names? 
e.g. cached/buffer on Linux, wired/inactive on OS X etc. 

2) Are we going to expose directly things like wired/active/inactive amounts 
for OS X or only the calculated results showing "available" and "free" memory stats

I see the value in cross-platform design but also imagine if you're developing 
psutil apps specific to a certain platform you may wish to look at 
platform-specific memory info as well.

[giampaolo]

From g.rodola on July 24, 2012 10:50:58

Nope, no aliases. We'll just have a namedtuple with different fields depending 
on the OS with the first 5 fields (total, available, percent, used, free) 
always available on all platforms. Basically it's the same thing we did with 
psutil.cpu_times().

On OSX we'll also have wired/active/inactive amounts.

The 'cross-platform design' will be guaranteed by the first 5 values, with 
'available' field being the most important as it'll represent the effective 
amount of available memory.

[giampaolo]

From g.rodola on July 24, 2012 11:31:17

Issue 256 has been merged into this issue.

[giampaolo]

From jlo...@gmail.com on July 24, 2012 12:05:42

Ah ok I see.. I missed that part about these being "always available" and then 
adding fields for the platform-specific ones. That makes sense and it's best of 
both worlds.

[giampaolo]

From g.rodola on July 24, 2012 14:42:22

Committed in r1498 for all platforms.

Status: FixedInSVN

[giampaolo]

From g.rodola on August 13, 2012 09:25:14

Fixed in version 0.6.0, released just now.

Status: Fixed
Labels: -Milestone-1.0.0 Milestone-0.6.0

[giampaolo]

From g.rodola on March 01, 2013 14:32:32

Issue 256 has been merged into this issue.

[giampaolo]

From g.rodola on March 02, 2013 04:07:04

Issue 256 has been merged into this issue.

[giampaolo]

From g.rodola on March 02, 2013 04:11:46

Updated csets after the SVN -> Mercurial migration: r1498 == revision ???