Solaris support

[giampaolo]

From jlo...@gmail.com on February 23, 2009 14:24:10

psutil support for Solaris OS

Original issue: http://code.google.com/p/psutil/issues/detail?id=18

[giampaolo]

From jlo...@gmail.com on February 23, 2009 05:24:59

Labels: -Type-Defect Type-Enhancement OpSys-Solaris

[giampaolo]

From billiej...@gmail.com on February 23, 2009 05:38:24

Do we have a Solaris OS to test against?

[giampaolo]

From jlo...@gmail.com on February 23, 2009 12:32:31

No, we need to install one in a VM using Solaris x86 or find a Solaris box we 
use.  Or of course, someone willing to take up the porting effort, since I'm 
doing OS X, Windows, and FreeBSD as it is ;)

[giampaolo]

From billiej...@gmail.com on February 23, 2009 17:56:55

I know, we'd need more C coders. =)
I could take the effort of installing a VM with Open Solaris and provide SSH 
for you and Dave, but I'd postpone this Issue for another release other than 

[giampaolo]

From billiej...@gmail.com on March 09, 2009 11:42:32

Status: New

[giampaolo]

From g.rodola on June 19, 2010 11:06:57

I'm closing this out for now as it's unlikely we'll find time to look into this 

Status: WontFix

[giampaolo]

From g.rodola on September 24, 2010 16:40:43

Status: PostPoned

[giampaolo]

From justin.v...@gmail.com on January 24, 2012 13:33:25

I actually have working code for reading proc on Solaris 10+ and access to a 
lot of test Solaris servers.  There are only a few things that I know of that 
don't work with implementation such as getting the Environment Settings from 
the process and CPU utilization is untested. I am unfamiliar with psutil's code 
base, but I be able share my implementation in the next few days.  It is pure 
python and uses 'struct' to decode the information.  I have shared a limited 

[giampaolo]

From g.rodola on January 24, 2012 13:59:20

Ideally, the whole API should be implemented by using the doc as reference: 
Practically speaking, I know this is a huge amount of work, therefore a partial 
implementation including base functionalities (system cpu/memory, process name, 

I don't have any experience with Solaris platforms, but I assume that /proc 
As such, it is likely that different things must be done in C. Any 
resource/link talking about C/kernel development on Solaris is welcome (see 

If you need any help with grokking psutil code base just ask out.

Status: ReOpened

[giampaolo]

From jlo...@gmail.com on January 24, 2012 15:29:05

Is /proc always available on Solaris? I might be misremembering but I thought 
support procfs was optional on Solaris systems or not available on all 
architectures. If it is available that would save us a lot of headaches dealing 

[giampaolo]

From justin.v...@gmail.com on January 24, 2012 16:38:44

I know on Solaris 10+ it is available on both Sparc and Intel architectures.  A 
system admin can disable it, but that issue exists on linux as well.  A simple 

if platform.system() != 'SunOS':
    raise OSError("Sorry, only SunOS is supported")

_sunos_version = int(platform.release().split('.')[-1])
if _sunos_version < 10:
    raise OSError("Sorry, version %d of SunOS is not supported" % 

if not os.listdir('/proc'):
    raise OSError("Sorry procfs is not available on your system")

[giampaolo]

From justin.v...@gmail.com on January 24, 2012 16:41:31

Over the next couple of days I will see if I can take the code that I have and 
make it fit the psutil api.  I will attach a patch when I have something to 

[giampaolo]

From jlo...@gmail.com on January 24, 2012 18:31:11

Excellent, thanks. It's been a while since I worked on Solaris with any 
regularity and I seemed to recall not having /proc available on our systems. 
That should vastly simplify porting efforts and it'd be great to add Solaris as 

[giampaolo]

From justin.v...@gmail.com on January 28, 2012 17:29:51

DroneD was released on Thursday and can make use of psutil for process 
management.  It also has some old platform support that it provides, but I 
would like to remove. One of the url's below provides the full working Solaris 
implementation that I have had for a long time and it supports the interfaces 
that DroneD expects. The code is Apache 2.0 licensed and if there is any 
concern about that I can work with legal department at Orbitz to clear it (I 
can't speak for legal, but psutil's license is in our approved list). Ideally I 
would like to move this support to this project and just let DroneD rely on 

Solaris original implementation: 
https://github.com/OrbitzWorldwide/droned/blob/master/droned/lib/kitt/proc/SunOS.py
 How DroneD uses psutil: 
https://github.com/OrbitzWorldwide/droned/blob/master/droned/lib/kitt/proc/psutil_backend.py

https://github.com/OrbitzWorldwide/droned/blob/master/droned/services/systemstats.py
 Main Page: https://github.com/OrbitzWorldwide/droned/ Some Documentation: 

[giampaolo]

From g.rodola on January 30, 2012 10:03:40

Mmm the implementation looks quite complicated/messy.
Where does this stuff come from? 
https://github.com/OrbitzWorldwide/droned/blob/master/droned/lib/kitt/proc/SunOS.py#L62
You said this works on Solaris 10+. What about other Solaris variants?
Are you willing to start writing an actual patch for integration with psutil?
Once we have that we can start looking into the missing parts.

[giampaolo]

From justin.v...@gmail.com on January 30, 2012 10:09:48

No doubt the original implementation is a mess and was done when Solaris 10 was 
still fairly new, also I was new to python when it was written. I haven't 
tested it with anything lower that Solaris 10.  Yes I am planning on writing a 
native set of patches for psutil, I just wanted to make everyone aware of where 

[giampaolo]

From g.rodola on January 30, 2012 10:20:42

From http://en.wikipedia.org/wiki/Solaris_(operating_system)#version-history - 
- Solaris 11 (5.11): November 15, 2010
We should support these two only, both Sparc and Intel architectures. 
Let's ignore any other previous version.

[giampaolo]

From justin.v...@gmail.com on May 26, 2012 17:40:53

I am not quite ready to submit my Solaris support as a patch, but I am 
providing a link to a github repo that I am currently working on (see end of 
message).  I have one outstanding issue that I have to rework to support the 
method ``psutil.Process.get_memory_maps`` and of course I need feedback before 

My code passes ``psutil.test``.  It has feature parity with the linux 
implementation with one exception, which is get_connections.  I cannot see an 
easy way to get the network connections for a given process without using 

This code is written in cython (mostly for clarity of code) and in my opinion 
is pretty straightforward to read.  The patch I am planning to submit in the 
future will contain the cython source, generated C source, and with clear 
instructions on how to emit the C source from the cython source. 

[giampaolo]

From justin.v...@gmail.com on May 27, 2012 01:30:18

I was able to track down and fix the issue that was preventing  
``psutil.Process.get_memory_maps`` from working in my Solaris implementation.  
Would anyone like to review the code (over the next few weeks) so I can put 

[giampaolo]

From g.rodola on May 28, 2012 02:01:54

Hi Justin, and congrats for the great progress.

I'm probably missing something but since most of the work is done by parsing 
/proc why did you decide to use Cython instead of Python? Ideally, it would be 

As for process connections, I notice you managed to implement process's 
Are you sure /proc does not provide the necessary pieces to interface with 
My best guess is that /proc/PID/fd is supposed to contain both regular file and 
As a last resort we might use the lsof parser as we're currently doing for 
FreeBSD: 
https://code.google.com/p/psutil/source/browse/tags/release-0.4.1/psutil/_psposix.py#115
 ...or it that's too much trouble we can decide to drop connections support for 

[giampaolo]

From justin.v...@gmail.com on May 28, 2012 10:27:04

In my opinion Cython is more aesthetically pleasing than pure C and python. 
While my opinion is certainly subjective I think it makes the implementation 
very easy to understand.  A minor technical bonus is reference counting is 

The real reason I chose Cython was all of the C only API's on Solaris such as 
kstat (the only way to get at kernel statics such as disk-io counters).  Also 
alot of the proc structured files have vector pointers and I think is easier to 
perform the pointer arithmetic in C (I'm not even sure how I would do it with 
struct and python).   Yes I could have used ctypes, but then code would would 
Lastly, I have better things to do than write in C :).

Unfortunately I didn't see a concise way get the socket connections from the 
proc information.  Honestly I didn't spend a whole lot of time on this code 
(maybe 30 hours over a few months) so I may have missed something easy.I know 
we can get the information from dtrace, with limitations.  I was kind of hoping 
someone would see an elegant solution to this and propose a fix.  I could 
probably go through the netstat code on opensolaris and see how they do it.  I 

[giampaolo]

From justin.v...@gmail.com on May 28, 2012 10:27:59

Oh and thanks for looking at the code, I appreciate the feedback.

[giampaolo]

From justin.v...@gmail.com on May 28, 2012 14:42:14

get_connections will probably take me an hour or so to add when I have the time.

#reference. 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/cmd/ptools/pfiles/pfiles.c

[giampaolo]

From justin.v...@gmail.com on May 28, 2012 19:11:57

I just noticed the unittests.  I may write one for solaris in the future as 

jvenus@solar:~/Projects/svn/psutil-read-only/test$ 
-2.6 python test_memory_leaks.py
test_cmdline (__main__.TestProcessObjectLeaks) ... ok
test_create_time (__main__.TestProcessObjectLeaks) ... ok
test_get_connections (__main__.TestProcessObjectLeaks) ... ERROR
test_get_cpu_affinity (__main__.TestProcessObjectLeaks) ... skipped-ok
test_get_cpu_times (__main__.TestProcessObjectLeaks) ... ok
test_get_environ (__main__.TestProcessObjectLeaks) ... skipped-ok
test_get_memory_info (__main__.TestProcessObjectLeaks) ... ok
test_get_memory_maps (__main__.TestProcessObjectLeaks) ... ok
test_get_num_threads (__main__.TestProcessObjectLeaks) ... ok
test_get_open_files (__main__.TestProcessObjectLeaks) ... ok
test_get_threads (__main__.TestProcessObjectLeaks) ... ok
test_getcwd (__main__.TestProcessObjectLeaks) ... skipped-ok
test_gids (__main__.TestProcessObjectLeaks) ... ok
test_is_running (__main__.TestProcessObjectLeaks) ... ok
test_name (__main__.TestProcessObjectLeaks) ... ok
test_ppid (__main__.TestProcessObjectLeaks) ... ok
test_resume (__main__.TestProcessObjectLeaks) ... skipped-ok
test_status (__main__.TestProcessObjectLeaks) ... ok
test_terminal (__main__.TestProcessObjectLeaks) ... ok
test_uids (__main__.TestProcessObjectLeaks) ... ok
test_username (__main__.TestProcessObjectLeaks) ... skipped-ok
test_cpu_times (__main__.TestModuleFunctionsLeaks) ... ok
test_disk_io_counters (__main__.TestModuleFunctionsLeaks) ... ok
test_disk_partitions (__main__.TestModuleFunctionsLeaks) ... ok
test_disk_usage (__main__.TestModuleFunctionsLeaks) ... skipped-ok
test_get_pid_list (__main__.TestModuleFunctionsLeaks) ... ok
test_get_users (__main__.TestModuleFunctionsLeaks) ... ok
test_network_io_counters (__main__.TestModuleFunctionsLeaks) ... ok
test_per_cpu_times (__main__.TestModuleFunctionsLeaks) ... ok
test_phymem_usage (__main__.TestModuleFunctionsLeaks) ... ok
test_pid_exists (__main__.TestModuleFunctionsLeaks) ... skipped-ok
test_process_iter (__main__.TestModuleFunctionsLeaks) ... ok
test_virtmem_usage (__main__.TestModuleFunctionsLeaks) ... ok

======================================================================
ERROR: test_get_connections (__main__.TestProcessObjectLeaks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jvenus/Projects/svn/psutil-read-only/test/test_psutil.py", line 
    return fun(self, *args, **kwargs)
  File "test_memory_leaks.py", line 194, in test_get_connections
    self.execute('get_connections', kind='all')
  File "test_memory_leaks.py", line 41, in execute 
    self.call(function, *args, **kwargs)
  File "test_memory_leaks.py", line 97, in call
    obj(*args, **kwargs)
  File 
"/home/jvenus/Projects/svn/psutil-read-only/build/lib.solaris-2.11-i86pc-2.6/psutil/__init__.py",
    return self._platform_impl.get_connections(kind)
  File "_psutil_solaris.pyx", line 1293, in 
AccessDenied: (pid=1864)

----------------------------------------------------------------------
Ran 33 tests in 81.994s

FAILED (errors=1)
jvenus@solar:~/Projects/svn/psutil-read-only/test$

[giampaolo]

From g.rodola on May 29, 2012 12:08:12

Independently from how good/better Cython is, the current code base does not 
rely on it and introducing it as a 3th party dep or rewriting the existent code 

Please note that most of the work on Solaris is done by reading /proc. That 
means that, once we figure out a way to "decode" /proc fs, that can be easily 
I don't know whether the resulting Python code would be better or worse 
compared to Cython's, but that's not the point here (consider that none of us 

The only parts requiring C are 4 or 5 (e.g. disk/net io counters), and some of 
them such as disk partitions and system users are basically the same as on 

In summary, I would be more keen on seeing a progress in this direction:
- a patch consisting in a py module and a C extension
- figure out how to decode /proc fs from python and use python wherever possible
- use C as last resort

Owner: g.rodola

[giampaolo]

From g.rodola on May 29, 2012 18:08:37

I've just tried to look into this directly.
I think one possibility is to rewrite _update_info() and _update_status() [1] 
I'll keep going down this route and possibly come up with a preliminary patch.

[1] 
https://github.com/JustinVenus/psutil-solaris/blob/master/_psutil_solaris.pyx#L790

[giampaolo]

From justin.v...@gmail.com on May 29, 2012 19:55:18

@g.rodola If you want to port it from Cython to straight C I am more than happy 

[giampaolo]

From g.rodola on May 30, 2012 02:48:44

How would you do that exactly? By hand or by using cython (doesn't it produce a 
messy C code?)? Would we still have an intermediate .py module or would it be 
Perhaps you can show me how you would implement, say, get_process_ppid()?

In case you need it, attached is a patch which creates the base structure (py 

Attachment: init.patch

[giampaolo]

From justin.v...@gmail.com on May 30, 2012 06:11:43

I can test your patches on my Solaris boxes.  For get_connections[1], as with 
most interesting things about a process in Solaris the proc files only give you 
clues, we have to examine the address space (/proc/%(pid)d/as) for the 



1) Overview get_connections
#this is how we can determine that a fd is a socket.

    def get_connections(self, kind='inet'):
        # use fd for filedescriptor list
        base = '/proc/%d/fd' % (self._pid,)

        for fd in os.listdir(base):
            f = os.path.join(base, fd)
            stats = os.stat(f)
            #S_IFMT and S_IFSOCK are defined in "sys/stat.h"
            if stats.st_mode & S_IFMT != S_IFSOCK:
                continue
            #the file descriptor is a socket

#Then we will have to make a few ioctl calls to take control of the process
#and examine the address space as the following code examples show.  This will 
get us #what port, ip, we are listening to and bread crumbs for the protocol 
#of the connection.  I am still reading over the following code to see what 
needs to be done at minimum to support 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/lib/libproc/common/Psyscall.c#Psyscall

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/lib/libproc/common/pr_getsockname.c#pr_getsockopt

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/cmd/ptools/pfiles/pfiles.c#714
 2): example of examining address space 
https://github.com/JustinVenus/psutil-solaris/blob/master/_psutil_solaris.pyx#L1011

[giampaolo]

From justin.v...@gmail.com on May 30, 2012 06:31:20

I just noticed your (init.patch) patch won't work because, we cannot directly 
include "sys/procfs.h" for definitions because of a conflict with "pyconfig.h". 
 Basically what happens is pyconfig.h defines _FILE_OFFSET_BITS as 64 which is 
incompatible with a definition that is included in "sys/procfs_old.h" and this 

We could do some preprocesser magic by checking if that #define is set, save it 
as another #define, undef the original, include "sys/procfs.h", udef yet again, 
and restore the original definition.  It would be ugly, but should work around 


1) procfs.h python fun. 
http://old.nabble.com/-RFA-Python--Fix-procfs.c-build-failure-on-32bit-solaris-(_FILE_OFFSET_BITS)-p30284417.html

[giampaolo]

From g.rodola on May 30, 2012 11:16:32

That is why I used:

#undef _FILE_OFFSET_BITS  // XXX

That worked on Open Solaris 5.11 i386 (32 bit).

[giampaolo]

From justin.v...@gmail.com on May 30, 2012 14:52:07

I missed that, my apologies.

[giampaolo]

From g.rodola on May 31, 2012 05:02:20

I'm working on a patch including preliminary basic process info - the ones 
I should be able to post it later today.
If that's ok with you I can commit it on a separate branch and from there we 

[giampaolo]

From justin.v...@gmail.com on May 31, 2012 06:49:53

Sounds good to me.

[giampaolo]

From g.rodola on May 31, 2012 12:32:12

The new branch is at: https://psutil.googlecode.com/svn/branches/sunos I 

- r1315 (base files structure + different Process methods)
- r1316 (get_users())
- r1317 (disk_partitions())

I also gave you commit access against the repo.

I marked the missing parts with "TODO" comments here: 
https://code.google.com/p/psutil/source/browse/branches/sunos/psutil/_pssunos.py?spec=svn1317&r=1317
I can go on and port the kstat* related code tomorrow.
Maybe you can take a look at process connections and open files?

[giampaolo]

From g.rodola on June 01, 2012 10:54:44

Further progress: r1319 : system CPU times r1320 : system disk IO counters 
r1322 : process uids/gids r1324 : process threads r1325 : process open files 

What remains to be done:

- system network io counters
- process connections
- process memory maps
- process environ

[giampaolo]

From g.rodola on June 25, 2012 15:47:16

Issue 288 has been merged into this issue.

[giampaolo]

From g.rodola on August 16, 2012 21:08:26

Further progress: r1515 : process memory maps r1516 : network IO counters r1518 

We're almost there. The only things missing are:

- process connections
- system swap memory

I'm having a hard time implementing the latter (swap memory) and have no clue 
Any help is appreciated.

[giampaolo]

From justin.v...@gmail.com on August 17, 2012 18:39:31

Yeah the swap usage per process was a real pain to figure out, I had to
read the source for the Solaris process utils to get as far as I did.
Speaking of that, did I share the cython code that implemented swap usage
per process?  If I didn't let me know and I will send it to you.

Justin Venus

[giampaolo]

From g.rodola on August 17, 2012 18:59:34

Hi Justin, 
you did but what we're looking for is to emulate the output of "swap -l", which 
The funny thing is that "swap" source code is available, and also pretty easy 
to read, nevertheless I failed to pull out the same numbers: 
https://code.google.com/p/psutil/source/browse/branches/sunos/psutil/_psutil_sunos.c?spec=svn1534&r=1530#209
 As for now I wrote a parser for "swap -l" in order to pass tests: 
https://code.google.com/p/psutil/source/browse/branches/sunos/psutil/_pssunos.py?spec=svn1534&r=1531#51
 As for process_connections(): netstat source code seems to be the right place 
where to look but it's messy as hell. Alternatively "pfiles" cmdline utility 
uses a different approach, but unfortunately it doesn't provide TCP connection 
I'll figure something out eventually (at least I hope :P)...

[giampaolo]

From g.rodola on December 13, 2012 06:13:56

Status: Started

[giampaolo]

From ad...@cvalka.info on February 27, 2013 07:50:15

The feature is currently being crowdfunded 

[giampaolo]

From g.rodola on March 01, 2013 12:50:36

Thanks a lot for your contribution.

[giampaolo]

From g.rodola on March 02, 2013 05:06:04

New cset links after the SVN -> Mercurial migration: r1315 == revision 
5084c50904c7 - base files structure + different Process methods r1316 == 
revision cebf7c743bb4 - get_users() r1317 == revision a32cdf467dff - 
disk_partitions() r1319 == revision b621eb32a999 - system CPU times r1320 == 
revision 1db3a9d7e23e - system disk IO counters r1322 == revision 99a42f7ef404 
- process uids/gids r1324 == revision 87a84972a237 - process threads r1325 == 
revision 9ff85cfae688 - process open files r1326 == revision 53e4513a9629 - 
process cwd r1327 == revision c4d712a4d5b2 - system virtmem r1515 == revision 
dd9408eb8e5e - process memory maps r1516 == revision 69d94ebb34d4 - network IO 
counters r1518 == revision 99342730b261 - system virtual memory r1519 == 
revision cfc634ff9207 - process context switches r1520 == revision bcc64055e3f7 

[giampaolo]

From g.rodola on March 16, 2013 02:50:16

Issue 358 has been merged into this issue.

[giampaolo]

From g.rodola on April 09, 2013 06:36:16

Issue 363 has been merged into this issue.

[giampaolo]

From g.rodola on June 08, 2013 05:22:54

Ok, it seems that aside from some minor quirks I'm going to fix soon we now 
sunos branch merged back into trunk as of revision bca91a5e7264 .
Thanks to Justin Venus for having written this in Cython. 
Without that I doubt I would have been able to make it.

Status: FixedInHG
Labels: Milestone-1.0.0

[giampaolo]

From g.rodola on July 11, 2013 01:54:35

Status: Fixed