RotatingFileHandler Logging fixes

[whitmo]

Currently if you spawn multiple processes and wish to log to the same file, the filedescriptors will not be changed as the files are filled up and renamed leading to an explosion of inconsistent behavior once a file size limit is reached (every process will again check the original file, and try to spawn a new file to which to log).

These commits create a descriptor that manages the 'stream' from a list of open file handles stored in a module global insure across multiple RFHandlers only one set of rotated files exists per filename. Tests included.

[mnaberez]

We should merge this or a variation of it before the next release.

[whitmo]

ok, will do my due dilligence.

-w

On Thu, Nov 10, 2011 at 11:53 PM, Mike Naberezny
reply@reply.github.com
wrote:

We should merge this or a variation of it before the next release.

---

Reply to this email directly or view it on GitHub:
#10 (comment)

<=>
david "whit" morriss

 "If you don't know where you are,
  you don't know anything at all"

  Dr. Edgar Spencer, Ph.D., 1995

[mnaberez]

This was released in 3.0b1 last week but looks like we missed a bug (see #130 and #121) that causes supervisord to crash when supervisorctl reload is issued. I think this is because when a stream is closed, it is left in the open_streams dictionary.

[mcdonc]

Houston, I think we have a problem even if we fix the file reference counting issue:

  [program:theprogramname]
  command=/bin/cat /dev/urandom
  stdout_logfile=/tmp/theprogramname.log
  stdout_logfile_maxbytes=1MB
  stdout_logfile_backups=10

  [program:theprogramname2]
  command=/bin/cat /dev/urandom
  stdout_logfile=/tmp/theprogramname.log
  stdout_logfile_maxbytes=2MB
  stdout_logfile_backups=5

That configuration can't be serviced properly when streams are shared because it's logically impossible to have the same file max out at both 1MB and 2MB or have both 10 backups and 5. Even if the maxbytes and the backups numbers are the same, the doRollover method of the handlers attached to each logger is going to clobber files in a bizarre way.

Maybe the feature added by this pull request should be reverted and we should instruct folks to use the recently added syslog logger if they want to multiplex multiple outputs into a single file.

[mnaberez]

Maybe the feature added by this pull request should be reverted and we should instruct folks
to use the recently added syslog logger if they want to multiplex multiple outputs into a single file.

I think this may be the right thing to do.