Automatic log-file rotation seems to be missing #173
Comments
|
That is a very good idea, we state to set this back to info or less in documentation. Can you point to the documentation of venus logging? Rhank you! |
|
Hi, what we do else where in Venus OS, is make apps and services, python, c or whatever, write their logging info to stdout and stderr. So no in-app writing to log files. Then, we use hope that helps! ps. important: in your next release ,can you add a line of code that just deletes that current.log and any other log files that your code might have generated? I'm sure doing that will help a lot of people. Or even better, as part of the update/install script that you might have do that: installing a newer version of dbus-opendtu on that system I was logged into is most likely impossible now, since data partition is full.. |
|
I have a different meaning. First of all, the user has a bigger problem than the log. I have been running this service for a year with occasional connection errors. My log is 62MB after a year. Have a look:
I think what we could do is give an option "logging=none" and write errors to stderr instead. This is manageable then. I also like your idea of clearing the log from time to time. Maybe check the size? Over 100Mb? |
|
Hi,
I have a different meaning on that. The user had a small problem, ie. something related only to, I suppose misconfiguration of, or a bug in-, dbus-opendtu. But because of how the logging of dbus-opendtu works, this small problem has escalated itself into near-bricking the whole GX device. And moreover it resulted in the user thinking there might be some bug per Venus OS v3.31 and that resulted in a post on our forum with this title: After Cerbo GX update to V3.31 I now get Error #67 BMS connection lost. Please think about what seeing such message does to our developers and my-self; as well as anyone else reading it.
100Mb is far too much. On that system, here is the number of services, its around 60. Sixty times 100Mb is 6 gigabyte. The data partition is only 1 Gb and has to do more than just store log files. Especially ancient logfiles. I'd really prefer that any open source project uses our method + configuration; (or if it exists a better one). Added advantage that I see of all doing it the same is that its more efficient, easier for anyone to help each other, and so forth. here is how that works: Contents of
What it does is make 4 files, and each 25.000 bytes large. So 100kB in total max; and in practice it varies between 75kB and 100kB. I hope that this in more detail explains why I'm concerned about this. |
|
ps. one thing that we have on our backlog that will help people once they are in trouble is this one: victronenergy/venus#1234 |
|
thank you, log rotation was already in, we have now 100kb max and setting to CRITICAL for quick fix, and will later on change to your log file handling #44 |
|
Is there a reason why you are not using the system build in logging and log rotation? It is as easy as creating a file with three lines. Just create a folder #!/bin/sh
exec 2>&1
exec multilog t s25000 n4 /var/log/dbus-opendtuThats it. Everything which is normally printed to the console is written to the log. See https://github.com/Louisvdw/dbus-serialbattery/tree/master/etc/dbus-serialbattery/service for an example. The handlers should then not be needed anymore in this code: Lines 43 to 58 in dfcc91d |
The granularly does not change, only the path and the fact that Venus OS handles log rotation and all other. Everyone would expect a log file to be found in the default log folder Give it a try :-) |
|
very good and thank you! that works One last question, I see that all log levels are now written, how are you handling, that only the log level required is written? To avoid to much logging, when not necessary? |
|
if I do not remove this part, we can limit the log detail level, will rework the PR and built this in. Thank you again, very helpful! |
|
Built into PR #174 - waiting for review |
|
Unfortunately I don't even know where the documentation is, but GitHub Copilot and ChatGPT help you, if you want to customize the log settings. You handle the log level with therefore in your case yo can use instead of Lines 43 to 58 in dfcc91d I do it this way: |
|
Thank you, but the documentation about how venus handles system log rotation. With the service and log folder? |
This shell script does the following:
In summary, this script is used to manage log files for some process, adding timestamps to each log entry and limiting the size and number of log files. Therefore it's not directly Venus OS which handles the log rotation but the |
|
Ok, I get that as well, sorry maybe to let to clearly articulate wanted to know where the best practise documentation is for how to properly write scripts that work best with venus os and in specific the piece that says do log rotation like you showed? anyway thank you it works and will hopefully soon merge into main.. |
|
I found also none documentation. My path was to get a Victron written script, check how it's build and running. Then I build my script the same way. |
|
Thank you @mr-manuel for your help and @henne49 for your work. I've found this one: |
|
Good morning!
We have this document: https://github.com/victronenergy/venus/wiki/howto-add-a-driver-to-Venus . I added an initial section about logging to it. |
|
Ps. all of you are quick! 👍 |
|
Thank you @mpvader for this link, that is very useful, also to review and align to best practise. |
else errors from 'run' script are not logged
else errors from 'run' script are not logged
else errors from 'run' script are not logged
use standard logging, similar to henne49/dbus-opendtu#173
else errors from 'run' script are not logged
use standard logging, similar to henne49/dbus-opendtu#173
Hi,
I tried to help a user, and encountered that his data partition is 100% full. Can I suggest that you change the project to prevent that, and at a minimum always have log file rotation? Perhaps just use the same type of logging that we do in venus os; it takes care of rotation.
See also: https://community.victronenergy.com/questions/286286/after-cerbo-gx-update-to-v331-i-now-get-error-67-b.html?childToView=287794#comment-287794
I checked his config.ini, logging is set to ERROR.
And the current.log file, 1.134.657.536 bytes large, is full of errors, like below one.
Thanks!
The text was updated successfully, but these errors were encountered: