yarplogger – new feature requested: export multiple .txt files with a single button

[sgiraz]

Is your feature request related to a problem? Please describe.

Attaching .log files from yarplogger helps diagnose issues in icub-tech-support (e.g., robotology/icub-tech-support#1742 (comment)). However, deciphering them is tricky due to numerical log levels that appear when the log is exported using the Save to file button (⬇️). For better readability, it would be great to export logs with text-based levels (DEBUG, INFO, WARNING, etc..).

5
<#STRING_START#>from BOARD 10.0.1.2 (left_arm-eb2-j0_1) time=17s 348m 664u : SYS: a service has detected that some CAN boards have stopped transmission. Type of service category is eomn_serv_category_all. Lost CAN boards are on (can1map, can2map) = ([ 2  ], [  ]). Time since last contact: 0 [ms]<#STRING_END#>
NULL-DATA
22.953496

Describe the solution you'd like
As you can see to find this error I have to use a text editor with regex features looking for 5\n in case I want to find the errors.
It would be nice, especially for maintainers technicians, to have the ERROR string instead of the numeric value (as it happens for the log of the yarprobotinterface.

ERROR
<#STRING_START#>from BOARD 10.0.1.2 (left_arm-eb2-j0_1) time=17s 348m 664u : SYS: a service has detected that some CAN boards have stopped transmission. Type of service category is eomn_serv_category_all. Lost CAN boards are on (can1map, can2map) = ([ 2  ], [  ]). Time since last contact: 0 [ms]<#STRING_END#>
NULL-DATA
22.953496

Of course, this is the specific example for the error case, but we should be able to do the same for all the log levels

Describe alternatives you've considered
Without this feature, we have to use the regex 5\n (where 5 corresponds to the ERROR).

Additional context

Steps to reproduce:

1. Open the yarplogger
2. Generate a log
3. Export the log->export current log to text file
4. Export the same log with file->Save log session ⬇️
5. Open the logs with a text editor and see the differences

@Nicogene @martinaxgloria @simeonedussoni @pattacini

[randaz81]

@sgiraz

• the .txt file "export current log to text file" is a text file and already has the long information in the format you want.
• the .log file "save log session" is a binary file, it should be not opened by humans with a text editor (generally speaking, it should be zlib compressed to allow storage of large logs on disk). It is designed to be opened by another yarploagger and perform searches through it using the built-in filter.

[sgiraz]

Hi @randaz81,

While I understand the purpose of raw log files, users often share them instead of the human-readable ones. To make them easier to understand, I suggest the following 2 options:

1. (maybe the easiest one) Modify yarplogger to replace the numerical value (e.g. 5) with its string representation (e.g. ERROR) when generating the .log.
2. move the export of the .log file within the context menu and replace the action of Save log session ⬇️ button with the exporting of the humand readable .txt file.

[randaz81]

users often share them instead of the human-readable ones

That's fine, you just need to open the provided .log with the yarplogger and inspect it from the gui...

Or I can add a new button to iterate on all the logged processes and export a .txt for each of them. But I still think that the first solution is better.

[pattacini]

Hi @randaz81

Just a comment on this specific point:

That's fine, you just need to open the provided .log with the yarplogger and inspect it from the GUI...

While we can easily assume that everyone has a text editor installed, it may happen that someone who needs to consume the text log doesn't have the yarplogger installed.

Therefore, if the text log exposes meaningful tags like ERROR, WARNING, and the like, instead of ids, this would improve readability.

[simeonedussoni]

IMO, the important piece of information is that the option with the big red button on the GUI is the one exporting cyphered logs.

an unexperienced user may easily use this one instead of the longest menu>export>export current log to text

moreover, as I experienced recently, the text editor (be it notepad++ or Visual Code or else) is usually more searchable when looking at some particular error (like the wrist bug for instance)

[randaz81]

@Ugo, that was not the purpose of a binary .log file. (maybe should be called .ycb?) It is not a matter of replacing a number with a string, the whole file is messed up, since it mixes the logs from multiple processes belogning to the same session.

.log -> a session of logs (multiple processes, not human readable)
.txt -> a log of a single process (human readable)

This is my proposed solution:

I can add a new button to iterate on all the logged processes and export a .txt for each of them.

Regarding:

While we can easily assume that everyone has a text editor installed, it may happen that someone who needs to consume the text log doesn't have the yarplogger installed.

I partially disagree on this, it's true that you can edit a xml file with notepad, but only if it is simple and not deeply nested. If you need to perform search/replace operations, ad-hoc xml processors perform the same job in a better way. In a similar way, the logger gui provides you tools to search things in the log much better than a notepad. For example, you can filter out all messages of a specific level (warning/errors) or containing specific words. So my recommendation is to use the gui, it was specifically designed to help you :-)

[randaz81]

@simeonedussoni big red button?!??

[simeonedussoni]

@simeonedussoni big red button?!??

actually, the one corresponding to the "save" action. not the actual red one which stops everything

[randaz81]

Ah ok. Indeed, the caption of that button is "save session" and it is meant to take a binary snapshot of all logged processes, both running and terminated, as mentioned before, so you can see what was running when an error occurred. This is not a piece of information which can be easily extracted from a text file as you need to crosscheck details of logs coming from different processes.

[randaz81]

Issue renamed.

[randaz81]

@simeonedussoni

moreover, as I experienced recently, the text editor (be it notepad++ or Visual Code or else) is usually more searchable when looking at some particular error (like the wrist bug for instance)

can you better clarify the issue? is this not giving the expected result? It should support regex too.

[pattacini]

Hi @randaz81

that was not the purpose of a binary .log file. (maybe should be called .ycb?)

It wasn't clear to me that a .log file is not a text file but a binary, given its content.
Perhaps, it may be beneficial to point it out, somehow, as you said.

For the remaining comments, I have the GUI and it's not a problem for me. However, I know other cases where some¹ simply do not have YARP installed and sometimes are involved in debugging teams. Since people tend to share .log files in maintenance issues², I think that what was reported above was in the direction of making their lives easier. That's it.

Having a way to export multiple files in text format would be of help.

Footnotes

1. For example, managers or technicians who are not into SW. ↩

2. Probably, it's not clear enough that people need to share other formats? ↩

[simeonedussoni]

Ah ok. Indeed, the caption of that button is "save session" and it is meant to take a binary snapshot of all logged processes, both running and terminated, as mentioned before, so you can see what was running when an error occurred. This is not a piece of information which can be easily extracted from a text file as you need to crosscheck details of logs coming from different processes.

Yes, I understood this. It would be helpful to have a similar button to export the .txt file for one or more processes as stated by @pattacini
And maybe, put somewhere in the yarplogger documentation a sentence like "use THIS to save and share text files and THAT for save and share binary ones (e.g. to be loaded and parsed using yarplogger itself)"
It can be even a context-sensitive help when pressing the buttons, with the possibility to choose between the two options. I leave the choice to the maintainers

PS: BTW, I have YARP-eries only on the Linux Laptop I used for wrist debug etc, and most of the time I look at logfiles with Notepad++ or Studio Code on my workstation. By chance I always saved the logfiles with the menu procedure since I am usually interested in only the running process and not in the dead ones and I just discovered the save sessione when @sgiraz showed me the tricky message codes