Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

383 custom data logger #523

Draft
wants to merge 15 commits into
base: develop
Choose a base branch
from
Draft

383 custom data logger #523

wants to merge 15 commits into from

Conversation

js-xc
Copy link
Contributor

@js-xc js-xc commented Sep 24, 2024

No description provided.

@js-xc
Copy link
Contributor Author

js-xc commented Sep 24, 2024

Open:

  • logged data is should also be zipped in result (for download from agents and also for storing from MC)
  • unit tests
  • should we add a delimiter to make sure we don't blow up our machines with logged data?
  • should be do any kind of csv sanity checks?
  • checks on scopename and extension according to system requirements for filenames (implicitly done, throws IOException if requirements are not met)
  • ...WIP

@jowerner
Copy link
Contributor

Optional feature: Before/while adding a custom log file to the zip archive, count the lines of the input file and remember the total number per scope for later display in the report as a new table column.

Comment on lines +132 to +138
if (file.getType() == FileType.FOLDER && !XltConstants.CONFIG_DIR_NAME.equals(file.getName().getBaseName().toString()))
{
for (final FileObject fo : file.getChildren())
{
findLogs(fo);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to avoid combining the file type check with another condition. Please move the second condition to a new if inside the body of this if.

Comment on lines +172 to +176
final boolean isCompressed = "gz".equalsIgnoreCase(file.getName().getExtension());

InputStream in = isCompressed ?
new GZIPInputStream(file.getContent().getInputStream(), 1024 * 16) : //TODO is my data EVER zipped? it should be, but how does it happen?
file.getContent().getInputStream();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Custom data is not gzipped at the moment, so we can remove this code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the overall goal was to have all data zipped, so instead if removing this we should get the custom data zipped?


FileObject results = ((ReportGeneratorConfiguration) getConfiguration()).getResultsDirectory();

baseDir = results.getPath();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When creating the report from a result archive, this won't work as the results directory is inside the archive.

To reproduce, simply create a zip file from your results directory and pass that to the report generator.

}
catch (IOException e)
{
System.err.println("Failed to walk file tree searching for custom data logs. Cause: " + e.getMessage());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't print anything to the console. It will likely be overwritten by the progress bar updates anyway. If the error is tolerable or if it is just a debug message, write the information via a regular logger (e.g. XltLogger.runtime) to the log file. If the error is fatal, rethrow it wrapped inside an XltException for the report generator to handle that case.

Same for all other usages of System.err.println in this class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to log custom data to a file and have that file accessible in the report
2 participants