Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a stepLog API to reduce code duplication. #876

Closed
wants to merge 11 commits into from

Conversation

esohel30
Copy link
Contributor

@esohel30 esohel30 commented Jun 10, 2024

#845

This pull request introduces a new StepLogger API to simplify and standardize the logging and performance measurement process in the Pysa taint analysis codebase. By encapsulating common patterns of starting timers, logging information, and recording performance metrics, this API helps to reduce code duplication and improve maintainability.

Questions:

Is this the right Idea for the steplog API?

image

IF this general Idea is correct. Please take look at the newly created steplog.ml and stepLog.mli file and provide any feedback.

I will then implement the feedback and update the taintAnalysis.ml file and ask for feedback once again!

@facebook-github-bot
Copy link
Contributor

Hi @esohel30!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

Copy link
Contributor

@arthaud arthaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the right Idea for the steplog API?

Yes, that's the idea :)

Since this would only be used by Pysa, could we move this in source/interprocedural_analysis/taint/stepLogger.ml?

Also, no need to wrap it into a module StepLogger = struct, since the file is stepLogger.ml it will automatically be available as a module StepLogger.start/finish. We module StepLogger = struct it would stutter, i.e StepLogger.StepLogger.start/finish.


let start ~start_message ~end_message =
let timer = Timer.start () in
Log.info "%s" start_message;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually have extra ... for the first message

Suggested change
Log.info "%s" start_message;
Log.info "%s..." start_message;


let finish ~step_logger ~section ~integers () =
let time_in_seconds = Timer.stop_in_sec step_logger.timer in
Log.log ~section "%s: %.3fs" step_logger.end_message time_in_seconds;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we always use Log.info, no need for section :)

Suggested change
Log.log ~section "%s: %.3fs" step_logger.end_message time_in_seconds;
Log.info "%s: %.3fs" step_logger.end_message time_in_seconds;

@esohel30
Copy link
Contributor Author

I updated the code according to the feedback. Please review again!

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

Copy link
Contributor

@arthaud arthaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!
Could you now start using this in taintAnalysis.ml?
This should be in the same PR IMO, as a way to test it. Otherwise we would be pushing code that's never used.

@@ -0,0 +1,24 @@
open Core
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove source/stepLogger.ml and .mli?

@esohel30
Copy link
Contributor Author

Finished adding steplog api to places where applicable in taintAnalysis.ml to reduce code duplication

source/interprocedural_analyses/taint/taintAnalysis.ml Outdated Show resolved Hide resolved
source/interprocedural_analyses/taint/taintAnalysis.ml Outdated Show resolved Hide resolved
source/interprocedural_analyses/taint/taintAnalysis.ml Outdated Show resolved Hide resolved
source/interprocedural_analyses/taint/stepLogger.ml Outdated Show resolved Hide resolved
source/interprocedural_analyses/taint/stepLogger.ml Outdated Show resolved Hide resolved
@arthaud
Copy link
Contributor

arthaud commented Jun 19, 2024

Thanks, that's mostly it!
We just have to make sure the end messages stay the same as before because we use those internally for statistics and monitoring.

esohel30 and others added 6 commits June 24, 2024 02:35
Co-authored-by: Maxime Arthaud <maxime@arthaud.me>
Co-authored-by: Maxime Arthaud <maxime@arthaud.me>
Co-authored-by: Maxime Arthaud <maxime@arthaud.me>
Co-authored-by: Maxime Arthaud <maxime@arthaud.me>
Co-authored-by: Maxime Arthaud <maxime@arthaud.me>
@esohel30
Copy link
Contributor Author

All feedback has been applied and code is ready for a final review. I double checked but as there is a lot of code I probably missed a thing or two again. Sorry in advanced and thank you for the great feedback! I am really learning a lot and feeling more confident.

@facebook-github-bot
Copy link
Contributor

@arthaud has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

~timer
()
in
StepLogger.finish ~step_logger ~integers:[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't compile, it should be:

Suggested change
StepLogger.finish ~step_logger ~integers:[];
StepLogger.finish ~integers:[] step_logger;

I will fix it myself so we don't take another week to merge this.
In the future, please run make test before submitting :)

~name:"Parsed taint models"
~phase_name:"Parsing taint models"
~timer
~integers:["models", Registry.size models; "queries", List.length queries]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep this too.

@facebook-github-bot
Copy link
Contributor

@arthaud merged this pull request in 847c396.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants