Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed-up analysis time with thread pool #1493

Open
guwirth opened this issue Jun 9, 2018 · 2 comments
Open

Speed-up analysis time with thread pool #1493

guwirth opened this issue Jun 9, 2018 · 2 comments
Labels
discussion

Comments

@guwirth
Copy link
Collaborator

guwirth commented Jun 9, 2018

As an user I like to have shorter cxx-plugin scanner times.

Hint:
At the moment we are using only one thread for analysis. Most of the current analysis time is spend in CxxSquidSensor.execute

Running scanner.scanFiles in a thread pool could speed-up the analysis.

Sample:
https://github.com/SonarSource/sonar-python/pull/106/files
ivangalkin added a commit to ivangalkin/sonar-cxx that referenced this issue Jun 25, 2018
@ivangalkin
introduce CxxSquidProfiler for CxxSquidSensor
a1d6558 
* if property set, print duration of each SquidCheck to `sonar.cxx.squidprofiler.out=<abs path>`
* currently only duration is logged
* for parallel execution we can log start/stop time also, in order to check
  the utilization level (see SonarOpenCommunity#1493)

* output: semi-JSON format for later python analysis (e.g. in http://jupyter.org/)
  `{ "visitor": "%s", "source_file": "%s", "duration_ms": "%d" }`
@ivangalkin ivangalkin mentioned this issue Jun 25, 2018
Closed
ivangalkin added a commit to ivangalkin/sonar-cxx that referenced this issue Jun 27, 2018
@ivangalkin
introduce CxxAstVisitorProfiler for CxxSquidSensor
7d800dd 
* introduce the hidden property sonar.cxx.squidprofiler.out=<path>
* if property set, decorate each AstVisitor (all squid checks, CPD, highlighting etc), so that it tracks the elapsed time per source file
* dump this duration as well as other invocation details to <path> in JSON format
* currently only duration is logged
* for parallel execution we can log start/stop time also, in order to check the CPU utilization level (see SonarOpenCommunity#1493)
* output: semi-JSON format for python analysis (e.g. in http://jupyter.org/)
  `{ "visitor": "%s", "source_file": "%s", "duration_ms": "%d" }`
@ivangalkin ivangalkin mentioned this issue Jul 8, 2018
Merged
@ivangalkin ivangalkin mentioned this issue Dec 24, 2018
Closed
@guwirth guwirth changed the title US: Speed-up analysis time with thread pool Speed-up analysis time with thread pool Dec 3, 2020
@neontty
Copy link

neontty commented Jun 29, 2021

I would also like to see this feature. Our project spends 95% of its time in the CxxSquidSensor. I can try to work on this if it is still desired.

@guwirth
Copy link
Collaborator Author

guwirth commented Jun 30, 2021

Hi @neontty,

help is always welcome!

Think there are still many things possible to improve speed:

  • precompiled header support
  • pragma once support
  • thread pool support
  • …

I’m not sure if preprocessing and AST creation can be paralyzed? Maybe you have to look first to https://github.com/SonarSource/sslr if it is possible.

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discussion
Milestone
No milestone
Development

No branches or pull requests
2 participants
@guwirth @neontty