Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UtilityAnalyzer: Use RegisterCompilationStartAction #6576

Merged
merged 32 commits into from
Feb 23, 2023

Conversation

martin-strecker-sonarsource
Copy link
Contributor

Fixes #6558

@martin-strecker-sonarsource
Copy link
Contributor Author

cc @pavel-mikula-sonarsource as this may conflict with your AnalyzerContext refactoring.

@costin-zaharia-sonarsource
Copy link
Member

costin-zaharia-sonarsource commented Dec 21, 2022

Hi @martin-strecker-sonarsource. What was the observed change in analysis time?

@martin-strecker-sonarsource
Copy link
Contributor Author

Unscientific comparison:

image

@martin-strecker-sonarsource
Copy link
Contributor Author

martin-strecker-sonarsource commented Dec 21, 2022

Here is a comparison of an AnalyzerRunner.exe run for a single UtilityAnalyzer:

AnalyzerRunner.exe C:\Projects\sonar-dotnet\analyzers\packaging\binaries C:\Projects\DeepSast\CsvHelper\CsvHelper.sln /a TokenTypeAnalyzer

Master PR
Runtime 27s 13s
Allocations 1107 MB 948 MB
TokenTypeAnalyzer 24.1s 7.6s

Current master:

Pausing 5 seconds before starting analysis...
Running analyzers for CsvHelper(net6.0)
Running analyzers for CsvHelper(net5.0)
Running analyzers for CsvHelper(netstandard2.1)
Running analyzers for CsvHelper(netstandard2.0)
Running analyzers for CsvHelper(net47)
Running analyzers for CsvHelper(net45)
Running analyzers for CsvHelper.Tests
Running analyzers for _build
Running analyzers for CsvHelper.Performance
Running analyzers for CsvHelper.Website
Found 0 diagnostics in 27228ms (1161163400 bytes allocated)
Statistics for TokenTypeAnalyzer:
Concurrent:                     False
Execution time (ms):            24099,7547
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Execution times (ms):
TokenTypeAnalyzer:   24100

This PR

Pausing 5 seconds before starting analysis...
Running analyzers for CsvHelper(net6.0)
Running analyzers for CsvHelper(net5.0)
Running analyzers for CsvHelper(netstandard2.1)
Running analyzers for CsvHelper(netstandard2.0)
Running analyzers for CsvHelper(net47)
Running analyzers for CsvHelper(net45)
Running analyzers for CsvHelper.Tests
Running analyzers for _build
Running analyzers for CsvHelper.Performance
Running analyzers for CsvHelper.Website
Found 0 diagnostics in 13211ms (994932760 bytes allocated)
Statistics for TokenTypeAnalyzer:
Concurrent:                     False
Execution time (ms):            7648,7773
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Execution times (ms):
TokenTypeAnalyzer:    7649

@martin-strecker-sonarsource
Copy link
Contributor Author

martin-strecker-sonarsource commented Dec 21, 2022

For all utility analyzers

AnalyzerRunner\Debug\net7.0\publish\AnalyzerRunner.exe C:\Projects\sonar-dotnet\analyzers\packaging\binaries C:\Projects\DeepSast\CsvHelper\CsvHelper.sln /a TokenTypeAnalyzer /a SymbolReferenceAnalyzer /a MetricsAnalyzer /a CopyPasteTokenAnalyzer /a FileMetadataAnalyzer /a LogAnalyzer

Master PR PR EnableConcurrentExecution
Runtime 75 s 18 s 20 s
Allocation 3.692 MB 1243 MB 1243 MB
CopyPasteTokenAnalyzer 0.892 0.940 1.005
FileMetadataAnalyzer 0.196 0.170 0.189
LogAnalyzer 0.167 0.158 0.174
MetricsAnalyzer 4.0 2.711 2.974
SymbolReferenceAnalyzer 59.9 2.325 2.555
TokenTypeAnalyzer 6.8 6.8 7.527

Master

Running analyzers for CsvHelper(net6.0)
Running analyzers for CsvHelper(net5.0)
Running analyzers for CsvHelper(netstandard2.1)
Running analyzers for CsvHelper(netstandard2.0)
Running analyzers for CsvHelper(net47)
Running analyzers for CsvHelper(net45)
Running analyzers for CsvHelper.Tests
Running analyzers for _build
Running analyzers for CsvHelper.Performance
Running analyzers for CsvHelper.Website
Found 0 diagnostics in 75016ms (3871965648 bytes allocated)
Statistics for CopyPasteTokenAnalyzer:
Concurrent:                     False
Execution time (ms):            892,2681
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for FileMetadataAnalyzer:
Concurrent:                     False
Execution time (ms):            195,5788
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for LogAnalyzer:
Concurrent:                     False
Execution time (ms):            166,5213
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for MetricsAnalyzer:
Concurrent:                     False
Execution time (ms):            4008,6636
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for SymbolReferenceAnalyzer:
Concurrent:                     False
Execution time (ms):            59912,6414
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for TokenTypeAnalyzer:
Concurrent:                     False
Execution time (ms):            6826,2595
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            10
Compilation Start Actions:      0
Compilation End Actions:        0
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         0
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Execution times (ms):
CopyPasteTokenAnalyzer:      892
FileMetadataAnalyzer:        196
LogAnalyzer:                 167
MetricsAnalyzer:            4009
SymbolReferenceAnalyzer:   59913
TokenTypeAnalyzer:          6826

PR

Pausing 5 seconds before starting analysis...
Running analyzers for CsvHelper(net6.0)
Running analyzers for CsvHelper(net5.0)
Running analyzers for CsvHelper(netstandard2.1)
Running analyzers for CsvHelper(netstandard2.0)
Running analyzers for CsvHelper(net47)
Running analyzers for CsvHelper(net45)
Running analyzers for CsvHelper.Tests
Running analyzers for _build
Running analyzers for CsvHelper.Performance
Running analyzers for CsvHelper.Website
Found 0 diagnostics in 18708ms (1303553416 bytes allocated)
Statistics for CopyPasteTokenAnalyzer:
Concurrent:                     False
Execution time (ms):            940,4342
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for FileMetadataAnalyzer:
Concurrent:                     False
Execution time (ms):            169,9631
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for LogAnalyzer:
Concurrent:                     False
Execution time (ms):            158,0171
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for MetricsAnalyzer:
Concurrent:                     False
Execution time (ms):            2711,442
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for SymbolReferenceAnalyzer:
Concurrent:                     False
Execution time (ms):            2324,8239
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for TokenTypeAnalyzer:
Concurrent:                     False
Execution time (ms):            6799,581
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Execution times (ms):
CopyPasteTokenAnalyzer:      940
FileMetadataAnalyzer:        170
LogAnalyzer:                 158
MetricsAnalyzer:            2711
SymbolReferenceAnalyzer:    2325
TokenTypeAnalyzer:          6800

With EnableConcurrentExecution = true

Pausing 5 seconds before starting analysis...
Running analyzers for CsvHelper(net6.0)
Running analyzers for CsvHelper(net5.0)
Running analyzers for CsvHelper(netstandard2.1)
Running analyzers for CsvHelper(netstandard2.0)
Running analyzers for CsvHelper(net47)
Running analyzers for CsvHelper(net45)
Running analyzers for CsvHelper.Tests
Running analyzers for _build
Running analyzers for CsvHelper.Performance
Running analyzers for CsvHelper.Website
Found 0 diagnostics in 20391ms (1303991808 bytes allocated)
Statistics for CopyPasteTokenAnalyzer:
Concurrent:                     True
Execution time (ms):            1004,5332
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for FileMetadataAnalyzer:
Concurrent:                     True
Execution time (ms):            189,3969
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for LogAnalyzer:
Concurrent:                     True
Execution time (ms):            173,524
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for MetricsAnalyzer:
Concurrent:                     True
Execution time (ms):            2974,0859
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for SymbolReferenceAnalyzer:
Concurrent:                     True
Execution time (ms):            2554,513
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Statistics for TokenTypeAnalyzer:
Concurrent:                     True
Execution time (ms):            7526,5965
Code Block Actions:             0
Code Block Start Actions:       0
Code Block End Actions:         0
Compilation Actions:            0
Compilation Start Actions:      10
Compilation End Actions:        10
Operation Actions:              0
Operation Block Actions:        0
Operation Block Start Actions:  0
Operation Block End Actions:    0
Semantic Model Actions:         10
Symbol Actions:                 0
Symbol Start Actions:           0
Symbol End Actions:             0
Syntax Node Actions:            0
Syntax Tree Actions:            0
Additional File Actions:        0
Suppression Actions:            0
Execution times (ms):
CopyPasteTokenAnalyzer:     1005
FileMetadataAnalyzer:        189
LogAnalyzer:                 174
MetricsAnalyzer:            2974
SymbolReferenceAnalyzer:    2555
TokenTypeAnalyzer:          7527

@costin-zaharia-sonarsource
Copy link
Member

Excellent. Thanks looks awesome!

@martin-strecker-sonarsource martin-strecker-sonarsource force-pushed the Martin/UtilityAnalyzer_Registration branch from 7402be7 to d442636 Compare January 3, 2023 09:24
@martin-strecker-sonarsource
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the code, without reviewing the UTs yet

Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SonarAnalysisContextBase changes have to be reverted

Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be doable with ReadParameters(SonarCompilationStartAnalysisContext)

Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final polishing

@martin-strecker-sonarsource martin-strecker-sonarsource force-pushed the Martin/UtilityAnalyzer_Registration branch from 3ac7fbe to dacee9b Compare February 1, 2023 17:44
Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed an issue with inheritnace. Should be easy to fix.

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@pavel-mikula-sonarsource pavel-mikula-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

On Akka.Net, I did not see any change in performance, see the semantic models are most likely cached as they are queried by different rules of the same analysis:
Before: 67. 65. 67
After: 62, 66, 69

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance: UtilityAnalyzerBase calls GetSemanticModel for each SyntaxTree
3 participants