Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: disable stack trace range chunking #3583

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

kolesnikovae
Copy link
Collaborator

@kolesnikovae kolesnikovae commented Sep 23, 2024

The PR disables stack trace range chunking because it results in higher CPU and memory consumption and complicates the implementation.

Our aim was to limit the memory consumption of the symbolication process. However, loading chunks synchronously drastically increases latency while still consuming a significant amount of memory. Currently, all the chunks are loaded into memory concurrently, so we are not seeing any benefits.

Spoiler warning
               Stacks per chunk:  10M(∞)  2M      1M     512k    256k    
------------------------------------------------------------------------------------
       Total size on disk (MiB):  27.6    30.5    33     36.8    41.8
     Total in-memory size (MiB):  44.6    51.5    58     67.7    79.5
 Max chunk in-memory size (MiB):  44.6    30.9    19.4   12.1    7.4
------------------------------------------------------------------------------------
 // In-memory size estimations are for reading; twice as much is consumed on write
 // On-disk size might be improved by better order of nodes (delta encoding)

------------------------------------------------------------------------------------

Block 01H2NAN6E6NTS013J0HE5XMN34: 3226104 rows (unique stack traces), 203269526 bytes
Output data set: 27646023 bytes written total (1 chunks, max stack traces per chunk: 10485760)
Estimated total in-memory size: 44612728 bytes
Chunk 0 is full
  Stack traces: 3226104
  Nodes: 5576591
  Ratio: 1.7285837654334764
  Built for 17.506652541s:
     Written bytes: 27646023
     Bytes per stack (on disk): 8
     Bytes per stack (in memory): 13
     In memory size: 44612728
     
------------------------------------------------------------------------------------

Block 01H2NAN6E6NTS013J0HE5XMN34: 3226104 rows (unique stack traces), 203269526 bytes
Output data set: 30505528 bytes written total (2 chunks, max stack traces per chunk: 2097152)
Estimated total in-memory size: 51554040 bytes
Chunk 0 is full
  Stack traces: 2097272
  Nodes: 3862877
  Ratio: 1.841857899213836
  Built for 9.483350875s:
     Written bytes: 18501081
     Bytes per stack (on disk): 8
     Bytes per stack (in memory): 14
     In memory size: 30903016
Chunk 1 is full
  Stack traces: 1128832
  Nodes: 2581378
  Ratio: 2.286768978909173
  Built for 2.76897675s:
     Written bytes: 12004447
     Bytes per stack (on disk): 10
     Bytes per stack (in memory): 18
     In memory size: 20651024
     
------------------------------------------------------------------------------------

Block 01H2NAN6E6NTS013J0HE5XMN34: 3226104 rows (unique stack traces), 203269526 bytes
Output data set: 33045258 bytes written total (4 chunks, max stack traces per chunk: 1048576)
Estimated total in-memory size: 58021856 bytes
Chunk 0 is full
  Stack traces: 1048636
  Nodes: 2222475
  Ratio: 2.119396053540027
  Built for 4.493750125s:
     Written bytes: 10024465
     Bytes per stack (on disk): 9
     Bytes per stack (in memory): 16
     In memory size: 17779800
Chunk 1 is full
  Stack traces: 1048636
  Nodes: 2198712
  Ratio: 2.0967351874244256
  Built for 2.383121625s:
     Written bytes: 10209418
     Bytes per stack (on disk): 9
     Bytes per stack (in memory): 16
     In memory size: 17589696
Chunk 2 is full
  Stack traces: 1048970
  Nodes: 2434981
  Ratio: 2.3213066150604877
  Built for 2.459848041s:
     Written bytes: 11258903
     Bytes per stack (on disk): 10
     Bytes per stack (in memory): 18
     In memory size: 19479848
Chunk 3 is full
  Stack traces: 79862
  Nodes: 396564
  Ratio: 4.965615687060179
  Built for 161.160334ms:
     Written bytes: 1552472
     Bytes per stack (on disk): 19
     Bytes per stack (in memory): 39
     In memory size: 3172512
     
------------------------------------------------------------------------------------

Block 01H2NAN6E6NTS013J0HE5XMN34: 3226104 rows (unique stack traces), 203269526 bytes
Output data set: 36842998 bytes written total (7 chunks, max stack traces per chunk: 524288)
Estimated total in-memory size: 67723448 bytes
Chunk 0 is full
  Stack traces: 524528
  Nodes: 1368112
  Ratio: 2.6082725803007656
  Built for 2.394663625s:
     Written bytes: 5837797
     Bytes per stack (on disk): 11
     Bytes per stack (in memory): 20
     In memory size: 10944896
Chunk 1 is full
  Stack traces: 525116
  Nodes: 1120434
  Ratio: 2.133688556433245
  Built for 1.057257s:
     Written bytes: 5005143
     Bytes per stack (on disk): 9
     Bytes per stack (in memory): 17
     In memory size: 8963472
Chunk 2 is full
  Stack traces: 524528
  Nodes: 1263265
  Ratio: 2.408384299789525
  Built for 1.057030333s:
     Written bytes: 5581112
     Bytes per stack (on disk): 10
     Bytes per stack (in memory): 19
     In memory size: 10106120
Chunk 3 is full
  Stack traces: 524738
  Nodes: 1375277
  Ratio: 2.6208831836078197
  Built for 1.068505917s:
     Written bytes: 6018478
     Bytes per stack (on disk): 11
     Bytes per stack (in memory): 20
     In memory size: 11002216
Chunk 4 is full
  Stack traces: 524652
  Nodes: 1433739
  Ratio: 2.732742846686947
  Built for 1.167284291s:
     Written bytes: 6271720
     Bytes per stack (on disk): 11
     Bytes per stack (in memory): 21
     In memory size: 11469912
Chunk 5 is full
  Stack traces: 524318
  Nodes: 1514972
  Ratio: 2.8894144393288044
  Built for 1.11052725s:
     Written bytes: 6605028
     Bytes per stack (on disk): 12
     Bytes per stack (in memory): 23
     In memory size: 12119776
Chunk 6 is full
  Stack traces: 78224
  Nodes: 389632
  Ratio: 4.980977705052158
  Built for 143.469125ms:
     Written bytes: 1523720
     Bytes per stack (on disk): 19
     Bytes per stack (in memory): 39
     In memory size: 3117056
     
------------------------------------------------------------------------------------

Block 01H2NAN6E6NTS013J0HE5XMN34: 3226104 rows (unique stack traces), 203269526 bytes
Output data set: 41776977 bytes written total (13 chunks, max stack traces per chunk: 262144)
Estimated total in-memory size: 79525928 bytes
Chunk 0 is full
  Stack traces: 262622
  Nodes: 887362
  Ratio: 3.378856302975379
  Built for 1.482846417s:
     Written bytes: 3615568
     Bytes per stack (on disk): 13
     Bytes per stack (in memory): 27
     In memory size: 7098896
Chunk 1 is full
  Stack traces: 262746
  Nodes: 586649
  Ratio: 2.2327609173878957
  Built for 655.994584ms:
     Written bytes: 2543225
     Bytes per stack (on disk): 9
     Bytes per stack (in memory): 17
     In memory size: 4693192
Chunk 2 is full
  Stack traces: 262832
  Nodes: 640279
  Ratio: 2.436077037803616
  Built for 484.878416ms:
     Written bytes: 2782450
     Bytes per stack (on disk): 10
     Bytes per stack (in memory): 19
     In memory size: 5122232
Chunk 3 is full
  Stack traces: 262578
  Nodes: 697386
  Ratio: 2.655919383954482
  Built for 491.707625ms:
     Written bytes: 2989352
     Bytes per stack (on disk): 11
     Bytes per stack (in memory): 21
     In memory size: 5579088
Chunk 4 is full
  Stack traces: 262200
  Nodes: 742836
  Ratio: 2.8330892448512586
  Built for 502.760334ms:
     Written bytes: 3172510
     Bytes per stack (on disk): 12
     Bytes per stack (in memory): 22
     In memory size: 5942688
Chunk 5 is full
  Stack traces: 262832
  Nodes: 776060
  Ratio: 2.9526846046143542
  Built for 518.939459ms:
     Written bytes: 3284356
     Bytes per stack (on disk): 12
     Bytes per stack (in memory): 23
     In memory size: 6208480
Chunk 6 is full
  Stack traces: 263124
  Nodes: 813045
  Ratio: 3.0899689880056553
  Built for 493.477208ms:
     Written bytes: 3429428
     Bytes per stack (on disk): 13
     Bytes per stack (in memory): 24
     In memory size: 6504360
Chunk 7 is full
  Stack traces: 262788
  Nodes: 851289
  Ratio: 3.239451573131193
  Built for 513.548916ms:
     Written bytes: 3563395
     Bytes per stack (on disk): 13
     Bytes per stack (in memory): 25
     In memory size: 6810312
Chunk 8 is full
  Stack traces: 262160
  Nodes: 849390
  Ratio: 3.2399679584986267
  Built for 551.642917ms:
     Written bytes: 3561803
     Bytes per stack (on disk): 13
     Bytes per stack (in memory): 25
     In memory size: 6795120
Chunk 9 is full
  Stack traces: 262284
  Nodes: 884597
  Ratio: 3.372668557746565
  Built for 503.683958ms:
     Written bytes: 3706102
     Bytes per stack (on disk): 14
     Bytes per stack (in memory): 26
     In memory size: 7076776
Chunk 10 is full
  Stack traces: 263000
  Nodes: 909931
  Ratio: 3.4598136882129276
  Built for 519.360542ms:
     Written bytes: 3808142
     Bytes per stack (on disk): 14
     Bytes per stack (in memory): 27
     In memory size: 7279448
Chunk 11 is full
  Stack traces: 262956
  Nodes: 928372
  Ratio: 3.5305222166446097
  Built for 510.007792ms:
     Written bytes: 3864171
     Bytes per stack (on disk): 14
     Bytes per stack (in memory): 28
     In memory size: 7426976
Chunk 12 is full
  Stack traces: 73982
  Nodes: 373545
  Ratio: 5.049133573031278
  Built for 141.845667ms:
     Written bytes: 1456475
     Bytes per stack (on disk): 19
     Bytes per stack (in memory): 40
     In memory size: 2988360

@kolesnikovae kolesnikovae marked this pull request as ready for review September 24, 2024 03:35
@kolesnikovae kolesnikovae requested a review from a team as a code owner September 24, 2024 03:35
@kolesnikovae kolesnikovae merged commit 784d4de into main Sep 24, 2024
18 checks passed
@kolesnikovae kolesnikovae deleted the chore/disable-stacktrace-chunking branch September 24, 2024 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants