Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix update-apm package-storage #8972

Merged
merged 4 commits into from
Aug 26, 2022
Merged

fix update-apm package-storage #8972

merged 4 commits into from
Aug 26, 2022

Conversation

v1v
Copy link
Member

@v1v v1v commented Aug 25, 2022

What

Set PACKAGE_STORAGE_BRANCH value only once.

Why

Solves the issue with the automation for the package-store.

Creating pull request for update-apm-20220825154830 into snapshot in elastic/package-storage

pull request create failed: GraphQL: Head sha can't be blank, Base sha can't be blank, No commits between snapshot and update-apm-20220825154830, Head ref must be a branch (createPullRequest)

elastic/package-storage#5414 was automatically created.

@v1v v1v added the backport-8.4 Automated backport with mergify label Aug 25, 2022
@v1v v1v requested review from a team August 25, 2022 15:02
@v1v v1v self-assigned this Aug 25, 2022
@v1v
Copy link
Member Author

v1v commented Aug 25, 2022

@elastic/apm-server and @mtojek , do we need to support the legacy system? or v2?

I've just seen #8717, so maybe this change is not needed anymore but removing the support for v1?

@mtojek
Copy link
Contributor

mtojek commented Aug 25, 2022

We may need it in case of emergency rollback to v1. Once we're good with elastic/kibana#137759 and there are no more blockers, we will switch to v2.

@apmmachine
Copy link
Contributor

apmmachine commented Aug 25, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-08-26T10:46:19.555+0000

  • Duration: 27 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 129
Skipped 0
Total 129

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate and publish the docker images.

  • /test windows : Build & tests on Windows.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@apmmachine
Copy link
Contributor

apmmachine commented Aug 25, 2022

📚 Go benchmark report

Diff with the main branch

name                                                                                              old time/op    new time/op     delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/errors_2.ndjson-12                                                                  129µs ±12%       75µs ±37%   -41.44%  (p=0.008 n=5+5)
BackendProcessor/errors_rum.ndjson-12                                                               29.3µs ±44%     15.1µs ±26%   -48.35%  (p=0.016 n=5+5)
BackendProcessor/errors_transaction_id.ndjson-12                                                    75.6µs ±24%     46.6µs ±27%   -38.36%  (p=0.016 n=5+5)
BackendProcessor/invalid-event-type.ndjson-12                                                       13.4µs ±15%      4.8µs ± 2%   -63.94%  (p=0.008 n=5+5)
BackendProcessor/invalid-event.ndjson-12                                                            37.0µs ±16%     24.9µs ±18%   -32.56%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-event.ndjson-12                                                       20.6µs ±25%      8.8µs ±22%   -57.38%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-metadata.ndjson-12                                                    20.8µs ±18%     13.8µs ±12%   -33.38%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata-2.ndjson-12                                                       7.57µs ± 6%     2.63µs ± 3%   -65.32%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata.ndjson-12                                                         7.39µs ±14%     2.71µs ± 3%   -63.40%  (p=0.008 n=5+5)
BackendProcessor/metadata-null-values.ndjson-12                                                     15.0µs ±31%      5.4µs ± 5%   -64.14%  (p=0.008 n=5+5)
BackendProcessor/metadata.ndjson-12                                                                 19.5µs ± 9%      7.8µs ± 6%   -59.80%  (p=0.008 n=5+5)
BackendProcessor/metricsets.ndjson-12                                                                104µs ±24%       71µs ±20%   -31.65%  (p=0.008 n=5+5)
BackendProcessor/minimal-service.ndjson-12                                                          25.6µs ±24%     14.8µs ±35%   -42.02%  (p=0.016 n=5+5)
BackendProcessor/optional-timestamps.ndjson-12                                                      44.0µs ±17%     32.9µs ± 9%   -25.25%  (p=0.008 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                              61.4µs ±14%     36.8µs ±11%   -40.11%  (p=0.008 n=5+5)
BackendProcessor/span-links.ndjson-12                                                               29.1µs ±10%     15.9µs ±13%   -45.44%  (p=0.008 n=5+5)
BackendProcessor/transactions-huge_traces.ndjson-12                                                 68.6µs ±34%     41.4µs ±14%   -39.68%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum.ndjson-12                                                   36.6µs ±12%     19.8µs ±25%   -45.86%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum_2.ndjson-12                                                 36.3µs ±10%     20.0µs ±13%   -44.73%  (p=0.008 n=5+5)
BackendProcessor/unknown-span-type.ndjson-12                                                         101µs ±13%       61µs ±12%   -39.89%  (p=0.008 n=5+5)
RUMV3Processor/rum_errors.ndjson-12                                                                 8.62µs ±12%     3.25µs ± 4%   -62.30%  (p=0.008 n=5+5)
RUMV3Processor/rum_events.ndjson-12                                                                 10.2µs ±13%      3.8µs ± 5%   -62.68%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/errors_transaction_id.ndjson-12         18.3µs ± 8%     20.9µs ±11%   +13.84%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-json-metadata.ndjson-12         6.53µs ± 1%     7.13µs ± 5%    +9.18%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/minimal.ndjson-12                       16.2µs ±12%     20.5µs ±14%   +27.05%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/spans.ndjson-12                         64.6µs ±13%     77.9µs ±19%   +20.57%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metricsets.ndjson-12                    6.96µs ± 1%     6.85µs ± 0%    -1.62%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/ratelimit.ndjson-12                     18.8µs ± 1%     18.6µs ± 1%    -1.17%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-metadata-2.ndjson-12           490ns ± 1%      504ns ± 2%    +2.83%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-metadata.ndjson-12             494ns ± 1%      509ns ± 0%    +2.96%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metadata.ndjson-12                    1.28µs ± 1%     1.27µs ± 1%    -1.14%  (p=0.024 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
WriteTransaction/nop_codec-12                                                                        380ns ±70%      780ns ±31%  +105.39%  (p=0.016 n=5+5)
ReadEvents/nop_codec/399_events-12                                                                   503µs ± 9%      429µs ±21%   -14.62%  (p=0.032 n=5+5)

name                                                                                              old alloc/op   new alloc/op    delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
CompressedRequestReader/gzip_sniff-12                                                               45.7kB ± 0%     45.7kB ± 0%    -0.03%  (p=0.029 n=4+4)
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/errors_2.ndjson-12                                                                 38.4kB ± 2%     40.0kB ± 1%    +4.03%  (p=0.008 n=5+5)
BackendProcessor/errors_rum.ndjson-12                                                               10.1kB ± 1%     10.8kB ± 2%    +7.11%  (p=0.008 n=5+5)
BackendProcessor/errors_transaction_id.ndjson-12                                                    21.2kB ± 1%     22.1kB ± 1%    +4.04%  (p=0.008 n=5+5)
BackendProcessor/invalid-event-type.ndjson-12                                                       3.58kB ± 2%     3.90kB ± 1%    +8.88%  (p=0.008 n=5+5)
BackendProcessor/invalid-event.ndjson-12                                                            9.87kB ± 2%    10.94kB ± 3%   +10.76%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-event.ndjson-12                                                       6.27kB ± 1%     7.01kB ± 2%   +11.79%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-metadata.ndjson-12                                                    5.27kB ± 1%     6.16kB ± 3%   +17.00%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata-2.ndjson-12                                                       2.73kB ± 3%     2.91kB ± 2%    +6.41%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata.ndjson-12                                                         2.69kB ± 2%     2.95kB ± 2%    +9.65%  (p=0.008 n=5+5)
BackendProcessor/metadata-null-values.ndjson-12                                                     5.54kB ± 4%     6.21kB ± 3%   +12.06%  (p=0.008 n=5+5)
BackendProcessor/metadata.ndjson-12                                                                 4.95kB ± 2%     5.51kB ± 2%   +11.41%  (p=0.008 n=5+5)
BackendProcessor/metricsets.ndjson-12                                                               31.1kB ± 2%     33.4kB ± 2%    +7.46%  (p=0.008 n=5+5)
BackendProcessor/minimal-service.ndjson-12                                                          11.3kB ± 1%     12.6kB ± 2%   +11.96%  (p=0.008 n=5+5)
BackendProcessor/minimal.ndjson-12                                                                  38.4kB ± 1%     41.4kB ± 3%    +8.02%  (p=0.008 n=5+5)
BackendProcessor/optional-timestamps.ndjson-12                                                      21.0kB ± 2%     22.3kB ± 2%    +6.20%  (p=0.008 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                              17.7kB ± 1%     18.9kB ± 2%    +6.90%  (p=0.008 n=5+5)
BackendProcessor/span-links.ndjson-12                                                               11.5kB ± 2%     12.7kB ± 3%   +10.74%  (p=0.008 n=5+5)
BackendProcessor/transactions-huge_traces.ndjson-12                                                 17.2kB ± 1%     18.7kB ± 2%    +8.21%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum.ndjson-12                                                   12.3kB ± 1%     13.6kB ± 2%   +10.60%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum_2.ndjson-12                                                 12.3kB ± 2%     13.2kB ± 2%    +7.73%  (p=0.008 n=5+5)
BackendProcessor/unknown-span-type.ndjson-12                                                        24.5kB ± 2%     25.7kB ± 1%    +4.86%  (p=0.008 n=5+5)
RUMV3Processor/rum_errors.ndjson-12                                                                 2.88kB ± 2%     3.19kB ± 3%   +10.73%  (p=0.008 n=5+5)
RUMV3Processor/rum_events.ndjson-12                                                                 3.10kB ± 2%     3.40kB ± 2%    +9.75%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/metadata.ndjson-12                      5.52kB ± 1%     5.45kB ± 1%    -1.36%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/transactions_spans_rum_2.ndjson-12      12.9kB ± 0%     12.7kB ± 1%    -0.96%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/invalid-metadata.ndjson-12              3.17kB ± 1%     3.19kB ± 1%    +0.77%  (p=0.032 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
ReadEvents/json_codec/399_events-12                                                                 1.54MB ± 0%     1.55MB ± 0%    +0.11%  (p=0.029 n=4+4)
ReadEvents/json_codec_big_tx/10_events-12                                                           95.5kB ± 0%     95.6kB ± 0%    +0.04%  (p=0.048 n=5+5)
ReadEvents/json_codec_big_tx/199_events-12                                                          1.34MB ± 0%     1.34MB ± 0%    -0.05%  (p=0.016 n=5+5)

name                                                                                              old allocs/op  new allocs/op   delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/decoder goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/errors.ndjson-12                                                                      464 ± 0%        465 ± 0%    +0.22%  (p=0.029 n=4+4)
BackendProcessor/metricsets.ndjson-12                                                                  113 ± 0%        114 ± 0%    +0.88%  (p=0.016 n=4+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                              old speed      new speed       delta
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/errors_2.ndjson-12                                                               37.1MB/s ±13%   65.2MB/s ±30%   +75.91%  (p=0.008 n=5+5)
BackendProcessor/errors_rum.ndjson-12                                                             71.2MB/s ±63%  127.6MB/s ±22%   +79.20%  (p=0.016 n=5+5)
BackendProcessor/errors_transaction_id.ndjson-12                                                  51.8MB/s ±29%   84.4MB/s ±25%   +62.80%  (p=0.016 n=5+5)
BackendProcessor/invalid-event-type.ndjson-12                                                     29.6MB/s ±15%   81.1MB/s ± 2%  +173.61%  (p=0.008 n=5+5)
BackendProcessor/invalid-event.ndjson-12                                                          21.0MB/s ±15%   31.1MB/s ±16%   +48.44%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-event.ndjson-12                                                     29.2MB/s ±30%   68.6MB/s ±23%  +134.56%  (p=0.008 n=5+5)
BackendProcessor/invalid-json-metadata.ndjson-12                                                  21.9MB/s ±16%   32.5MB/s ±11%   +48.50%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata-2.ndjson-12                                                     57.6MB/s ± 6%  166.0MB/s ± 3%  +188.05%  (p=0.008 n=5+5)
BackendProcessor/invalid-metadata.ndjson-12                                                       60.7MB/s ±15%  164.9MB/s ± 3%  +171.55%  (p=0.008 n=5+5)
BackendProcessor/metadata-null-values.ndjson-12                                                   36.2MB/s ±39%   97.8MB/s ± 5%  +169.75%  (p=0.008 n=5+5)
BackendProcessor/metadata.ndjson-12                                                               64.0MB/s ± 9%  159.1MB/s ± 6%  +148.48%  (p=0.008 n=5+5)
BackendProcessor/metricsets.ndjson-12                                                             24.8MB/s ±21%   36.2MB/s ±22%   +46.32%  (p=0.008 n=5+5)
BackendProcessor/minimal-service.ndjson-12                                                        16.9MB/s ±28%   29.8MB/s ±29%   +76.00%  (p=0.016 n=5+5)
BackendProcessor/optional-timestamps.ndjson-12                                                    23.5MB/s ±19%   31.3MB/s ± 9%   +32.96%  (p=0.008 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                            31.0MB/s ±14%   51.4MB/s ±12%   +65.73%  (p=0.008 n=5+5)
BackendProcessor/span-links.ndjson-12                                                             23.5MB/s ±10%   43.2MB/s ±14%   +84.02%  (p=0.008 n=5+5)
BackendProcessor/transactions-huge_traces.ndjson-12                                               48.6MB/s ±35%   77.3MB/s ±15%   +58.96%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum.ndjson-12                                                 31.7MB/s ±11%   59.9MB/s ±26%   +89.17%  (p=0.008 n=5+5)
BackendProcessor/transactions_spans_rum_2.ndjson-12                                               30.9MB/s ±10%   56.2MB/s ±13%   +81.67%  (p=0.008 n=5+5)
BackendProcessor/unknown-span-type.ndjson-12                                                      32.9MB/s ±12%   55.0MB/s ±13%   +67.06%  (p=0.008 n=5+5)
RUMV3Processor/rum_errors.ndjson-12                                                                112MB/s ±13%    295MB/s ± 4%  +163.77%  (p=0.008 n=5+5)
RUMV3Processor/rum_events.ndjson-12                                                                388MB/s ±14%   1032MB/s ± 5%  +166.04%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/errors_transaction_id.ndjson-12        209MB/s ± 7%    184MB/s ±12%   -11.80%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-json-metadata.ndjson-12       68.5MB/s ± 1%   62.8MB/s ± 5%    -8.32%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/minimal.ndjson-12                     64.0MB/s ±11%   50.7MB/s ±15%   -20.77%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/spans.ndjson-12                        125MB/s ±13%    104MB/s ±17%   -16.65%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metricsets.ndjson-12                   366MB/s ± 1%    372MB/s ± 0%    +1.64%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/ratelimit.ndjson-12                    224MB/s ± 1%    227MB/s ± 1%    +1.18%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-metadata-2.ndjson-12         889MB/s ± 1%    865MB/s ± 2%    -2.75%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-metadata.ndjson-12           902MB/s ± 1%    876MB/s ± 0%    -2.88%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/metadata.ndjson-12                   970MB/s ± 1%    981MB/s ± 1%    +1.15%  (p=0.032 n=5+5)

report generated with https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

.ci/scripts/Makefile Outdated Show resolved Hide resolved
Co-authored-by: Andrew Wilkins <axwalk@gmail.com>
@v1v v1v merged commit 5dfaba7 into elastic:main Aug 26, 2022
mergify bot pushed a commit that referenced this pull request Aug 26, 2022
v1v added a commit that referenced this pull request Aug 26, 2022
(cherry picked from commit 5dfaba7)

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.4 Automated backport with mergify
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants