Small artifacts for bloat reports (#9331)

* Small-artifact bloat reports #### Problem The current bloat report requires preserving large binaries, and has trouble matching tree parents for comparison. #### Change overview This change makes example builds generate small artifacts containing json file(s) containing build item identification and section sizes. Artifacts names have the form “Size,PR,WORKFLOW,CURRENT_SHA,PARENT_SHA”; these contain one or more json files, each containing a report on one build target. The reporting scripts then examines the list of available artifacts to list of artifacts to identify pairs with the same $WORKFLOW where one's $CURRENT_SHA is the other's $PARENT_SHA, and only then downloads and processes those artifacts to generate comparisons for matching builds. - Changes to `examples-…` workflows: - Adds some `GH_EVENT_…` variables to workflow `env`, to identify the current run. - Uses the `gh_sizes.py` script to generate size report json files. - Uploads size report artifacts. - Changes to `scripts/tools/memory`: - Adds minimal platform config files for recently-added platforms. - Adds markdown output options (required for github comments) along with some associated cleanup. - Adds a script `scripts/tools/memory/gh_sizes.py` for use by workflows; this is similar to `report_summary.py` with a suitable consistent set of arguments. - Adds a script `scripts/tools/memory/gh_report.py` to analyze size report artifacts. - Modifies `bloat_check.py` to ignore the size report artifacts. The github comments produced by `gh_report.py` are slightly different fromt the existing reports. Since this change enables reports for many more builds, a full report will be over a hundred lines, and is placed inside a details tag. Only size changes above a configurable threshold are called out ‘above the fold’. Note that this PR does _not_ include a change to actually send size report comments on github; this is left to a followup after that code has been verified on real-world size artifacts. #### Testing Fork-CI runs and offline verification on resulting artifacts. * remove leftover debug print * small json format change * changes from review - restore original scripts/examples/esp_example.sh and scripts/examples/gn_efr32_example.sh - avoid a problem using pyelftools `describe_p_type` * telink output moved * report sections directly rather than aggregated symbols by section
project-chip · Sep 27, 2021 · 2532096 · 2532096
1 parent 586ee80
commit 2532096
Show file tree

Hide file tree

Showing 26 changed files with 1,723 additions and 168 deletions.
diff --git a/.github/workflows/examples-efr32.yaml b/.github/workflows/examples-efr32.yaml
@@ -30,6 +30,9 @@ jobs:
  env:
  EFR32_BOARD: BRD4161A
  BUILD_TYPE: gn_efr32
+ GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+ GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+ GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
  runs-on: ubuntu-latest
  if: github.actor != 'restyled-io[bot]'
@@ -57,25 +60,29 @@ jobs:
  .environment/pigweed-venv/*.log
  - name: Build example EFR32 Lock App for BRD4161A
  timeout-minutes: 10
- run:
- scripts/examples/gn_efr32_example.sh examples/lock-app/efr32/
- out/lock_app_debug BRD4161A
+ run: |
+ scripts/examples/gn_efr32_example.sh examples/lock-app/efr32/ out/lock_app_debug BRD4161A
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A lock-app \
+ out/lock_app_debug/BRD4161A/chip-efr32-lock-example.out /tmp/bloat_reports/
  - name: Build example EFR32 Lighting App for BRD4161A
  timeout-minutes: 10
- run:
- scripts/examples/gn_efr32_example.sh
- examples/lighting-app/efr32/ out/lighting_app_debug BRD4161A
+ run: |
+ scripts/examples/gn_efr32_example.sh examples/lighting-app/efr32/ out/lighting_app_debug BRD4161A
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A lighting-app \
+ out/lighting_app_debug/BRD4161A/chip-efr32-lighting-example.out /tmp/bloat_reports/
  - name: Build example EFR32 Lighting App for BRD4161A with RPCs
  timeout-minutes: 10
- run:
- scripts/examples/gn_efr32_example.sh
- examples/lighting-app/efr32/ out/lighting_app_debug_rpc BRD4161A
- -args='import("//with_pw_rpc.gni")'
+ run: |
+ scripts/examples/gn_efr32_example.sh examples/lighting-app/efr32/ out/lighting_app_debug_rpc BRD4161A \
+ -args='import("//with_pw_rpc.gni")'
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A+rpc lighting-app \
+ out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out /tmp/bloat_reports/
  - name: Build example EFR32 Window Covering for BRD4161A
  timeout-minutes: 10
- run:
- scripts/examples/gn_efr32_example.sh examples/window-app/efr32/
- out/window_app_debug BRD4161A
+ run: |
+ scripts/examples/gn_efr32_example.sh examples/window-app/efr32/ out/window_app_debug BRD4161A
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A window-app \
+ out/window_app_debug/BRD4161A/chip-efr32-window-example.out /tmp/bloat_reports/
  - name: Binary artifact suffix
  id: outsuffix
  uses: haya14busa/action-cond@v1.0.0
@@ -94,3 +101,9 @@ jobs:
  out/lock_app_debug/BRD4161A/chip-efr32-lock-example.out.map
  out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out
  out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out.map
+ - name: Uploading Size Reports
+ uses: actions/upload-artifact@v2
+ with:
+ name: Size,EFR32-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+ path: |
+ /tmp/bloat_reports/
diff --git a/.github/workflows/examples-esp32.yaml b/.github/workflows/examples-esp32.yaml
@@ -30,6 +30,9 @@ jobs:
 
  env:
  BUILD_TYPE: esp32
+ GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+ GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+ GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
  runs-on: ubuntu-latest
  if: github.actor != 'restyled-io[bot]'
@@ -64,6 +67,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/all-clusters-app/esp32/build/chip-all-clusters-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 m5stack all-clusters-app \
+ example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf \
+ /tmp/bloat_reports/
  - name: Build example All Clusters App C3
  timeout-minutes: 10
  run: scripts/examples/esp_example.sh all-clusters-app sdkconfig_c3devkit.defaults
@@ -72,6 +79,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/all-clusters-app/esp32/build/chip-all-clusters-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 c3devkit all-clusters-app \
+ example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf \
+ /tmp/bloat_reports/
  - name: Build example Pigweed App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh pigweed-app sdkconfig.defaults
@@ -80,6 +91,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/pigweed-app/esp32/build/chip-pigweed-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-pigweed-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default pigweed-app \
+ example_binaries/$BUILD_TYPE-build/chip-pigweed-app.elf \
+ /tmp/bloat_reports/
  - name: Build example Lock App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh lock-app sdkconfig.defaults
@@ -88,6 +103,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/lock-app/esp32/build/chip-lock-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-lock-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default lock-app \
+ example_binaries/$BUILD_TYPE-build/chip-lock-app.elf \
+ /tmp/bloat_reports/
  - name: Build example Bridge App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh bridge-app
@@ -96,6 +115,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/bridge-app/esp32/build/chip-bridge-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-bridge-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default bridge-app \
+ example_binaries/$BUILD_TYPE-build/chip-bridge-app.elf \
+ /tmp/bloat_reports/
  - name: Build example Persistent Storage App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh persistent-storage sdkconfig.defaults
@@ -104,6 +127,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/persistent-storage/esp32/build/chip-persistent-storage.elf \
  example_binaries/$BUILD_TYPE-build/chip-persistent-storage.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default persistent-storage \
+ example_binaries/$BUILD_TYPE-build/chip-persistent-storage.elf \
+ /tmp/bloat_reports/
  - name: Build example Shell App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh shell sdkconfig.defaults
@@ -112,6 +139,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/shell/esp32/build/chip-shell.elf \
  example_binaries/$BUILD_TYPE-build/chip-shell.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default shell \
+ example_binaries/$BUILD_TYPE-build/chip-shell.elf \
+ /tmp/bloat_reports/
  - name: Build example Temperature Measurement App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh temperature-measurement-app sdkconfig.optimize.defaults
@@ -120,6 +151,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/temperature-measurement-app/esp32/build/chip-temperature-measurement-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-temperature-measurement-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 optimize temperature-measurement-app \
+ example_binaries/$BUILD_TYPE-build/chip-temperature-measurement-app.elf \
+ /tmp/bloat_reports/
  - name: Build example IPv6 Only App
  timeout-minutes: 5
  run: scripts/examples/esp_example.sh ipv6only-app sdkconfig.defaults
@@ -128,6 +163,10 @@ jobs:
  mkdir -p example_binaries/$BUILD_TYPE-build
  cp examples/ipv6only-app/esp32/build/chip-ipv6only-app.elf \
  example_binaries/$BUILD_TYPE-build/chip-ipv6only-app.elf
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ esp32 default ipv6only-app \
+ example_binaries/$BUILD_TYPE-build/chip-ipv6only-app.elf \
+ /tmp/bloat_reports/
  - name: Binary artifact suffix
  id: outsuffix
  uses: haya14busa/action-cond@v1.0.0
@@ -145,3 +184,8 @@ jobs:
  ${{ env.BUILD_TYPE }}-example-build-${{
  steps.outsuffix.outputs.value }}
  path: /tmp/output_binaries/${{ env.BUILD_TYPE }}-build
+ - name: Uploading Size Reports
+ uses: actions/upload-artifact@v2
+ with:
+ name: Size,ESP32-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+ path: /tmp/bloat_reports/
diff --git a/.github/workflows/examples-infineon.yaml b/.github/workflows/examples-infineon.yaml
@@ -28,6 +28,11 @@ jobs:
  name: Infineon examples building
  timeout-minutes: 30
 
+ env:
+ GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+ GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+ GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
+
  runs-on: ubuntu-latest
  if: github.actor != 'restyled-io[bot]'
 
@@ -55,4 +60,13 @@ jobs:
  run: |
  scripts/run_in_build_env.sh \
  "scripts/build/build_examples.py --no-log-timestamps --platform infineon --app lock build"
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ p6 default lock-app \
+ out/infineon-p6board-lock/chip-p6-lock-example.out
 
+ - name: Uploading Size Reports
+ uses: actions/upload-artifact@v2
+ with:
+ name: Size,P6-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+ path: |
+ out/infineon-p6board-lock/p6-default-lock-app-sizes.json
diff --git a/.github/workflows/examples-k32w.yaml b/.github/workflows/examples-k32w.yaml
@@ -29,6 +29,9 @@ jobs:
 
  env:
  BUILD_TYPE: gn_k32w
+ GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+ GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+ GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
  runs-on: ubuntu-latest
  if: github.actor != 'restyled-io[bot]'
@@ -56,16 +59,28 @@ jobs:
  .environment/pigweed-venv/*.log
  - name: Build example K32W Lock App
  timeout-minutes: 5
- run: scripts/examples/k32w_example.sh
- examples/lock-app/k32w out/lock_app_debug
+ run: |
+ scripts/examples/k32w_example.sh examples/lock-app/k32w out/lock_app_debug
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ k32w k32w061+debug lock-app \
+ out/lock_app_debug/chip-k32w061-lock-example \
+ /tmp/bloat_reports/
  - name: Build example K32W Shell App
  timeout-minutes: 5
- run: scripts/examples/k32w_example.sh
- examples/shell/k32w out/shell_app_debug
+ run: |
+ scripts/examples/k32w_example.sh examples/shell/k32w out/shell_app_debug
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ k32w k32w061+debug shell \
+ out/shell_app_debug/chip-k32w061-shell-example \
+ /tmp/bloat_reports/
  - name: Build example K32W Lighting App with Secure Element
  timeout-minutes: 5
- run: scripts/examples/k32w_se_example.sh
- examples/lighting-app/k32w out/lighting_app_se_release
+ run: |
+ scripts/examples/k32w_se_example.sh examples/lighting-app/k32w out/lighting_app_se_release
+ .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+ k32w k32w061+se05x+release lighting-app \
+ out/lighting_app_se_release/chip-k32w061-light-example \
+ /tmp/bloat_reports/
  - name: Binary artifact suffix
  id: outsuffix
  uses: haya14busa/action-cond@v1.0.0
@@ -82,3 +97,9 @@ jobs:
  path: |
  out/lock_app_debug/chip-k32w061-lock-example.out
  out/lock_app_debug/chip-k32w061-lock-example.out.map
+ - name: Uploading Size Reports
+ uses: actions/upload-artifact@v2
+ with:
+ name: Size,K32W-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+ path: |
+ /tmp/bloat_reports/