Skip to content

Incompatibility with flutter candidate 4f49888 and benchmarks #9456

@srawlins

Description

@srawlins

When attempting to bump the flutter candidate to 4f49888a06baf5c52562075c0e2e3163bb4cdfda (from 86327198ff0678f46cbdfb3e12f1759d03aa768d), we see failures in the benchmarks CI task:

https://github.com/flutter/devtools/actions/runs/18231277406/job/51915214753?pr=9454


              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              ...
            ]
     Which: has length of <44>
  
  package:matcher                               expect
  benchmark/devtools_benchmarks_test.dart 81:5  _runBenchmarks
  Expected: an object with length of <30>
    Actual: [
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              Instance of 'BenchmarkScore',
              ...
            ]
     Which: has length of <44>
  
  package:matcher                               expect
  benchmark/devtools_benchmarks_test.dart 81:5  _runBenchmarks

@elliette notes:

I'm guessing it's not actually the wasm build (like it should be) but the JS build, because extra frames are included in the JS benchmarks, see https://github.com/flutter/packages/blob/e401aeb3aae46b82ce26096d960900748a13131a/packages/web_benchmarks/lib/src/metrics.dart#L152-L155

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1high priority issues at the top of the work list, actively being worked on.bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions