Upload fails when running locally

[MDFL64]

I've been trying to run the CLI tool locally for the advent challenge. As far as I can tell, I've set up everything correctly: I've logged in, enabled the repo, and tried passing the token to the app.

The profiles appear to complete successfully, but the upload fails with a strange message:

Error: error decoding response body: missing field `status` at line 1 column 35
Error: missing field `status` at line 1 column 35

And now there are a bunch of empty runs in the dashboard: https://codspeed.io/MDFL64/iamspeed/runs?page=1

I spent some time trying to debug it myself, but I assume it's an issue with the backend.

I'm running it on WSL in case that could have any effect.

[adriencaccia]

Hey @MDFL64, thanks for the issue, this is indeed an error on our end that we noticed thanks to you.
Will fix ASAP

[adriencaccia]

@MDFL64 The issue is fixed, you should be able to make local runs now!

[MDFL64]

Sorry, I only just got around to trying to run it again. Now the upload succeeds, but it gets stuck on "Fetching the results". The entry on the dashboard is still empty. https://codspeed.io/MDFL64/iamspeed/runs/6750e920ce962696a058ed24

I will try to run it on something other than my WSL system.

[MDFL64]

Getting the same results on another machine. Stuck on "Fetching the results" with a 404 in the dashboard.

[adriencaccia]

Hey @MDFL64 thanks for testing, and sorry that it is still not working 😖

In order to make it work, you will need to make a run in the CI on the default branch, so that we have existing reports. Once you have done that, local runs will work and will be compared to the latest remote run.

[MDFL64]

Got it working. Thank you!

[Jeremie-Chauvel]

Hey 👋 ,
not urgent at all (CLI is in beta and all), but seems I'm kind of having the same issue of the cli exiting successfully with the local run being created in codspeed but empty.

• Forked runner (I'm using pop os an ubuntu derivative) : https://github.com/Jeremie-Chauvel/runner (1 commit, I didn't want to make the pr to increase your support scope, and if supporting more ubuntu derivatives, there is probably a better way to do it)

• repo with the bench: https://github.com/Jeremie-Chauvel/codspeed-experiment, I'm using https://docs.astral.sh/uv for python. install it, then run uv python install 3.12 && cd bench-python && uv sync && sudo -v && CODSPEED_LOG=debug codspeed run uv run pytest

• I added some notes about the CLI:

  • needs to run as sudo (for setarch), run sudo -v before
  • fork for pop os archi, easy to install the runner once built from source (no docs though?)
  • assume using system python, you need to copy a shared lib in your virtual env:
    • for my python version using uv : ln -srf ~/.local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/libpython3.12.so.1.0 ./.venv/lib/

• You can run the bench using cd bench-python && sudo -v && CODSPEED_LOG=debug codspeed run uv run pytest (you can find the command in the taskfile)

Complete logs of running the bench with debug

[DEBUG::codspeed::run] config: Config {
upload_url: Url {
scheme: "https",
cannot_be_a_base: false,
username: "",
password: None,
host: Some(
Domain(
"api.codspeed.io",
),
),
port: None,
path: "/upload",
query: None,
fragment: None,
},
token: None,
working_directory: None,
command: "uv run pytest",
instruments: Instruments {
mongodb: None,
},
skip_upload: false,
skip_setup: false,
}
[DEBUG::codspeed::run] Using the token from the CodSpeed configuration file
[DEBUG::codspeed::run::check_system] System info: SystemInfo {
os: "pop",
os_version: "22.04",
arch: "x86_64",
host: "c",
user: "jeremiec",
cpu_brand: "Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz",
cpu_name: "cpu0",
cpu_vendor_id: "GenuineIntel",
cpu_cores: 6,
total_memory_gb: 31,
}
[DEBUG::codspeed::run::runner] CODSPEED_RUNNER_MODE is not set, using valgrind
[DEBUG::codspeed::run::runner::helpers::profile_folder] Created profile folder: /tmp/profile.clsFlH4Jz0.out
PREPARING THE ENVIRONMENT

Preparing the environment...
[DEBUG::codspeed::run::runner::valgrind::setup] Valgrind is already installed with the correct version, skipping installation
Environment ready

RUNNING THE BENCHMARKS

Running the benchmarks...
[DEBUG::codspeed::run::runner::valgrind::helpers::ignored_objects_path] Failed to get python shared objects: Some(Os { code: 2, kind: NotFound, message: "No such file or directory" })
[DEBUG::codspeed::run::runner::valgrind::helpers::ignored_objects_path] objects_path_to_ignore before normalization: ["/home/jeremiec/.local/share/fnm/node-versions/v18.17.0/installation/bin/node"]
[DEBUG::codspeed::run::runner::valgrind::helpers::ignored_objects_path] objects_path_to_ignore after normalization: ["/home/jeremiec/.local/share/fnm/node-versions/v18.17.0/installation/bin/node"]
[DEBUG::codspeed::run::runner::valgrind::measure] cmd: ARCH="x86_64" CODSPEED_ENV="runner" PATH="/tmp/codspeed_introspected_node:[...SANITIZED...]" PYTHONHASHSEED="0" PYTHONMALLOC="malloc" "setarch" "x86_64" "-R" "valgrind" "-q" "--tool=callgrind" "--trace-children=yes" "--cache-sim=yes" "--I1=32768,8,64" "--D1=32768,8,64" "--LL=8388608,16,64" "--instr-atstart=no" "--collect-systime=nsec" "--compress-strings=no" "--combine-dumps=yes" "--dump-line=no" "--trace-children-skip=*esbuild" "--obj-skip=/home/jeremiec/.local/share/fnm/node-versions/v18.17.0/installation/bin/node" "--callgrind-out-file=/tmp/profile.clsFlH4Jz0.out/%p.out" "--log-file=/tmp/profile.clsFlH4Jz0.out/valgrind.log" "sh" "-c" "uv run pytest"
============================= test session starts ==============================
platform linux -- Python 3.12.6, pytest-8.3.4, pluggy-1.5.0
codspeed: 3.1.0 (enabled, mode: instrumentation, callgraph: enabled)
rootdir: /home/jeremiec/Documents/perso/codspeed/codspeed-experiments/bench-python
configfile: pyproject.toml
plugins: codspeed-3.1.0
collected 1 item

test_count_even.py . [100%]
================================ 1 benchmarked =================================

============================== 1 passed in 0.61s ===============================

UPLOADING PERFORMANCE DATA

Uploading performance data...
[DEBUG::codspeed::run::logger] Persisting log file to /tmp/profile.clsFlH4Jz0.out/runner.log
[DEBUG::codspeed::run::uploader::upload] CI provider detected: "Local"
[DEBUG::codspeed::run::uploader::upload] Upload metadata: UploadMetadata {
repository_provider: GitHub,
version: Some(
5,
),
tokenless: false,
profile_md5: "LeWznVhkS2K0gy1CUkbL/A==",
runner: Runner {
name: "codspeed-runner",
version: "3.2.1",
instruments: [],
executor: Valgrind,
system_info: SystemInfo {
os: "pop",
os_version: "22.04",
arch: "x86_64",
host: "c",
user: "jeremiec",
cpu_brand: "Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz",
cpu_name: "cpu0",
cpu_vendor_id: "GenuineIntel",
cpu_cores: 6,
total_memory_gb: 31,
},
},
platform: "local",
commit_hash: "00a46232ada7eebfb2b84d886239406436adbd3c",
ci_provider_metadata: CIProviderMetadata {
ref_: "00a46232ada7eebfb2b84d886239406436adbd3c",
head_ref: Some(
"main",
),
base_ref: None,
owner: "Jeremie-Chauvel",
repository: "codspeed-experiment",
event: Local,
sender: None,
gh_data: None,
gl_data: None,
repository_root_path: "/home/jeremiec/Documents/perso/codspeed/codspeed-experiments/",
},
}
Linked repository: Jeremie-Chauvel/codspeed-experiment

Preparing upload...
[DEBUG::reqwest::connect] starting new connection: https://api.codspeed.io/
[DEBUG::codspeed::run::uploader::upload] runId: 676d7943fe6f0dea6ffca2cf
Uploading performance data...
[DEBUG::codspeed::run::uploader::upload] Uploading 29209 bytes...
[DEBUG::reqwest::connect] starting new connection: https://prod-codspeed-storagestack-xxxxxxxx.s3.eu-west-1.amazonaws.com/
Performance data uploaded

FETCHING THE RESULTS

Fetching the results...
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
[DEBUG::reqwest::connect] starting new connection: https://gql.codspeed.io/
Impact: +7% (allowed regression: -10%)

To see the full report, visit: https://codspeed.io/Jeremie-Chauvel/codspeed-experiment/runs/676d7943fe6f0dea6ffca2cf

As you can see on https://codspeed.io/Jeremie-Chauvel/codspeed-experiment/runs/676d7943fe6f0dea6ffca2cf, seems that the report is empty (traces not available), might be related to the logs:

[DEBUG::codspeed::run::runner::valgrind::helpers::ignored_objects_path] Failed to get python shared objects: Some(Os { code: 2, kind: NotFound, message: "No such file or directory" })

Happy to help should you need anything, I might just have an incorrect assumption that local run create traces and not just a performance impact.

Cheers!