nix copy --to s3:// progress bar is incorrect

[arianvp]

Describe the bug

When uploading to S3 with nix copy the progress bar will show hundreds of thousands of megabytes copied for a 1GB closure

Steps To Reproduce

[arian@framework:~/Projects/terraform-aws-ssm-nixos-deploy-document/examples/nix-store-path]$ nix copy --to "s3://cache2024912328328&secret-key=./key.sec" "${out_path}"
[8/460/639 copied (150616.3/143782.0 MiB)] 

Expected behavior

The progress bar shows a sane number. Not 150616.3/143782.0 MiB

Metadata

nix --version
nix (Nix) 2.24.9

Additional context

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[Mic92]

Is this maybe caused by transparent http compression? I noticed something similar when downloading nars from harmonia and zstd compression.

[Mic92]

Mhm. But than the value should be actually smaller instead of bigger.

[Mic92]

In any case, can you see compression beeing used for upload?
Run nix copy -vvvvvvv (or add enough -v until you can see http headers).

[NaN-git]

Is it possible to develop some test cases for this or a minimal example?

Some of the logic seems to be quite dirty. According to my understanding the progress bar should not work for S3 at all. I wrote some code, but I don't know whether it actually works: https://github.com/NaN-git/nix/tree/s3-upload

Can someone test it?