Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make file locations appear in zebrad error messages #4956

Closed
Tracked by #3096
teor2345 opened this issue Aug 26, 2022 · 6 comments
Closed
Tracked by #3096

Make file locations appear in zebrad error messages #4956

teor2345 opened this issue Aug 26, 2022 · 6 comments
Labels
A-diagnostics Area: Diagnosing issues or monitoring performance A-rust Area: Updates to Rust code C-bug Category: This is a bug C-enhancement Category: This is an improvement

Comments

@teor2345
Copy link
Contributor

teor2345 commented Aug 26, 2022

Motivation

When Zebra's Rust test harness fails in CI, I can't see the file and line in the panic backtrace:

12: zebra_test::command::TestChild<T>::expect_stdout_line_matches::h471f18611d79fb45
    at <unknown source file>:<unknown line>
13: acceptance::lightwalletd_integration_test::h1e497ff748a26c03
    at <unknown source file>:<unknown line>

https://github.com/ZcashFoundation/zebra/runs/7992118921?check_suite_focus=true#step:6:246

This happens in both Docker, and in release builds on my local machine.

The application panicked (crashed).
Message:  Opening database "/home/.../.cache/zebra/state/v25/mainnet" failed: Error { message: "IO error: While lock file: /home/.../.cache/zebra/state/v25/mainnet/LOCK: Resource temporarily unavailable" }. Hint: Check if another zebrad process is running. Try changing the state cache_dir in the Zebra config.
Location: zebra-state/src/service/finalized_state/disk_db.rs:460
...
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 7 frames hidden ⋮                               
   8: zebra_state::service::finalized_state::disk_db::DiskDb::new::h0adc7941d0b09e1e
      at <unknown source file>:<unknown line>
   9: zebra_state::service::finalized_state::FinalizedState::new::hd85ae10e187a7755
      at <unknown source file>:<unknown line>
...

But debug builds on my local machine show the file locations:

9: zebra_state::service::finalized_state::disk_db::DiskDb::new::hfc24129cd47798c6
      at /home/dev/zebra/zebra-state/src/service/finalized_state/disk_db.rs:<unknown line>
  10: zebra_state::service::finalized_state::zebra_db::ZebraDb::new::hcc01b459aaf10f3f
      at /home/dev/zebra/zebra-state/src/service/finalized_state/zebra_db.rs:50
  11: zebra_state::service::finalized_state::FinalizedState::new::hbcbdd50bfd733126
      at /home/dev/zebra/zebra-state/src/service/finalized_state.rs:84

Tasks

Try adding debugging info to the Rust release profile:

[profile.release]

@teor2345 teor2345 added C-bug Category: This is a bug A-rust Area: Updates to Rust code A-devops Area: Pipelines, CI/CD and Dockerfiles C-enhancement Category: This is an improvement S-needs-triage Status: A bug report needs triage P-Medium ⚡ A-diagnostics Area: Diagnosing issues or monitoring performance labels Aug 26, 2022
@mpguerra mpguerra moved this to 🆕 New in Zebra Sep 22, 2022
@mpguerra mpguerra added this to Zebra Sep 22, 2022
@teor2345
Copy link
Contributor Author

This makes diagnosing errors in the docker tests tricky, but it can wait until next sprint.

@mpguerra mpguerra removed the S-needs-triage Status: A bug report needs triage label Sep 29, 2022
@teor2345
Copy link
Contributor Author

teor2345 commented Oct 4, 2022

This important, but it's not causing any particular issues right now.

@mpguerra
Copy link
Contributor

@gustavovalverde does it make sense to look at this issue opportunistically since we're making other changes to the Dockerfile? Especifically this part:

Try changing the build working directory to the zebrad source directory:

WORKDIR /opt/zebrad

@gustavovalverde
Copy link
Member

After I'm done with #5933 I might add this one, as #5933 is pilling up with untagged images in the Registry.

@teor2345 teor2345 changed the title Make file locations appear in zebrad Docker error messages Make file locations appear in zebrad error messages Jan 23, 2023
@teor2345 teor2345 removed the A-devops Area: Pipelines, CI/CD and Dockerfiles label Jan 24, 2023
@teor2345
Copy link
Contributor Author

@gustavovalverde @mpguerra I just re-checked this on my local machine, and it happens in all release builds, even outside Docker.

So I don't think we can fix it by changing the Docker config.

@teor2345
Copy link
Contributor Author

I had a look at this, we can fix this issue using the debug setting in .cargo/config.toml:
https://doc.rust-lang.org/cargo/reference/profiles.html#debug

But the debug info adds 300 MB to the zebrad binary, making it 11 times its usual 29 MB size. I'm not sure if that's worth it. (For comparison, zcashd is 18 MB.)

Since this hasn't been a blocker for diagnosing any bugs, I made it into a branch which developers can use if we need it:
https://github.com/ZcashFoundation/zebra/tree/detailed-backtraces

@teor2345 teor2345 closed this as not planned Won't fix, can't repro, duplicate, stale Jan 24, 2023
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Zebra Jan 24, 2023
@mpguerra mpguerra moved this from ✅ Done to 🛑 Won't Fix in Zebra Jan 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-diagnostics Area: Diagnosing issues or monitoring performance A-rust Area: Updates to Rust code C-bug Category: This is a bug C-enhancement Category: This is an improvement
Projects
Archived in project
Development

No branches or pull requests

3 participants