Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve load-from-logs CLI command #1132

Merged
merged 1 commit into from
Sep 5, 2024
Merged

Conversation

glevco
Copy link
Contributor

@glevco glevco commented Sep 5, 2024

Motivation

Currently, the load-from-logs CLI command reads a raw logs file as outputted by a full node. This PR changes it so instead you have to process the log file with the parse-logs command, only then to use it with load-from-logs. This makes the log files much smaller, as the parser strips every information but the vertex bytes.

Also, previously that command only called on_new_tx in a loop, but didn't actually run the reactor or the full node itself. This PR also changes this.

The new intended workflow would be something like this:

# Run a full node and save its logs to a file
$ poetry run hathor-cli run_node --testnet --memory-storage --json-logs --log-vertex-bytes &> json_logs
# After stopping the full node, parse the logs file
$ poetry run hathor-cli parse-logs --output-file parsed_logs --json-logs-file json_logs
# Run load-from-logs to create a full node that receives each vertex from the log file
$ poetry run hathor-cli load-from-logs --testnet --memory-storage --x-localhost-only --log-dump parsed_logs

Acceptance Criteria

  • Implement new parse-logs CLI command.
  • Update load-from-logs CLI command so it takes files from parse-logs and so it actually runs a full node.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@glevco glevco self-assigned this Sep 5, 2024
Copy link

github-actions bot commented Sep 5, 2024

🐰Bencher

ReportThu, September 5, 2024 at 15:22:31 UTC
Projecthathor-core
Branchfeat/improve-load-from-logs
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyLatency Results
nanoseconds (ns) | (Δ%)
Latency Lower Boundary
nanoseconds (ns) | (%)
Latency Upper Boundary
nanoseconds (ns) | (%)
sync-v2 (up to 20000 blocks)✅ (view plot)101,409,921,726.48 (-0.77%)91,981,574,820.96 (90.70%)112,421,924,781.17 (90.20%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

@glevco glevco marked this pull request as ready for review September 5, 2024 15:32
Copy link

codecov bot commented Sep 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.41%. Comparing base (50ae3b2) to head (0703970).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1132      +/-   ##
==========================================
+ Coverage   84.39%   84.41%   +0.01%     
==========================================
  Files         317      317              
  Lines       24240    24240              
  Branches     3684     3684              
==========================================
+ Hits        20458    20462       +4     
+ Misses       3069     3065       -4     
  Partials      713      713              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@glevco glevco merged commit ec2563e into master Sep 5, 2024
13 checks passed
@glevco glevco deleted the feat/improve-load-from-logs branch September 5, 2024 19:02
@jansegre jansegre mentioned this pull request Oct 4, 2024
2 tasks
@jansegre jansegre mentioned this pull request Dec 11, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants