Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exp 191 refactor inscription manager #249

Merged
merged 19 commits into from
Aug 30, 2024

Conversation

bewakes
Copy link
Contributor

@bewakes bewakes commented Aug 23, 2024

Description

This is a refatoring of inscription/writer task.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor

Checklist

  • I have performed a self-review of my code.
  • I have commented my code where necessary.
  • I have updated the documentation if needed.
  • My changes do not introduce new warnings.
  • I have added tests that prove my changes are effective or that my feature works.
  • New and existing tests pass with my changes.

Related Issues

Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall a good start.

crates/btcio/src/broadcaster/handle.rs Outdated Show resolved Hide resolved
crates/storage/src/managers/inscription.rs Outdated Show resolved Hide resolved
@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch 4 times, most recently from 427ed2d to 6ffd0ad Compare August 26, 2024 13:27
@bewakes bewakes marked this pull request as ready for review August 27, 2024 08:16
@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch from 1505ad5 to 294434d Compare August 27, 2024 08:36
@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch from 294434d to 0f46f55 Compare August 28, 2024 10:01
Copy link

codecov bot commented Aug 28, 2024

Codecov Report

Attention: Patch coverage is 50.67873% with 218 lines in your changes missing coverage. Please review.

Project coverage is 58.71%. Comparing base (538be4d) to head (fc691bb).
Report is 29 commits behind head on master.

Files with missing lines Patch % Lines
crates/btcio/src/writer/task.rs 21.26% 174 Missing ⚠️
sequencer/src/main.rs 0.00% 16 Missing ⚠️
crates/storage/src/ops/inscription.rs 70.73% 12 Missing ⚠️
crates/consensus-logic/src/duty/worker.rs 0.00% 8 Missing ⚠️
sequencer/src/rpc_server.rs 0.00% 5 Missing ⚠️
crates/btcio/src/broadcaster/task.rs 71.42% 2 Missing ⚠️
crates/btcio/src/writer/config.rs 0.00% 1 Missing ⚠️
@@            Coverage Diff             @@
##           master     #249      +/-   ##
==========================================
+ Coverage   57.90%   58.71%   +0.81%     
==========================================
  Files         119      123       +4     
  Lines       13425    13406      -19     
==========================================
+ Hits         7774     7872      +98     
+ Misses       5651     5534     -117     
Files with missing lines Coverage Δ
crates/btcio/src/broadcaster/error.rs 0.00% <ø> (ø)
crates/btcio/src/broadcaster/handle.rs 42.42% <100.00%> (+42.42%) ⬆️
crates/btcio/src/broadcaster/state.rs 96.72% <100.00%> (+1.72%) ⬆️
crates/btcio/src/writer/builder.rs 97.85% <ø> (+2.21%) ⬆️
crates/btcio/src/writer/signer.rs 100.00% <100.00%> (ø)
crates/btcio/src/writer/test_utils.rs 100.00% <100.00%> (ø)
crates/db/src/traits.rs 100.00% <ø> (ø)
crates/db/src/types.rs 100.00% <ø> (ø)
crates/primitives/src/buf.rs 79.04% <100.00%> (+1.19%) ⬆️
crates/rocksdb-store/src/sequencer/db.rs 99.16% <100.00%> (+0.64%) ⬆️
... and 10 more

... and 9 files with indirect coverage changes

@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch from 0f46f55 to 2b4b502 Compare August 28, 2024 12:03
@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch from 2b4b502 to 910d1bb Compare August 28, 2024 12:35
@bewakes bewakes force-pushed the EXP-191-refactor-inscription-manager branch from 910d1bb to 1cb8fbc Compare August 28, 2024 12:39
crates/btcio/src/broadcaster/task.rs Show resolved Hide resolved
crates/btcio/src/writer/signer.rs Outdated Show resolved Hide resolved
crates/btcio/src/writer/signer.rs Outdated Show resolved Hide resolved
crates/btcio/src/writer/signer.rs Outdated Show resolved Hide resolved
crates/btcio/src/writer/signer.rs Outdated Show resolved Hide resolved
sequencer/src/main.rs Outdated Show resolved Hide resolved
sequencer/src/main.rs Outdated Show resolved Hide resolved
sequencer/src/main.rs Outdated Show resolved Hide resolved
sequencer/src/rpc_server.rs Show resolved Hide resolved
sequencer/src/rpc_server.rs Outdated Show resolved Hide resolved
Copy link
Member

@storopoli storopoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fc691bb
Overall this PR is ready to be merged. I don't want to bikeshed much here.
Good work!

@bewakes bewakes dismissed delbonis’s stale review August 30, 2024 00:39

The change requests have been made.

@bewakes bewakes merged commit ed8f35c into master Aug 30, 2024
14 of 15 checks passed
Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inscription worker similarly to the earlier version of the broadcaster worker relies on polling the database to be notified of new work. This adds latency and io overhead to the system, so it should instead use a channel to wake up the inscription worker when there's new work to do. It should still timeout and poll to check to see if any of the txs it was waiting on got finalized or anything (also eventually we'd want this to be done based on a wakeup from the L1 reader task) and update whatever it needs to in the database, which means we can use a much slower poll rate without compromising latency.

) -> anyhow::Result<(Buf32, Buf32)> {
let (commit, reveal) = build_inscription_txs(&blobentry.blob, &client, config).await?;

debug!("Signing commit transaction {}", commit.compute_txid());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use a param like:

let commit_txid = commit.compute_txid();
debug!(%commit_txid, "signing inscription commit tx");

so that we can search the logs more easily

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the changes in this commit: 77ed0fa

Comment on lines +36 to +47
warn!("Received intent not meant for L1: {}", intent.commitment());
return Ok(());
}

let entry = BlobEntry::new_unsigned(intent.payload().to_vec());
debug!("Received intent: {}", intent.commitment());
if self
.ops
.get_blob_entry_blocking(*intent.commitment())?
.is_some()
{
warn!("Received duplicate intent {:?}", intent.commitment());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all use param logs instead of making it part of the message.

.ops
.put_blob_entry_async(*intent.commitment(), entry)
.await?)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same things about the params here.

l1_status,
)
.await
.unwrap()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to unwrap this error here? I might not remember the specifics about how the task manager should be used but don't we want to bubble the error types up to it to let it do the error handling instead of panicking and not being able to pick up the structured errors?

loop {
interval.as_mut().tick().await;

if let Some(blobentry) = insc_ops.get_blob_entry_by_idx_async(curr_blobidx).await? {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still relying on polling the database instead of being woken up by a message on a channel in the writer handle indicating that there's new blobs to sign, right?

Comment on lines +25 to +28
pub struct InscriptionHandle {
ops: Arc<InscriptionDataOps>,
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs some way to signal to the writer task that there's new blobs to sign instead of waiting on the task to poll the database.

Also should be called InscriberHandle or InscriptionWorkerHandle or something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this polling will not be that frequent as we are posting inscriptions every 10 mins or so, polling db didn't seem that concerning and is much simpler. The broadcaster does have the signaling because it needs to broadcast frequently as txs might come from L1Writer or from the bridge operator.

Comment on lines 522 to 523
// NOTE: It would be nice to return reveal txid from the submit method. But creation of txs
// is deferred to signer in the writer module
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could return the blob index here.

@storopoli storopoli deleted the EXP-191-refactor-inscription-manager branch September 4, 2024 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants