Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ignore] experiment and investigation #15617

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions execution/executor-benchmark/src/transaction_committer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ use std::{
time::{Duration, Instant},
};

use aptos_db::utils::ShardedStateKvSchemaBatch;

pub(crate) fn gen_li_with_sigs(
block_id: HashValue,
root_hash: HashValue,
Expand All @@ -47,10 +49,18 @@ pub(crate) fn gen_li_with_sigs(
)
}

// TODO (bowu)
pub struct CommitBatches {
state_kv_metadata_batch: SchemaBatch,
sharded_state_kv_batches: ShardedStateKvSchemaBatch,
}

pub struct TransactionCommitter<V> {
executor: Arc<BlockExecutor<V>>,
start_version: Version,
block_receiver: mpsc::Receiver<CommitBlockMessage>,
batch_sender: mpsc::Sender<CommitBatches>,
batch_receiver: mpsc::Receiver<CommitBatches>,
}

impl<V> TransactionCommitter<V>
Expand All @@ -62,16 +72,38 @@ where
start_version: Version,
block_receiver: mpsc::Receiver<CommitBlockMessage>,
) -> Self {
// spawn a new thread in backgrond to do the actual commit
let (batch_sender, batch_receiver) = mpsc::channel();

Self {
executor,
start_version,
block_receiver,
batch_sender,
batch_receiver,
}
}

fn commit_batch(&self, batch: SchemaBatch) -> Result<()> {
Ok(())
}
Comment on lines +87 to +89
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit_batch() method currently returns success without performing any operations, which will silently discard batches instead of persisting them. This empty implementation needs to be replaced with logic that actually commits the SchemaBatch to storage to prevent data loss. Consider adding error handling and verification of the commit as well.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.


fn prepare_commit(&self, block_id: u64, ledger_info_sigs: LedgerInfoWithSignatures) -> Result<()> {
self.executor.pre_commit_block(block_id)?;
self.executor.commit_ledger(ledger_info_sigs)?;
Ok(())
}

pub fn run(&mut self) {
info!("Start with version: {}", self.start_version);

// Spawn a new thread in backgrond to do the actual commit
let commit_thread = thread::spawn(move || {
while let Ok(batch) = self.batch_receiver.recv() {
self.commit_batch(batch).unwrap();
}
});
Comment on lines +101 to +105
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spawned commit_thread needs to be joined to ensure all commits are processed before the thread terminates. Consider storing the JoinHandle and joining it at the end of run() or in Drop implementation. This prevents potential data loss from premature thread termination.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.


while let Ok(msg) = self.block_receiver.recv() {
let CommitBlockMessage {
block_id,
Expand All @@ -93,8 +125,7 @@ where
let version = output.expect_last_version();
let commit_start = Instant::now();
let ledger_info_with_sigs = gen_li_with_sigs(block_id, root_hash, version);
self.executor.pre_commit_block(block_id).unwrap();
self.executor.commit_ledger(ledger_info_with_sigs).unwrap();
self.prepare_commit(block_id, ledger_info_sigs).unwrap();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable ledger_info_with_sigs is assigned on line 94, but is passed as ledger_info_sigs to prepare_commit(). Please update the argument name to match the original variable.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.


report_block(
self.start_version,
Expand Down
6 changes: 4 additions & 2 deletions execution/executor/src/block_executor/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,16 +50,18 @@ pub mod block_tree;
pub struct BlockExecutor<V> {
pub db: DbReaderWriter,
inner: RwLock<Option<BlockExecutorInner<V>>>,
sender: Option<mpsc::Sender<CommitBatches>>,
}

impl<V> BlockExecutor<V>
where
V: VMBlockExecutor,
{
pub fn new(db: DbReaderWriter) -> Self {
pub fn new(db: DbReaderWriter, sender: Option<mpsc::Sender<CommitBatches>>) -> Self {
Self {
db,
inner: RwLock::new(None),
sender,
}
}

Expand Down Expand Up @@ -124,7 +126,7 @@ where
.ledger_update(block_id, parent_block_id)
}

fn pre_commit_block(&self, block_id: HashValue) -> ExecutorResult<()> {
fn pre_commit_block(&self, block_id: HashValue, sender:) -> ExecutorResult<()> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There appears to be a syntax error in the function signature - the parameter sender: has a type annotation missing and ends with an erroneous colon. This will prevent compilation.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

let _guard = CONCURRENCY_GAUGE.concurrency_with(&["block", "pre_commit_block"]);

self.inner
Expand Down
Empty file.
Loading