Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zcoin): allow ARRR to sync using a start date #1922

Merged
merged 46 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e7b459b
save dev state — impl LightWalletSyncParams for block synchronization…
borngraced Jul 19, 2023
eefe9d4
save dev state — add docs and minor changes
borngraced Jul 19, 2023
7585df2
save dev state — more changes to get_tree_state and coin params
borngraced Jul 19, 2023
f260b7a
save dev state — minor changes
borngraced Jul 19, 2023
56621c7
save dev state — minor changes to fnand variable namings
borngraced Jul 19, 2023
aabd924
fix review notes
borngraced Jul 21, 2023
0de4fbf
fix import error
borngraced Jul 21, 2023
5e4ab78
use appropriate error handling instead of unwrap and minor changes to…
borngraced Jul 24, 2023
8e9f78a
rename buffer -> avg_blocktime
borngraced Jul 24, 2023
3693272
fix review notes — refactoring
borngraced Jul 26, 2023
6b33b54
remove get_minimum_header_from_cache
borngraced Aug 1, 2023
c4b0426
Merge branch 'dev' into zcoin_light_client_opt
borngraced Aug 2, 2023
58748d9
fix buffer calculation and minor changes
borngraced Aug 3, 2023
b00ee84
add SyncConfError error type for UtxoConfError
borngraced Aug 3, 2023
7dde1c0
rename LightClientSyncParams to SyncStartPoint
borngraced Aug 3, 2023
3edb9de
remove ZcoinRpcError and fix review notes
borngraced Aug 4, 2023
dd55663
test github zcash param workflow for winx86 and linux
borngraced Aug 4, 2023
bf529d5
test github zcash param workflow for winx86
borngraced Aug 4, 2023
0aec022
add wget64 install path
borngraced Aug 4, 2023
86e7b50
Merge branch 'dev' into zcoin_light_client_opt
borngraced Aug 4, 2023
b7ebdaf
fix review notes and refactored WalletDbShared initialization
borngraced Aug 10, 2023
ea2a5f3
Merge remote-tracking branch 'origin/zcoin_light_client_opt' into zco…
borngraced Aug 10, 2023
d975aeb
ignore activate_pirate_light
borngraced Aug 10, 2023
5c62f87
allow activate_z_coin_light and ignore activate_pirate_light
borngraced Aug 13, 2023
4cd0c7c
remove dead code and add doc comment for checkpoint_block_from_height fn
borngraced Aug 14, 2023
617d595
remove GrpcError from WalletDbError
borngraced Aug 15, 2023
a79dff0
extend sync feature to modified sync height if it's pre-sapling and i…
borngraced Aug 16, 2023
4b2b3bb
fix wasm lint
borngraced Aug 16, 2023
4f6ee57
replaced additional_info with FirstSyncBlock
borngraced Aug 16, 2023
cdfa1a5
minor fix
borngraced Aug 17, 2023
d360972
fix older sync date bug
borngraced Aug 17, 2023
ed2279d
fix wasm lint
borngraced Aug 17, 2023
2d4eb0e
use sapling_activation_height if current block height is lesser than …
borngraced Aug 18, 2023
f5ca4f7
fix new sync height
borngraced Aug 18, 2023
3269c8e
fix integration test
borngraced Aug 18, 2023
8e2e622
make first_sync_block Optional in ZCoinActivationResult struct
borngraced Aug 18, 2023
bda9470
polish
borngraced Aug 18, 2023
f0cc28b
impl enable_z_coin_light_with_changing_height unit test
borngraced Aug 21, 2023
b7ea7df
fix non-blockers
borngraced Aug 22, 2023
a0fe7ea
use flatten in get_earliest_block
borngraced Aug 22, 2023
3470c93
using alias for ZcoinRpcMode::Light and use replace info! with log! i…
borngraced Aug 23, 2023
de50159
add alias for electrum_servers
borngraced Aug 29, 2023
75a807a
Merge remote-tracking branch 'origin/dev' into zcoin_light_client_opt
borngraced Sep 4, 2023
ca0dbdc
fix merge conflicts
borngraced Sep 4, 2023
04b8f6c
add doc comments
borngraced Sep 5, 2023
1b7c689
add remaining doc comments
borngraced Sep 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 47 additions & 26 deletions mm2src/coins/z_coin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ cfg_native!(
use zcash_primitives::transaction::builder::Builder as ZTxBuilder;
use zcash_proofs::default_params_folder;
use z_rpc::{init_native_client};

pub mod z_rpc_methods;
use crate::z_coin::z_rpc_methods::{GetMinimumHeightError, GetMinimumHeightResponse};
);

#[allow(unused)] mod z_coin_errors;
Expand Down Expand Up @@ -749,12 +752,46 @@ impl ZCoin {
paging_options: request.paging_options,
})
}

#[cfg(not(target_arch = "wasm32"))]
pub async fn get_minimum_header_from_cache(&self) -> MmResult<GetMinimumHeightResponse, GetMinimumHeightError> {
let sync = self
.sync_status()
.await
.map_err(|err| GetMinimumHeightError::TemporaryError(err.to_string()))?;
let wallet_db = self.z_fields.light_wallet_db.db.lock();
let extrema = wallet_db
.block_height_extrema()
.map_to_mm(|err| GetMinimumHeightError::StorageError(err.to_string()))?;

match sync {
SyncStatus::UpdatingBlocksCache { .. } => MmError::err(GetMinimumHeightError::UpdatingBlocksCache(
"Blocks cache still bulding, try again later".to_string(),
)),
SyncStatus::BuildingWalletDb { .. } => MmError::err(GetMinimumHeightError::BuildingWalletDb(
"Wallets cache still bulding, try again later".to_string(),
)),
SyncStatus::TemporaryError(_) => MmError::err(GetMinimumHeightError::TemporaryError(
"Temporary error occured".to_string(),
)),
SyncStatus::Finished { .. } => Ok(GetMinimumHeightResponse {
height: extrema.map(|(min, _max)| min.into()),
status: "Finished".to_string(),
}),
}
}
}

impl AsRef<UtxoCoinFields> for ZCoin {
fn as_ref(&self) -> &UtxoCoinFields { &self.utxo_arc }
}

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct LightWalletSyncParams {
pub date: Option<u32>,
pub height: Option<u32>,
}
shamardy marked this conversation as resolved.
Show resolved Hide resolved

#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(tag = "rpc", content = "rpc_data")]
pub enum ZcoinRpcMode {
laruh marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -763,6 +800,14 @@ pub enum ZcoinRpcMode {
Light {
electrum_servers: Vec<ElectrumRpcRequest>,
shamardy marked this conversation as resolved.
Show resolved Hide resolved
light_wallet_d_servers: Vec<String>,
/// Specifies the parameters for synchronizing the wallet from a specific block. This overrides the
/// `CheckPointBlockInfo` configuration in the coin settings.
///
/// # Note:
/// The `LightWalletSyncParams.date` field takes the highest priority, overriding both the
/// `LightWalletSyncParams.height` and `CheckPointBlockInfo` if specified. Followed by `LightWalletSyncParams
/// .height`.
sync_params: Option<LightWalletSyncParams>,
},
}

Expand Down Expand Up @@ -888,39 +933,15 @@ impl<'a> UtxoCoinBuilder for ZCoinBuilder<'a> {
);

let blocks_db = self.blocks_db().await?;
let wallet_db = WalletDbShared::new(&self)
.await
.map_err(|err| ZCoinBuildError::ZcashDBError(err.to_string()))?;

let (sync_state_connector, light_wallet_db) = match &self.z_coin_params.mode {
#[cfg(not(target_arch = "wasm32"))]
ZcoinRpcMode::Native => {
let native_client = self.native_client()?;
init_native_client(
self.ticker.into(),
native_client,
blocks_db,
wallet_db,
self.protocol_info.consensus_params.clone(),
self.z_coin_params.scan_blocks_per_iteration,
self.z_coin_params.scan_interval_ms,
)
.await?
init_native_client(&self, native_client, blocks_db).await?
},
ZcoinRpcMode::Light {
light_wallet_d_servers, ..
} => {
init_light_client(
self.ticker.into(),
light_wallet_d_servers.clone(),
blocks_db,
wallet_db,
self.protocol_info.consensus_params.clone(),
self.z_coin_params.scan_blocks_per_iteration,
self.z_coin_params.scan_interval_ms,
)
.await?
},
} => init_light_client(&self, light_wallet_d_servers.clone(), blocks_db).await?,
};
let z_fields = ZCoinFields {
dex_fee_addr,
Expand Down
113 changes: 107 additions & 6 deletions mm2src/coins/z_coin/storage/walletdb/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,16 @@ use crate::z_coin::{ZCoinBuilder, ZcoinClientInitError};
use mm2_err_handle::prelude::*;

cfg_native!(
use crate::z_coin::{extended_spending_key_from_protocol_info_and_policy, ZcoinConsensusParams};
use crate::z_coin::z_rpc::create_wallet_db;
use crate::z_coin::{CheckPointBlockInfo, extended_spending_key_from_protocol_info_and_policy, ZcoinConsensusParams,
ZcoinRpcMode};
use crate::z_coin::z_rpc::{create_wallet_db, ZRpcOps};

use common::now_sec;
use common::log::info;
use hex::{FromHex, FromHexError};
use parking_lot::Mutex;
use rpc::v1::types::{Bytes, H256};
use std::str::FromStr;
use std::sync::Arc;
use zcash_client_sqlite::WalletDb;
use zcash_primitives::zip32::ExtendedFullViewingKey;
Expand All @@ -21,6 +27,13 @@ pub enum WalletDbError {
ZcoinClientInitError(ZcoinClientInitError),
ZCoinBuildError(String),
IndexedDBError(String),
GrpcError(String),
DecodeError(String),
}

#[cfg(not(target_arch = "wasm32"))]
impl From<tonic::Status> for WalletDbError {
fn from(err: tonic::Status) -> Self { WalletDbError::GrpcError(err.to_string()) }
}

#[derive(Clone)]
Expand All @@ -33,27 +46,115 @@ pub struct WalletDbShared {
ticker: String,
}

/// Calculates the starting block height based on a given date and the current block height.
///
/// # Arguments
/// * `date`: The date in seconds representing the desired starting date.
/// * `current_block_height`: The current block height at the time of calculation.
///
#[cfg(not(target_arch = "wasm32"))]
fn calculate_starting_height_from_date(date: u32, current_block_height: u64) -> Result<u32, String> {
let blocks_prod_per_day = 24 * 60;
shamardy marked this conversation as resolved.
Show resolved Hide resolved
let current_time_s = now_sec();

let date = date as u64;
shamardy marked this conversation as resolved.
Show resolved Hide resolved
if current_time_s < date {
return Err("sync_param_date must be earlier then current date".to_string());
};

let secs_since_date = current_time_s - date;
let days_since_date = (secs_since_date / 86400) - 1;
shamardy marked this conversation as resolved.
Show resolved Hide resolved
let blocks_to_sync = (days_since_date * blocks_prod_per_day) + blocks_prod_per_day;
shamardy marked this conversation as resolved.
Show resolved Hide resolved
let block_to_sync_from = current_block_height - blocks_to_sync;

Ok(block_to_sync_from as u32)
shamardy marked this conversation as resolved.
Show resolved Hide resolved
}

#[cfg(not(target_arch = "wasm32"))]
impl<'a> WalletDbShared {
pub async fn new(zcoin_builder: &ZCoinBuilder<'a>) -> MmResult<Self, WalletDbError> {
pub async fn new(zcoin_builder: &ZCoinBuilder<'a>, rpc: &mut impl ZRpcOps) -> MmResult<Self, WalletDbError> {
let z_spending_key = match zcoin_builder.z_spending_key {
Some(ref z_spending_key) => z_spending_key.clone(),
None => extended_spending_key_from_protocol_info_and_policy(
&zcoin_builder.protocol_info,
&zcoin_builder.priv_key_policy,
)
.map_err(|err| WalletDbError::ZCoinBuildError(err.to_string()))?,
.mm_err(|err| WalletDbError::ZCoinBuildError(err.to_string()))?,
};

let sync_block = match zcoin_builder.z_coin_params.mode.clone() {
ZcoinRpcMode::Light { sync_params, .. } => {
let sync_height = match sync_params {
Some(params) => {
if let Some(date) = params.date {
let current_block_height = rpc
.get_block_height()
.await
.map_err(|err| WalletDbError::GrpcError(err.to_string()))?;

let starting_height = calculate_starting_height_from_date(date, current_block_height)
.map_err(|err| {
WalletDbError::ZcoinClientInitError(ZcoinClientInitError::ZcashDBError(err))
})?;

info!(
"Found date in sync params for {}. Walletdb will be built using block height {} as the \
starting \
block for scanning and updating walletdb",
zcoin_builder.ticker, starting_height
);
Some(starting_height)
} else {
if params.height.is_some() {
info!(
"Found height in sync params for {}. Walletdb will be built using block height {:?} as the \
starting block for scanning and updating walletdb",
zcoin_builder.ticker, params.height
);
}
params.height
}
},
None => None,
};

match sync_height {
Some(height) => {
let block = rpc
.get_tree_state(height)
.await
.map_err(|err| WalletDbError::GrpcError(err.to_string()))?;
let hash = H256::from_str(&block.hash)
.map_err(|err| WalletDbError::DecodeError(err.to_string()))?
.reversed();
let sapling_tree = Bytes::new(
FromHex::from_hex(&block.tree)
.map_err(|err: FromHexError| WalletDbError::DecodeError(err.to_string()))?,
);

Some(CheckPointBlockInfo {
height: block.height as u32,
hash,
time: block.time,
sapling_tree,
})
},
None => zcoin_builder.protocol_info.check_point_block.clone(),
shamardy marked this conversation as resolved.
Show resolved Hide resolved
}
},
ZcoinRpcMode::Native => zcoin_builder.protocol_info.check_point_block.clone(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Native should be the same as light client as discussed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be done in another PR as we agreed. Please don't forget to open an issue for it.

};

let wallet_db = create_wallet_db(
zcoin_builder
.db_dir_path
.join(format!("{}_wallet.db", zcoin_builder.ticker)),
zcoin_builder.protocol_info.consensus_params.clone(),
zcoin_builder.protocol_info.check_point_block.clone(),
sync_block,
ExtendedFullViewingKey::from(&z_spending_key),
)
.await
.map_err(|err| MmError::new(WalletDbError::ZcoinClientInitError(err.into_inner())))?;
.mm_err(WalletDbError::ZcoinClientInitError)?;

Ok(Self {
db: Arc::new(Mutex::new(wallet_db)),
Expand Down
26 changes: 26 additions & 0 deletions mm2src/coins/z_coin/z_coin_errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -279,3 +279,29 @@ pub enum SpendableNotesError {

#[derive(Debug, Display)]
pub enum ZCoinBalanceError {}

#[derive(Debug, Display)]
pub enum ZcoinRpcError {
#[cfg(not(target_arch = "wasm32"))]
GrpcError(tonic::Status),
UtxoRpcError(UtxoRpcError),
JsonRpcError(JsonRpcError),
shamardy marked this conversation as resolved.
Show resolved Hide resolved
UpdateBlocksCacheErr(UpdateBlocksCacheErr),
}

#[cfg(not(target_arch = "wasm32"))]
impl From<tonic::Status> for ZcoinRpcError {
fn from(err: tonic::Status) -> Self { Self::GrpcError(err) }
}

impl From<UtxoRpcError> for ZcoinRpcError {
fn from(err: UtxoRpcError) -> Self { ZcoinRpcError::UtxoRpcError(err) }
}

impl From<JsonRpcError> for ZcoinRpcError {
fn from(err: JsonRpcError) -> Self { ZcoinRpcError::JsonRpcError(err) }
}

impl From<UpdateBlocksCacheErr> for ZcoinRpcError {
fn from(err: UpdateBlocksCacheErr) -> Self { ZcoinRpcError::UpdateBlocksCacheErr(err) }
}
Loading