feat: forester: multiple trees + rollover support #998
Conversation
sergeytimoshin
force-pushed
the
sergey/forester-trees
branch
2 times, most recently
from
d3d13a4 to
0742fa0
Compare
sergeytimoshin
requested review from
vadorovsky,
SwenSchaeferjohann and
ananas-block
as code owners
sergeytimoshin
force-pushed
the
sergey/forester-trees
branch
from
f7ffdd6 to
7abd47a
Compare
forester/src/config.rs
Outdated
| #[derive(Debug, Clone)] | ||
| pub struct TreeSyncMetadata { | ||
| pub last_synced_at: DateTime<Utc>, | ||
| pub last_synced_root: [u8; 32], |
There was a problem hiding this comment.
what's the conceptual idea behind this?
how often do you sync trees?
There was a problem hiding this comment.
Removed it for now.
I sync the trees on every start, but that's obviously not enough, and syncing on every rollover is not enough either.
I see 2 options:
- Separate thread that would subscribe to account_compression account changes and do a sync on each change.
- Just a cron job, every N minutes.
There was a problem hiding this comment.
The forester epochs and slots will change this a bit. (Foresters are only eligible to forest during their own slots.)
It’s probably enough to sync the tree leading up and during the slot.
forester/src/main.rs
Outdated
| async fn run_subscribe_addresses<R: RpcConnection>( | ||
| config: Arc<ForesterConfig>, | ||
| rpc_pool: RpcPool<R>, | ||
| tree_data: TreeData, | ||
| rollover_state: Arc<RolloverState>, | ||
| ) { | ||
| let indexer_rpc = init_rpc(config.clone(), false).await; | ||
| let indexer = Arc::new(tokio::sync::Mutex::new(PhotonIndexer::new( | ||
| config.external_services.indexer_url.to_string(), | ||
| indexer_rpc, | ||
| ))); | ||
|
|
||
| subscribe_addresses(config.clone(), rpc_pool, indexer).await; | ||
| subscribe_addresses(config.clone(), rpc_pool, indexer, tree_data, rollover_state).await; | ||
| } | ||
|
|
||
| async fn run_nullify_state<R: RpcConnection>(config: Arc<ForesterConfig>, rpc_pool: RpcPool<R>) { | ||
| async fn run_nullify_addresses<R: RpcConnection>( | ||
| config: Arc<ForesterConfig>, | ||
| rpc_pool: RpcPool<R>, | ||
| tree_data: TreeData, | ||
| rollover_state: Arc<RolloverState>, | ||
| ) { | ||
| let indexer_rpc = init_rpc(config.clone(), false).await; | ||
| let indexer = Arc::new(tokio::sync::Mutex::new(PhotonIndexer::new( | ||
| config.external_services.indexer_url.to_string(), | ||
| indexer_rpc, | ||
| ))); | ||
|
|
||
| nullify_state(config.clone(), rpc_pool, indexer).await; | ||
| nullify_addresses(config.clone(), rpc_pool, indexer, tree_data, rollover_state).await; | ||
| } |
There was a problem hiding this comment.
Not sure how much but it feels like there is some duplicate code in this file.
It can probably be unified more or redundant functions removed.
forester/src/rollover/operations.rs
Outdated
There was a problem hiding this comment.
really would like to move code like this which is duplicate in forester and test-utils into a forester-utils crate which forester and test-utils depend on.
There was a problem hiding this comment.
I started to extract duplicated code from forester & test-utils, and I think it would be better to make it in different PR, too many changes there.
sergeytimoshin
force-pushed
the
sergey/forester-trees
branch
2 times, most recently
from
3367e25 to
65c73c4
Compare
sergeytimoshin
force-pushed
the
sergey/forester-trees
branch
from
65c73c4 to
a63c02a
Compare
No description provided.