Split metrics into data metrics and server metrics

[drmingdrmer]

RaftMetrics encapsulates a range of information about the internal states of
a Raft node, including the server state (e.g., Leader, Follower), and
data-related states such as the last log index and the last applied log index.

Applications interested in monitoring these details can subscribe to
RaftMetrics to receive updates when these metrics change, with
Raft::metrics().

However, some metrics, like last_log_index, are frequently updated, while
others, such as current_term and state, change less often.

For applications solely concerned with server state transitions (like switching
from Leader to Follower), being notified of changes to last_log_index can
result in unnecessary wake-ups.

To optimize this, it would be beneficial to add two subsets of the metrics:

• Metrics that are not data-related, such as current_term, vote, and state.
• Metrics that are data-related, including last_log_index and last_applied.

By segregating these metrics, applications can subscribe to the relevant subset
and reduce the overhead of processing frequent updates that are not pertinent to
their needs.

TODO:

• add Raft::data_metrics() to return a watch::Receiver<RaftDataMetrics>
  that contains only metrics about log, state machine and snapshot.

• add Raft::server_metrics() to return a
  watch::Receiver<RaftServerMetrics> that contains only server related
  metrics, such as vote, leader, id, running_state, membership

Checklist

• Modify openraft/src/metrics/raft_metrics.rs ✓ 98ef2e3 Edit
• Running GitHub Actions for openraft/src/metrics/raft_metrics.rs ⋯ Edit
• Modify openraft/src/core/raft_core.rs ▶ Edit
• Running GitHub Actions for openraft/src/core/raft_core.rs ▶ Edit
• Modify openraft/src/core/raft_core.rs ▶ Edit
• Running GitHub Actions for openraft/src/core/raft_core.rs ▶ Edit

[sweep-ai]

✨ Track Sweep's progress on our progress dashboard!

⚡ Sweep Basic Tier: I'm using GPT-4. You have 5 GPT-4 tickets left for the month and 3 for the day. (tracking ID: 61ec7617ff)

For more GPT-4 tickets, visit our payment portal. For a one week free trial, try Sweep Pro (unlimited GPT-4 tickets).

---

Tip

I'll email you at drdr.xp@gmail.com when I complete this pull request!

Install Sweep Configs: Pull Request

---

Actions (click)

• ↻ Restart Sweep

GitHub Actions✓

Here are the GitHub Actions logs prior to making any changes:

Sandbox logs for ab7f309

Checking openraft/src/metrics/raft_metrics.rs for syntax errors... ✅ openraft/src/metrics/raft_metrics.rs has no syntax errors! 1/1 ✓

Checking openraft/src/metrics/raft_metrics.rs for syntax errors...
✅ openraft/src/metrics/raft_metrics.rs has no syntax errors!

Sandbox passed on the latest main, so sandbox checks will be enabled for this issue.

---

Step 1: 🔎 Searching

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I think are relevant in decreasing order of relevance (click to expand). If some file is missing from here, you can mention the path in the ticket description.

openraft/openraft/src/metrics/raft_metrics.rs

Lines 17 to 69 in ab7f309

	#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize), serde(bound = ""))]
	pub struct RaftMetrics<NID, N>
	where
	NID: NodeId,
	N: Node,
	{
	pub running_state: Result<(), Fatal<NID>>,
	/// The ID of the Raft node.
	pub id: NID,
	// ---
	// --- data ---
	// ---
	/// The current term of the Raft node.
	pub current_term: u64,
	/// The last accepted vote.
	pub vote: Vote<NID>,
	/// The last log index has been appended to this Raft node's log.
	pub last_log_index: Option<u64>,
	/// The last log index has been applied to this Raft node's state machine.
	pub last_applied: Option<LogId<NID>>,
	/// The id of the last log included in snapshot.
	/// If there is no snapshot, it is (0,0).
	pub snapshot: Option<LogId<NID>>,
	/// The last log id that has purged from storage, inclusive.
	///
	/// `purged` is also the first log id Openraft knows, although the corresponding log entry has
	/// already been deleted.
	pub purged: Option<LogId<NID>>,
	// ---
	// --- cluster ---
	// ---
	/// The state of the Raft node.
	pub state: ServerState,
	/// The current cluster leader.
	pub current_leader: Option<NID>,
	/// The current membership config of the cluster.
	pub membership_config: Arc<StoredMembership<NID, N>>,
	// ---
	// --- replication ---
	// ---
	/// The replication states. It is Some() only when this node is leader.
	pub replication: Option<ReplicationMetrics<NID>>,

openraft/openraft/src/core/raft_core.rs

Lines 521 to 551 in ab7f309

	#[tracing::instrument(level = "debug", skip_all)]
	pub(crate) fn report_metrics(&self, replication: Option<ReplicationMetrics<C::NodeId>>) {
	let st = &self.engine.state;
	let m = RaftMetrics {
	running_state: Ok(()),
	id: self.id,
	// --- data ---
	current_term: st.vote_ref().leader_id().get_term(),
	vote: *st.io_state().vote(),
	last_log_index: st.last_log_id().index(),
	last_applied: st.io_applied().copied(),
	snapshot: st.io_snapshot_last_log_id().copied(),
	purged: st.io_purged().copied(),
	// --- cluster ---
	state: st.server_state,
	current_leader: self.current_leader(),
	membership_config: st.membership_state.effective().stored_membership().clone(),
	// --- replication ---
	replication,
	};
	tracing::debug!("report_metrics: {}", m.summary());
	let res = self.tx_metrics.send(m);
	if let Err(err) = res {
	tracing::error!(error=%err, id=display(self.id), "error reporting metrics");
	}

I also found the following external resources that might be helpful:

Summaries of links found in the content:

https://github.com/datafuselabs/openraft/blob/147de251551c1ed364bd60bbe57047393e1d625e/openraft/src/metrics/raft_metrics.rs#L18:

The page describes the RaftMetrics struct in the raft_metrics.rs file of the openraft repository. RaftMetrics encapsulates various information about the internal states of a Raft node, including the server state, last log index, last applied log index, and more. The page suggests adding two subsets of metrics: one for data-related metrics and another for server-related metrics. This would allow applications to subscribe to the relevant subset of metrics and reduce unnecessary updates. The page also proposes adding Raft::data_metrics() and Raft::server_metrics() methods to retrieve the subsets of metrics.

---

Step 2: ⌨️ Coding

• Modify openraft/src/metrics/raft_metrics.rs ✓ 98ef2e3 Edit

Modify openraft/src/metrics/raft_metrics.rs with contents:
• Split the `RaftMetrics` struct into two separate structs: `RaftDataMetrics` and `RaftServerMetrics`.
• The `RaftDataMetrics` struct should contain the following fields: `last_log_index`, `last_applied`, `snapshot`, and `purged`.
• The `RaftServerMetrics` struct should contain the following fields: `running_state`, `id`, `current_term`, `vote`, `state`, `current_leader`, `membership_config`, and `replication`.
• Ensure that both new structs derive the necessary traits for serialization and deserialization.

--- 
+++ 
@@ -12,28 +12,14 @@
 use crate::StoredMembership;
 use crate::Vote;
 
-/// A set of metrics describing the current state of a Raft node.
+/// A set of data-related metrics describing the current state of a Raft node.
 #[derive(Clone, Debug, PartialEq, Eq)]
 #[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize), serde(bound = ""))]
-pub struct RaftMetrics
+pub struct RaftDataMetrics
 where
     NID: NodeId,
     N: Node,
 {
-    pub running_state: Result<(), Fatal>,
-
-    /// The ID of the Raft node.
-    pub id: NID,
-
-    // ---
-    // --- data ---
-    // ---
-    /// The current term of the Raft node.
-    pub current_term: u64,
-
-    /// The last accepted vote.
-    pub vote: Vote,
-
     /// The last log index has been appended to this Raft node's log.
     pub last_log_index: Option,
 
@@ -45,7 +31,38 @@
     pub snapshot: Option>,
 
     /// The last log id that has purged from storage, inclusive.
-    ///
+    pub purged: Option>,
+/// A set of server-related metrics describing the current state of a Raft node.
+#[derive(Clone, Debug, PartialEq, Eq)]
+#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize), serde(bound = ""))]
+pub struct RaftServerMetrics
+where
+    NID: NodeId,
+    N: Node,
+{
+    pub running_state: Result<(), Fatal>,
+
+    /// The ID of the Raft node.
+    pub id: NID,
+
+    /// The current term of the Raft node.
+    pub current_term: u64,
+
+    /// The last accepted vote.
+    pub vote: Vote,
+
+    /// The state of the Raft node.
+    pub state: ServerState,
+
+    /// The current cluster leader.
+    pub current_leader: Option,
+
+    /// The current membership config of the cluster.
+    pub membership_config: Arc>,
+
+    /// The replication states. It is Some() only when this node is leader.
+    pub replication: Option>,
+}
     /// `purged` is also the first log id Openraft knows, although the corresponding log entry has
     /// already been deleted.
     pub purged: Option>,

• Running GitHub Actions for openraft/src/metrics/raft_metrics.rs ⋯ Edit

Check openraft/src/metrics/raft_metrics.rs with contents:

Ran GitHub Actions for 98ef2e355a70fdc24178115c9cc265598c665ff9:
• tests-feature-test (nightly, loosen-follower-log-revert): ⋯
• tests-feature-test (nightly, single-term-leader): ⋯
• openraft-test (nightly, 0, single-term-leader): ⋯
• tests-feature-test (nightly): ⋯
• openraft-feature-test (nightly, single-term-leader,serde,singlethreaded): ⋯
• openraft-feature-test (nightly, single-term-leader): ⋯
• openraft-test (nightly, 30): ⋯
• openraft-feature-test (nightly, serde): ⋯
• Build (nightly, bench,serde,bt,singlethreaded): ⋯
• openraft-test (stable, 0): ⋯
• openraft-feature-test (nightly): ⋯
• lint: ⋯
• examples (nightly, raft-kv-rocksdb): ⋯
• stores (sledstore): ⋯
• examples (nightly, raft-kv-memstore): ⋯
• external-stores (stores/rocksstore-v2): ⋯
• examples (stable, raft-kv-rocksdb): ⋯
• stores (rocksstore-compat07): ⋯
• examples (stable, raft-kv-memstore): ⋯
• stores (rocksstore): ⋯
• cluster-benchmark (cluster_benchmark): ⋯
• stores (memstore): ⋯
• openraft-test-bench (nightly): ⋯
• check-subject: ⋯

• Modify openraft/src/core/raft_core.rs ▶ Edit

Modify openraft/src/core/raft_core.rs with contents:
• Modify the `report_metrics` function to create instances of the `RaftDataMetrics` and `RaftServerMetrics` structs instead of the `RaftMetrics` struct.
• Send the `RaftDataMetrics` and `RaftServerMetrics` instances via their respective channels.
• Handle any errors that may occur when sending the metrics.

• Running GitHub Actions for openraft/src/core/raft_core.rs ▶ Edit

Check openraft/src/core/raft_core.rs with contents:

• Modify openraft/src/core/raft_core.rs ▶ Edit

Modify openraft/src/core/raft_core.rs with contents:
• Add two new methods to the `Raft` struct: `data_metrics` and `server_metrics`.
• The `data_metrics` method should return a `watch::Receiver`.
• The `server_metrics` method should return a `watch::Receiver`.
• Both methods should be public and take no arguments.

• Running GitHub Actions for openraft/src/core/raft_core.rs ▶ Edit

Check openraft/src/core/raft_core.rs with contents:

---

Step 3: 🔁 Code Review

Working on it...

---

🎉 Latest improvements to Sweep:

• We just released a dashboard to track Sweep's progress on your issue in real-time, showing every stage of the process – from search to planning and coding.
• Sweep uses OpenAI's latest Assistant API to plan code changes and modify code! This is 3x faster and significantly more reliable as it allows Sweep to edit code and validate the changes in tight iterations, the same way as a human would.
• Try using the GitHub issues extension to create Sweep issues directly from your editor! GitHub Issues and Pull Requests.

---

💡 To recreate the pull request edit the issue title or description. To tweak the pull request, leave a comment on the pull request.
^{Join Our Discord}

This is an automated message generated by Sweep AI.

[github-actions]

👋 Thanks for opening this issue!

Get help or engage by:

• /help : to print help messages.
• /assignme : to assign this issue to you.