Add configurable logging system #407

enigbe · 2024-11-20T16:01:23Z

Overview

This PR introduces a flexible logging system for LDK Node by implementing a LogWriter interface that supports writing logs to different destinations.

What this PR does

Introduces a LogWriter interface, allowing Writer variants to handle log output destinations. The supported Writer variants can now:
- Write logs to the filesystem,
- Forward to a log implementer,
- Relay logs to a custom logger.
Exposes LogWriter to bindings.
Test logging to different destinations with:
- In-memory log logger,
- In-memory LogWriter logger.

Related Issue(s)

Expose logger interface #309
Cleaner implementation of closed draft feat: add customizable logging system #401

tnull

Cool, thank you for looking into this! This already looks pretty good, but I have some comments after the first round of review.

bindings/ldk_node.udl

...in/ldk-node-android/lib/src/androidTest/kotlin/org/lightningdevkit/ldknode/AndroidLibTest.kt

Cargo.toml

src/builder.rs

src/logger.rs

tnull · 2024-11-27T13:37:30Z

tests/common/mod.rs

 }

 /// Simple in-memory mock `log` logger for tests.
+#[derive(Debug)]


Why is this Debug?

NodeBuilder is Debug, and it fields an optional LogWriterConfig. The config object has to be Debug and custom log writer variant needs to implement it for this reason.

tests/common/mod.rs

tests/integration_tests_rust.rs

tnull · 2024-12-09T08:14:56Z

@enigbe Is there any update on this? Please let me know if you're hitting any blockers.

This seems to need a minor rebase by now.

enigbe · 2024-12-10T03:50:34Z

No blockers on this. I'll be pushing updates later today.

tnull · 2024-12-18T09:32:43Z

@enigbe Please let me know if/when this is ready for the next round of review!

enigbe

@tnull I believe this is ready for another review. I have addressed the majority of the concerns you raised in the first pass.

Regarding your concerns about testing the logging to custom loggers, I agree that the necessary refactor would be extensive and could detract from the purpose of this PR. As such, I plan to address these test-related changes in a follow-up PR.

Additionally, I encountered some challenges testing a custom logger in Kotlin and would greatly appreciate your guidance or suggestions.

tnull

Did another round of review, already looks pretty good.

I took a quick look at the Kotlin failures, but also couldn't immediately spot what's up, let me know if you want me to have a closer look though.

Btw, you could consider rebasing on #426 (or on main after it lands) which generally fixes pre-existing CI failures.

bindings/ldk_node.udl

src/config.rs

src/builder.rs

src/logger.rs

tnull · 2025-01-06T13:39:38Z

src/logger.rs

+
+impl LogWriter for Writer {
+	fn log(&self, record: LogRecord) {
+		let log = format!(


Hmm, so format! allocates a string on the heap. As we try to avoid allocations (to reduce heap fragmentation where possible), can we a) only create this string when we're sure we need it to log (i.e, after passing the log level filtering), and b) can we avoid it altogether for the CustomWriter case, and possibly even the LogFacadeWriter case? In the latter case we should be able to give the arguments to the respective macros directly, no?

src/logger.rs

...in/ldk-node-android/lib/src/androidTest/kotlin/org/lightningdevkit/ldknode/AndroidLibTest.kt

src/logger.rs

tnull · 2025-01-17T12:35:46Z

@enigbe Sorry, this needs a rebase now that #426 landed (which however also should fix the pre-existing CI failures, finally)., Let me know if this is ready for another round of review!

tnull · 2025-01-21T13:30:27Z

src/payment/unified_qr.rs

Feel free to drop this commit and rebase cleanly on main.

tnull

Cool, basically LGTM, just a few last-minute nits/and cleanups.

Let's address them quickly and land this soon.

bindings/kotlin/ldk-node-jvm/lib/src/test/kotlin/org/lightningdevkit/ldknode/LibraryTest.kt

tnull · 2025-01-27T11:41:11Z

src/builder.rs

-	/// Sets the log file path if the log file needs to live separate from the storage directory path.
-	pub fn set_log_file_path(&mut self, log_dir_path: String) -> &mut Self {
-		self.config.log_file_path = Some(log_dir_path);
+	/// Configures the [`Node`] instance to write logs to the filesystem with an optional


nit: Let's end the first paragraph after filesystem and give explaining the two arguments a short sentence of their own, something like:

Suggested change

/// Configures the [`Node`] instance to write logs to the filesystem with an optional

/// Configures the [`Node`] instance to write logs to the filesystem.

///

/// If set to `None`, `log_file_path` will default to [`DEFAULT_LOG_FILE_PATH`].

/// If set to `None`, `log_level` will default to [`DEFAULT_LOG_LEVEL`].

tnull · 2025-01-27T11:42:07Z

src/builder.rs

+		self
+	}
+
+	/// Configures the [`Node`] instance to write logs to the provided custom log writer.


nit: Let's link LogWriter here:

Suggested change

/// Configures the [`Node`] instance to write logs to the provided custom log writer.

/// Configures the [`Node`] instance to write logs to the provided custom [`LogWriter`].

tnull · 2025-01-27T11:42:19Z

src/builder.rs

-	/// Sets the log file path if logs need to live separate from the storage directory path.
-	pub fn set_log_file_path(&self, log_file_path: String) {
-		self.inner.write().unwrap().set_log_file_path(log_file_path);
+	/// Configures the [`Node`] instance to write logs to the filesystem with an optional


Adjust as above.

tnull · 2025-01-27T11:42:31Z

src/builder.rs

+		self.inner.write().unwrap().set_log_facade_logger(log_level);
+	}
+
+	/// Configures the [`Node`] instance to write logs to the provided custom log writer.


Adjust as above.

tnull · 2025-01-27T11:54:28Z

src/lib.rs

@@ -111,6 +111,8 @@ pub use event::Event;

 pub use io::utils::generate_entropy_mnemonic;

+pub use config::FilesystemLoggerConfig;


As mentioned above, no need to export this anymore.

tnull · 2025-01-27T11:55:31Z

src/logger.rs

 use std::fs;
 use std::io::Write;
 use std::path::Path;
+use std::sync::Arc;
+
+/// A unit of logging output with Metadata to enable filtering module_path,


nit: Lowercase metadata and tick all the variable names:

Suggested change

/// A unit of logging output with Metadata to enable filtering module_path,

/// A unit of logging output with metadata to enable filtering `module_path`,

tnull · 2025-01-27T11:55:58Z

src/logger.rs

+	pub line: u32,
+}
+
+#[cfg(feature = "uniffi")]


This needs docs (copied from above).

tnull · 2025-01-27T11:57:55Z

src/logger.rs

-	level: Level,
+/// Defines the behavior required for writing log records.
+///
+/// This version is used when the `uniffi` feature is enabled.


Let's still add the second paragraph from above here before going into the Uniffi-specific differences.

tnull · 2025-01-27T12:00:31Z

tests/integration_tests_rust.rs

@@ -216,6 +217,9 @@ fn start_stop_reinit() {
 	setup_builder!(builder, config);
 	builder.set_chain_source_esplora(esplora_url.clone(), Some(sync_config));

+	let log_file_path = format!("{}/{}", config.storage_dir_path, "ldk_node.log");


Let's drop this? Or any reason why this test would need a special case? If we want to test the interface works, we could consider adding a test for that which then however should confirm that only the correct types of messages are logged. But let's maybe add that as a follow-up.

* Add flexible log writer interface for multiple destinations * Implement filesystem writing capability via FilesystemLogger * Prefix LDK-based objects with 'Ldk' for consistency * Add configuration options for log file path and log level

- modify tests to forward logs to mock in-memory `log` logger - correct "Forwards" spelling error

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger. * Fix setting log's global logger twice. * Revert the renaming of LogLevel to LdkLevel.

This commit addresses a series of fixes, refactors, and documentation changes meant to optimize the new logging system to be better with: - Improved memory usage by eliminating unnecessary data allocations and improving log record handling. - Enhanced logging system with support for foreign implementations (as tested with Kotlin). - Improved documentation across board. - Cleaner refactors and simpler code that improve clarity. - Streamlined logging configuration. - Pre-existing bug fixes.

tnull

LGTM 🎉

Thank you very much!

amackillop · 2025-01-30T16:29:07Z

Sweet will try this out

enigbe mentioned this pull request Nov 20, 2024

feat: add customizable logging system #401

Closed

7 tasks

G8XSU requested a review from tnull November 21, 2024 18:37

tnull reviewed Nov 27, 2024

View reviewed changes

enigbe force-pushed the 2024-11-configurable-logging-system branch from 1d57cab to d8eb0e1 Compare December 11, 2024 12:18

G8XSU mentioned this pull request Dec 12, 2024

Ldk-Server Milestone 3 lightningdevkit/ldk-server#11

Open

9 tasks

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 7fa2928 to 283fe85 Compare December 16, 2024 00:32

enigbe force-pushed the 2024-11-configurable-logging-system branch 4 times, most recently from 0660dfa to 6766c60 Compare December 19, 2024 21:54

enigbe commented Dec 20, 2024

View reviewed changes

tnull reviewed Jan 6, 2025

View reviewed changes

tnull reviewed Jan 13, 2025

View reviewed changes

src/logger.rs Outdated Show resolved Hide resolved

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 322f97f to 3b71223 Compare January 21, 2025 10:01

tnull reviewed Jan 21, 2025

View reviewed changes

src/payment/unified_qr.rs Outdated

Copy link

Collaborator

tnull Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to drop this commit and rebase cleanly on main.

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from f06097b to c43fe47 Compare January 27, 2025 10:40

tnull reviewed Jan 27, 2025

View reviewed changes

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 241bfcc to 96522d9 Compare January 29, 2025 19:56

tnull mentioned this pull request Jan 30, 2025

Minor follow-ups to #407 #450

Merged

enigbe added 2 commits January 30, 2025 11:42

feat(logger): add & impl LogWriter for Writer

6d8c65e

feat(facade): forward logs to log facade

fe0f547

- modify tests to forward logs to mock in-memory `log` logger - correct "Forwards" spelling error

enigbe added 2 commits January 30, 2025 13:04

enigbe force-pushed the 2024-11-configurable-logging-system branch from 96522d9 to e509cf8 Compare January 30, 2025 12:24

tnull approved these changes Jan 30, 2025

View reviewed changes

tnull merged commit 83159d0 into lightningdevkit:main Jan 30, 2025
6 of 14 checks passed

tnull mentioned this pull request Jan 30, 2025

Expose logger interface #309

Closed

tnull mentioned this pull request Jan 31, 2025

Fix log-facade logging and clarify max level #454

Merged

This was referenced Feb 10, 2025

Test logging interface enigbe/ldk-node#4

Open

Test logging interface #463

Open

rustaceanrob mentioned this pull request Feb 14, 2025

Logging requires heap allocations rustaceanrob/kyoto#298

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configurable logging system #407

Add configurable logging system #407

enigbe commented Nov 20, 2024

tnull left a comment

tnull Nov 27, 2024

enigbe Dec 17, 2024 •

edited

Loading

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024

tnull commented Dec 18, 2024

enigbe left a comment

tnull left a comment

tnull Jan 6, 2025

tnull commented Jan 17, 2025

tnull Jan 21, 2025

tnull left a comment

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull Jan 27, 2025

tnull left a comment •

edited

Loading

amackillop commented Jan 30, 2025

-	/// Configures the [`Node`] instance to write logs to the filesystem with an optional
+	/// Configures the [`Node`] instance to write logs to the filesystem.
+	///
+	/// If set to `None`, `log_file_path` will default to [`DEFAULT_LOG_FILE_PATH`].
+	/// If set to `None`, `log_level` will default to [`DEFAULT_LOG_LEVEL`].

	/// Configures the [`Node`] instance to write logs to the provided custom log writer.
	/// Configures the [`Node`] instance to write logs to the provided custom [`LogWriter`].

		@@ -111,6 +111,8 @@ pub use event::Event;

		pub use io::utils::generate_entropy_mnemonic;

		pub use config::FilesystemLoggerConfig;

	/// A unit of logging output with Metadata to enable filtering module_path,
	/// A unit of logging output with metadata to enable filtering `module_path`,

Add configurable logging system #407

Add configurable logging system #407

Conversation

enigbe commented Nov 20, 2024

Overview

What this PR does

Related Issue(s)

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enigbe Dec 17, 2024 • edited Loading

Choose a reason for hiding this comment

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024

tnull commented Dec 18, 2024

enigbe left a comment

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnull commented Jan 17, 2025

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnull left a comment • edited Loading

Choose a reason for hiding this comment

amackillop commented Jan 30, 2025

enigbe Dec 17, 2024 •

edited

Loading

tnull left a comment •

edited

Loading