Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a non blocking file appender #673

Merged
merged 46 commits into from
Apr 14, 2020
Merged
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
945f4a8
Introduce a non blocking file appender
Apr 7, 2020
9ccaa5d
Use public interface for Rotation enum
zekisherif Apr 8, 2020
2a34863
simply if statement
zekisherif Apr 8, 2020
39f2709
Make WriterFactory pub(crate)
zekisherif Apr 8, 2020
f251bbe
Make BufWriterFactory pub(crate)
zekisherif Apr 8, 2020
de77617
Make InnerAppender pub(crate)
zekisherif Apr 8, 2020
7c9ce25
Make InnerAppender constructor pub(crate)
zekisherif Apr 8, 2020
e028dfa
make refresh_writer pub(crate)
zekisherif Apr 8, 2020
55bdffe
Make should_rollover pub(crate)
zekisherif Apr 8, 2020
1a70543
Make WorkerState pub(crate)
zekisherif Apr 8, 2020
5b4eacf
Make Worker pub(crate)
zekisherif Apr 8, 2020
18d0340
make constructor pub(crate)
zekisherif Apr 8, 2020
c4a8447
Apply suggested readability improvements and using pub(crate) where a…
zekisherif Apr 8, 2020
1c33af9
Use new Rotation const in match expressions. derive Copy trait for Ro…
Apr 8, 2020
6e8cb23
Remove copy trait from Rotation.
Apr 8, 2020
75ee728
Use Path for log_directory and filename prefix
zekisherif Apr 8, 2020
e5944c0
Properly handle use of Path in non public facing apis
Apr 8, 2020
cc4fde7
Use Arc for error_counter, modify fileappender constructor
Apr 9, 2020
d9968b2
Fix getters for FileAppender, clone writer for getter
Apr 9, 2020
a1f0d01
WIP: Decoupling file appender and nonblocking
Apr 10, 2020
9316656
Apply suggestions from code review
zekisherif Apr 10, 2020
ac14ca3
Update Rotation helpers and add constructor
Apr 10, 2020
b3aadef
Remove generic from InnerAppender, use BufWriter always
Apr 10, 2020
f77a6a8
Move creation of log file to Inner. Get rid of BufWriterFactory
Apr 10, 2020
4417342
Remove multiple impl of InnerAppender
Apr 10, 2020
64b356f
Impl MakeWriter on NonBlocking and remove NonBlockingWriter
Apr 10, 2020
98db6f7
Override write_all
zekisherif Apr 10, 2020
ec049e0
Use T:Write, get rid of RollingFileWriter
Apr 10, 2020
be7aeb4
cargo fmt
Apr 10, 2020
e87b631
Add lossy option for non_blocking
Apr 10, 2020
5d8475c
Add WorkerGuard and ensure closure of worker thread on drop
Apr 10, 2020
2983370
Apply suggestions from code review
zekisherif Apr 13, 2020
8e6a395
address comments
Apr 13, 2020
80bf8e2
Add comment about worker thread yielding
Apr 13, 2020
8f9bdc1
Fix clippy warnings
Apr 13, 2020
e467bdd
Fix 1 more clippy warning
Apr 13, 2020
4b73a1c
Add 2 examples
Apr 14, 2020
2b6cb51
Take suggestions for example doc improvements
Apr 14, 2020
9e7620d
fix clippy warning about needless main
Apr 14, 2020
b415265
Revert "fix clippy warning about needless main"
Apr 14, 2020
4ed792c
Add tracing dev dependency, fix clippy warning
Apr 14, 2020
93b26af
Apply suggestions from code review
zekisherif Apr 14, 2020
f494d53
Supress clippy warning using attribute
Apr 14, 2020
5216e6c
Fix fmt
Apr 14, 2020
97e5e15
Update tracing-appender/src/rolling.rs
zekisherif Apr 14, 2020
714e5bd
fix doc
Apr 14, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions tracing-appender/src/file_appender.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,36 @@ use chrono::Utc;
use crossbeam_channel::{bounded, Sender};
use std::path::Path;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use std::{io, thread};
use tracing_subscriber::fmt::MakeWriter;

#[allow(dead_code)]
zekisherif marked this conversation as resolved.
Show resolved Hide resolved
pub struct FileAppender {
log_writer: FileWriter,
worker_thread: thread::JoinHandle<()>,
error_counter: Arc<AtomicU64>,
}

impl FileAppender {
zekisherif marked this conversation as resolved.
Show resolved Hide resolved
fn new(
pub fn new(log_directory: &str, log_filename_prefix: &str) -> Self {
FileAppenderBuilder::default()
.build(log_directory, log_filename_prefix)
.expect("Failed to create FileAppender")
}

fn create_appender(
sender: Sender<Vec<u8>>,
worker: Worker<BufWriterFactory>,
error_counter: &'static AtomicU64,
error_counter: Arc<AtomicU64>,
) -> Self {
Self {
log_writer: FileWriter {
channel: sender,
error_counter,
error_counter: error_counter.clone(),
},
worker_thread: worker.worker_thread(),
error_counter,
}
}

Expand All @@ -36,6 +45,10 @@ impl FileAppender {
pub fn get_writer(self) -> FileWriter {
self.log_writer
}

pub fn get_error_counter(self) -> Arc<AtomicU64> {
self.error_counter.clone()
}
zekisherif marked this conversation as resolved.
Show resolved Hide resolved
}

pub struct FileAppenderBuilder {
Expand All @@ -53,7 +66,6 @@ impl FileAppenderBuilder {
self,
log_directory: impl AsRef<Path>,
log_filename_prefix: impl AsRef<Path>,
error_counter: &'static AtomicU64,
) -> io::Result<FileAppender> {
let (sender, receiver) = bounded(self.buffered_lines_limit);

Expand All @@ -65,7 +77,11 @@ impl FileAppenderBuilder {
BufWriterFactory {},
Utc::now(),
);
Ok(FileAppender::new(sender, worker?, error_counter))
Ok(FileAppender::create_appender(
sender,
worker?,
Arc::new(AtomicU64::new(0)),
))
}
}

Expand All @@ -80,7 +96,7 @@ impl Default for FileAppenderBuilder {

pub struct FileWriter {
channel: Sender<Vec<u8>>,
error_counter: &'static AtomicU64,
error_counter: Arc<AtomicU64>,
}

impl std::io::Write for FileWriter {
Expand Down