Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logging: persisting logs on user's file system #239

Open
lrlna opened this issue Feb 5, 2021 · 0 comments
Open

logging: persisting logs on user's file system #239

lrlna opened this issue Feb 5, 2021 · 0 comments
Labels
feature 🎉 new commands, flags, functionality, and improved error messages needs decision 🤝

Comments

@lrlna
Copy link
Member

lrlna commented Feb 5, 2021

Description

This issue is to provide Rover with a persisted log file collecting all logs emitted by Rover. Log file should be machine readable, and with appropriate file locks/unlocks, be written from multiple processes.

Requirements

  1. Log file should be stored in their own directory along side Rover's config. (Should we consider system-wide log storage?)
  2. Logs should be able to be written from multiple processes.
  3. We should write all levels of logs.
  4. Logs should be machine-readable - either the structured logs that tracing crates provides, or ndjson. This will depend on what we decide to do in logging: replace tracing crate with femme #223. Essentially, let's have logs store raw event data.
  5. Logs should not contain sensitive information, i.e. API keys. (This point can be discussed, if anyone feels strongly that logs should also have API keys)

Decisions to make

  1. We should decide whether local user logging is on by default.
  2. We should decide how often to create a new file. As in, do we write to a new file every 24hrs or every week? I am keen on every 24hrs, so the files can be named by timestamp.
  3. We should think about doing a clean up of log files. This can perhaps be done as part of another issue.

Writing to file

To make sure that multiple processes can write to the same file without a race condition, I propose we use advisory locks. There is a very good JavaScript library that was ported to Rust, that I think fits our bill. It essentially allows for a single processes to write to a file descriptor (fd) by locking other access. If another process wants access, the file system will put its claim on hold and provide a backpressure mechanism for all other incoming data from that process until the fd is unlocked.

Essentially, once we finish #220, we will have our logging in one place. From there, we can write all information with:

let mut log_file = File::create("20210502.log")?
let mut f = FdLock::new(log_file);
f.lock()?.write_all(b"log message")?;

Advisory locks, an illustration

persisting user logs with advisory locks

@abernix abernix added feature 🎉 new commands, flags, functionality, and improved error messages needs decision 🤝 labels Feb 12, 2021
@lrlna lrlna added this to the May 25 milestone Apr 8, 2021
@abernix abernix modified the milestones: May 25, June 8 May 19, 2021
@EverlastingBugstopper EverlastingBugstopper modified the milestones: June 8, June 22 Jun 8, 2021
@abernix abernix modified the milestones: June 22, July 6, 2021, July 6, Qx Jun 9, 2021
@EverlastingBugstopper EverlastingBugstopper removed this from the Qx milestone Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature 🎉 new commands, flags, functionality, and improved error messages needs decision 🤝
Projects
None yet
Development

No branches or pull requests

3 participants