Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast logger migration #274

Merged
merged 65 commits into from
Mar 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
53bcb66
Add rotate log config option
RiugaBachi Mar 29, 2023
d2e30d6
Add monad-logger and fast-logger deps
RiugaBachi Mar 29, 2023
38897c4
Remove (unused)
RiugaBachi Mar 29, 2023
194015e
Update `RotatingLog` to wrap a FastLogger and its release callback
RiugaBachi Mar 29, 2023
1a6e828
Add defaultRotationSpec
RiugaBachi Mar 29, 2023
f6ca7cc
Update imports/exports
RiugaBachi Mar 29, 2023
7e3ed8c
Refactor withLogger to use a FastLogger to rotating files or stderr
RiugaBachi Mar 29, 2023
db54ccc
Update withRotatingLogger to use a FastLogger to a rotating file
RiugaBachi Mar 29, 2023
6805a6e
Rename for clarity
RiugaBachi Mar 29, 2023
2f7c736
Change addHunk to fast logging function
RiugaBachi Mar 29, 2023
7cfb8fc
Change rotating log close fn
RiugaBachi Mar 29, 2023
1e5c35d
Add yaml parsing for rotate-logs
RiugaBachi Mar 29, 2023
29a673f
Update changelog
RiugaBachi Mar 29, 2023
ab6cf90
Fix typo
RiugaBachi Mar 29, 2023
8306f4c
Added log name change examples for apps
RiugaBachi Mar 29, 2023
2c2dbd0
Implement KeterM monad
RiugaBachi Mar 30, 2023
954411b
Add unliftio-core dependency
RiugaBachi Mar 30, 2023
397aed8
Expose Keter.Context
RiugaBachi Mar 30, 2023
c0e9dc8
Refactor Keter.Main to use KeterM
RiugaBachi Mar 30, 2023
1725181
Refactor Keter.App to use KeterM
RiugaBachi Mar 30, 2023
801e0e4
Refactor Keter.AppManager to use KeterM
RiugaBachi Mar 30, 2023
839700f
Refactor Keter.Cli to use KeterM
RiugaBachi Mar 30, 2023
8fc1dd1
Remove `ascLog` field (unused)
RiugaBachi Mar 30, 2023
c0a668f
Remove again
RiugaBachi Mar 30, 2023
b51b51d
Refactor Keter.HostManager to use KeterM
RiugaBachi Mar 30, 2023
7b9ec22
Refactor Keter.PortPool to use KeterM
RiugaBachi Mar 30, 2023
7833c71
Refactor Keter.Proxy to use KeterM (where possible)
RiugaBachi Mar 30, 2023
8b5e0e3
Update monitorProcess logger callback to include log level
RiugaBachi Mar 30, 2023
147e1ab
Remove old log message ADT
RiugaBachi Mar 30, 2023
10c6762
Deprecate old exception logger for now
RiugaBachi Mar 30, 2023
ed22d6c
Stub out parts of test suite that broke for now
RiugaBachi Mar 30, 2023
8f04804
Fix breaking changes in test suite
RiugaBachi Mar 31, 2023
3dc60b2
Update test suite build dependencies
RiugaBachi Mar 31, 2023
5eb0eff
Bump to version 2.1.0
RiugaBachi Mar 31, 2023
dbcf03d
Update changelog
RiugaBachi Mar 31, 2023
85f1c88
Minor reformat
RiugaBachi Mar 31, 2023
fb78544
Create logging directory if missing
RiugaBachi Mar 31, 2023
f6583f1
Add to config template
RiugaBachi Mar 31, 2023
e464cfb
Lowercase
RiugaBachi Mar 31, 2023
fad5525
Rename RotatingLog to Logger
RiugaBachi Mar 31, 2023
51785cf
Move logger creation to `createLoggerViaConfig`
RiugaBachi Mar 31, 2023
60cf7cc
Documentation
RiugaBachi Mar 31, 2023
7b9b79b
Use createLoggerViaConfig
RiugaBachi Mar 31, 2023
cb7d5a6
Imports
RiugaBachi Mar 31, 2023
7887b89
Rename rotating log references
RiugaBachi Mar 31, 2023
921cea2
Use `createLoggerViaConfig`
RiugaBachi Mar 31, 2023
bb6bfd3
Add blog post link to `KeterM` doc
RiugaBachi Mar 31, 2023
8ed4fe5
Add 'keter|' tag to internal log format
RiugaBachi Mar 31, 2023
07337ed
Remove `LogLevel` from process monitor log fn
RiugaBachi Mar 31, 2023
d586327
Refactor app & process-monitor log format
RiugaBachi Mar 31, 2023
dd1b71c
Imports
RiugaBachi Mar 31, 2023
02c0623
Unquote app name
RiugaBachi Mar 31, 2023
15a6cfd
Fix keter tag
RiugaBachi Mar 31, 2023
95cef93
Remove `logEx`
RiugaBachi Mar 31, 2023
5b47603
Remove todo
RiugaBachi Mar 31, 2023
a6bb6a4
Reintroduce commented out log calls
RiugaBachi Mar 31, 2023
6847ba4
Add upper dependency bounds
RiugaBachi Mar 31, 2023
3255e86
Rename LogFile to Log
RiugaBachi Mar 31, 2023
8a18651
Move Keter.Conduit.LogFile to Keter.Logger
RiugaBachi Mar 31, 2023
40fcca6
Update exposed modules
RiugaBachi Mar 31, 2023
fd4905c
Rename imports
RiugaBachi Mar 31, 2023
cd9e110
Store LogType in Logger
RiugaBachi Mar 31, 2023
aa22c8d
Update format comment
RiugaBachi Mar 31, 2023
57f5c0d
Refactor log formatter to emit tags only when logging to stderr
RiugaBachi Mar 31, 2023
d58364f
Refactor to use formatTag
RiugaBachi Mar 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions ChangeLog.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,35 @@
## 2.1

+ Log naming and directory scheme has changed for both main keter logs and app logs.
Old logs were named `dir/current.log` for the current log and `%Y%m%d_%H%M%S.log`
(`time` package conventions) for rotated logs.
Current logs have been brought up one level and named after their old directory:
`logs/keter/current.log` -> `logs/keter.log`
Rotated logs will now simply have `.1` `.2` ascending appended to the name of the base logs
rather than be named after the date and time they were rotated at:
`logs/keter/20230413_231415.log` -> `logs/keter.log.1`
`logs/__builtin__/20230413_231415.log` -> `logs/__builtin__.log.1`
`logs/app-foo/20230413_231415.log` -> `logs/app-foo.log.1`
Please update anything that depended on the old log naming and directory conventions accordingly.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this work with several apps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed a typo here actually 🙏 Added some examples for app log changes just now.

+ Added the `rotate-logs` option (default: true) in the keter config file.
When true, the main keter (non-app!) logs will rotate like they have in previous versions.
When false, the main keter logs will emit straight to stderr; this is useful e.g. if you're
running keter via systemd, which captures stderr output for you.
+ Internal logging implementation has been switched over to `fast-logger` instead of the
old in-house logging solution.
Please be aware that compared to the old logging solution, the usage of `fast-logger` does
not 100% guarantee consistent time ordering. If anything previously depended on tailing the last
line of a log and critically assumed that messages will be in order, it should now parse via
timestamp instead.
+ The `LogMessage` ADT has been removed.
+ Replaced individual logging calls with TH splice -style calls where sensible to have access to source location info.
+ Updated log message format to make use of the additional info:
`"$time|$module$:$line_num|$log_level> $msg"`
+ Added `Keter.Context`, exposing the new `KeterM` monad and related functions.
This monad carries a mappable global config and logger around, eliminating the need to pass various configuration data and the logger to everything.
+ Refactored most `Keter.*` module functions to be actions in `KeterM`
Please anticipate breaking changes for anything written against the exposed API.

## 2.0.1

+ Force usage of http-reverse-proxy versions above 0.6.0.1.
Expand Down
3 changes: 3 additions & 0 deletions etc/keter-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
# the config file directory.
root: ..

# Whether or not to rotate logs, otherwise log straight to stderr
rotate-logs: false

# Keter can listen on multiple ports for incoming connections. These ports can
# have HTTPS either enabled or disabled.
listeners:
Expand Down
10 changes: 8 additions & 2 deletions keter.cabal
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Cabal-version: >=1.10
Name: keter
Version: 2.0.1
Version: 2.1.0
Synopsis: Web application deployment manager, focusing on Haskell web frameworks
Description:
Deployment system for web applications, originally intended for hosting Yesod
Expand Down Expand Up @@ -62,6 +62,9 @@ Library
, attoparsec >= 0.10
, http-client >= 0.5.0
, http-conduit >= 2.1
, fast-logger < 3.3
, monad-logger < 0.4
, unliftio-core < 0.3
, case-insensitive
, array
, mtl
Expand Down Expand Up @@ -95,14 +98,15 @@ Library
Keter.AppManager
Keter.LabelMap
Keter.Cli
Keter.Context
Keter.Main
Keter.PortPool
Keter.Proxy
Keter.HostManager
Keter.Rewrite
Keter.Yaml.FilePath
Keter.TempTarball
Keter.Conduit.LogFile
Keter.Logger
Keter.Conduit.Process.Unix
Other-Modules:
Keter.Aeson.KeyHelper
Expand All @@ -124,6 +128,8 @@ test-suite test
type: exitcode-stdio-1.0
build-depends: base
, transformers
, mtl
, monad-logger
, conduit
, bytestring
, unix
Expand Down
Loading