Skip to content

Commit

Permalink
Merge pull request #73 from onkel-dirtus/cleanups
Browse files Browse the repository at this point in the history
Various Cleanups, bumps to version 0.0.12
  • Loading branch information
fireproofsocks authored Jul 19, 2021
2 parents aa74c9f + 9e93447 commit 0f37982
Show file tree
Hide file tree
Showing 7 changed files with 188 additions and 145 deletions.
4 changes: 4 additions & 0 deletions .formatter.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
]
30 changes: 13 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
LoggerFileBackend
=================
# LoggerFileBackend

A simple `Logger` backend which writes logs to a file. It does not handle log
rotation for you, but it does tolerate log file renames, so it can be
used in conjunction with external log rotation.
A simple Elixir `Logger` backend which writes logs to a file. It does not handle log rotation, but it does tolerate log file renames, so it can be used in conjunction with external log rotation.

**Note** The following of file renames does not work on Windows, because `File.Stat.inode` is used to determine whether the log file has been (re)moved and, on non-Unix, `File.Stat.inode` is always 0.
**Note** The renaming of log files does not work on Windows, because `File.Stat.inode` is used to determine whether the log file has been (re)moved and, on non-Unix, `File.Stat.inode` is always 0.

**Note** If you are running this with the Phoenix framework, please review the Phoenix specific instructions later on in this file.

Expand All @@ -14,17 +11,17 @@ used in conjunction with external log rotation.
`LoggerFileBackend` is a custom backend for the elixir `:logger` application. As
such, it relies on the `:logger` application to start the relevant processes.
However, unlike the default `:console` backend, we may want to configure
multiple log files, each with different log levels formats, etc. Also, we want
multiple log files, each with different log levels, formats, etc. Also, we want
`:logger` to be responsible for starting and stopping each of our logging
processes for us. Because of these considerations, there must be one `:logger`
backend configured for each log file we need. Each backend has a name like
`{LoggerFileBackend, id}`, where `id` is any elixir term (usually an atom).

For example, let's say we want to log error messages to
"/var/log/my_app/error.log". To do that, we will need to configure a backend.
`"/var/log/my_app/error.log"`. To do that, we will need to configure a backend.
Let's call it `{LoggerFileBackend, :error_log}`.

Our config.exs would have an entry similar to this:
Our `config.exs` would have an entry similar to this:

```elixir
# tell logger to load a LoggerFileBackend processes
Expand Down Expand Up @@ -53,12 +50,11 @@ multiple log files.

`LoggerFileBackend` supports the following configuration values:

* path - the path to the log file
* level - the logging level for the backend
* format - the logging format for the backend
* metadata - the metadata to include
* metadata_filter - metadata terms which must be present in order to log

* `path` - the path to the log file
* `level` - the logging level for the backend
* `format` - the logging format for the backend
* `metadata` - the metadata to include
* `metadata_filter` - metadata terms which must be present in order to log

### Examples

Expand Down Expand Up @@ -88,6 +84,7 @@ config :logger, :error,
path: "/path/to/error.log",
level: :error
```

#### Filtering specific metadata terms

This example only logs `:info` statements originating from the `:ui` OTP app; the `:application` metadata key is auto-populated by `Logger`.
Expand Down Expand Up @@ -127,12 +124,11 @@ Logger.metadata(device: 1)
Logger.info("statement") # <= already tagged with the device_1 metadata
```


## Additional Phoenix Configurations

Phoenix makes use of its own `mix.exs` file to track dependencies and additional applications. Add the following to your `mix.exs`:

```
```elixir
def application do
[applications: [
...,
Expand Down
2 changes: 1 addition & 1 deletion config/config.exs
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ use Mix.Config
# here (which is why it is important to import them last).
#

import_config "#{Mix.env}.exs"
import_config "#{Mix.env()}.exs"
140 changes: 86 additions & 54 deletions lib/logger_file_backend.ex
Original file line number Diff line number Diff line change
@@ -1,40 +1,40 @@
defmodule LoggerFileBackend do
@moduledoc """
Custom `:logger` backend.
`LoggerFileBackend` is a custom backend for the elixir `:logger` application.
"""

@behaviour :gen_event


@type path :: String.t
@type file :: :file.io_device
@type inode :: integer
@type format :: String.t
@type level :: Logger.level
@type metadata :: [atom]
@type path :: String.t()
@type file :: :file.io_device()
@type inode :: integer
@type format :: String.t()
@type level :: Logger.level()
@type metadata :: [atom]

require Record
Record.defrecordp :file_info, Record.extract(:file_info, from_lib: "kernel/include/file.hrl")
Record.defrecordp(:file_info, Record.extract(:file_info, from_lib: "kernel/include/file.hrl"))

@default_format "$time $metadata[$level] $message\n"

def init({__MODULE__, name}) do
{:ok, configure(name, [])}
end


def handle_call({:configure, opts}, %{name: name} = state) do
{:ok, :ok, configure(name, opts, state)}
end


def handle_call(:path, %{path: path} = state) do
{:ok, {:ok, path}, state}
end


def handle_event({level, _gl, {Logger, msg, ts, md}}, %{level: min_level, metadata_filter: metadata_filter} = state) do
if (is_nil(min_level) or Logger.compare_levels(level, min_level) != :lt) and metadata_matches?(md, metadata_filter) do
def handle_event(
{level, _gl, {Logger, msg, ts, md}},
%{level: min_level, metadata_filter: metadata_filter} = state
) do
if (is_nil(min_level) or Logger.compare_levels(level, min_level) != :lt) and
metadata_matches?(md, metadata_filter) do
log_event(level, msg, ts, md, state)
else
{:ok, state}
Expand All @@ -56,18 +56,28 @@ defmodule LoggerFileBackend do
{:ok, state}
end

defp log_event(level, msg, ts, md, %{path: path, io_device: nil} = state) when is_binary(path) do
defp log_event(level, msg, ts, md, %{path: path, io_device: nil} = state)
when is_binary(path) do
case open_log(path) do
{:ok, io_device, inode} ->
log_event(level, msg, ts, md, %{state | io_device: io_device, inode: inode})

_other ->
{:ok, state}
end
end

defp log_event(level, msg, ts, md, %{path: path, io_device: io_device, inode: inode, rotate: rotate} = state) when is_binary(path) do
defp log_event(
level,
msg,
ts,
md,
%{path: path, io_device: io_device, inode: inode, rotate: rotate} = state
)
when is_binary(path) do
if !is_nil(inode) and inode == get_inode(path) and rotate(path, rotate) do
output = format_event(level, msg, ts, md, state)

try do
IO.write(io_device, output)
{:ok, state}
Expand All @@ -77,6 +87,7 @@ defmodule LoggerFileBackend do
{:ok, io_device, inode} ->
IO.write(io_device, prune(output))
{:ok, %{state | io_device: io_device, inode: inode}}

_other ->
{:ok, %{state | io_device: nil, inode: nil}}
end
Expand All @@ -88,87 +99,97 @@ defmodule LoggerFileBackend do
end

defp rename_file(path, keep) do

File.rm("#{path}.#{keep}")

Enum.each(keep-1..1, fn(x) -> File.rename("#{path}.#{x}", "#{path}.#{x+1}") end)
Enum.each((keep - 1)..1, fn x -> File.rename("#{path}.#{x}", "#{path}.#{x + 1}") end)

case File.rename(path, "#{path}.1") do
:ok -> false
_ -> true
_ -> true
end

end

defp rotate(path, %{max_bytes: max_bytes, keep: keep }) when is_integer(max_bytes) and is_integer(keep) and keep > 0 do

defp rotate(path, %{max_bytes: max_bytes, keep: keep})
when is_integer(max_bytes) and is_integer(keep) and keep > 0 do
case :file.read_file_info(path, [:raw]) do
{:ok, file_info(size: size)} ->
if size >= max_bytes, do: rename_file(path, keep) , else: true
if size >= max_bytes, do: rename_file(path, keep), else: true

_ ->
true
end

end

defp rotate(_path, nil), do: true



defp open_log(path) do
case (path |> Path.dirname |> File.mkdir_p) do
case path |> Path.dirname() |> File.mkdir_p() do
:ok ->
case File.open(path, [:append, :utf8]) do
{:ok, io_device} -> {:ok, io_device, get_inode(path)}
other -> other
end
other -> other

other ->
other
end
end


defp format_event(level, msg, ts, md, %{format: format, metadata: keys}) do
Logger.Formatter.format(format, level, msg, ts, take_metadata(md, keys))
end

@doc false
@spec metadata_matches?(Keyword.t, nil|Keyword.t) :: true|false
@spec metadata_matches?(Keyword.t(), nil | Keyword.t()) :: true | false
def metadata_matches?(_md, nil), do: true
def metadata_matches?(_md, []), do: true # all of the filter keys are present
def metadata_matches?(md, [{key, val}|rest]) do
# all of the filter keys are present
def metadata_matches?(_md, []), do: true

def metadata_matches?(md, [{key, val} | rest]) do
case Keyword.fetch(md, key) do
{:ok, ^val} ->
metadata_matches?(md, rest)
_ -> false #fail on first mismatch

# fail on first mismatch
_ ->
false
end
end



defp take_metadata(metadata, :all), do: metadata

defp take_metadata(metadata, keys) do
metadatas = Enum.reduce(keys, [], fn key, acc ->
case Keyword.fetch(metadata, key) do
{:ok, val} -> [{key, val} | acc]
:error -> acc
end
end)
metadatas =
Enum.reduce(keys, [], fn key, acc ->
case Keyword.fetch(metadata, key) do
{:ok, val} -> [{key, val} | acc]
:error -> acc
end
end)

Enum.reverse(metadatas)
end


defp get_inode(path) do
case :file.read_file_info(path, [:raw]) do
{:ok, file_info(inode: inode)} -> inode
{:error, _} -> nil
end
end


defp configure(name, opts) do
state = %{name: nil, path: nil, io_device: nil, inode: nil, format: nil, level: nil, metadata: nil, metadata_filter: nil, rotate: nil}
state = %{
name: nil,
path: nil,
io_device: nil,
inode: nil,
format: nil,
level: nil,
metadata: nil,
metadata_filter: nil,
rotate: nil
}

configure(name, opts, state)
end

Expand All @@ -177,30 +198,41 @@ defmodule LoggerFileBackend do
opts = Keyword.merge(env, opts)
Application.put_env(:logger, name, opts)

level = Keyword.get(opts, :level)
metadata = Keyword.get(opts, :metadata, [])
format_opts = Keyword.get(opts, :format, @default_format)
format = Logger.Formatter.compile(format_opts)
path = Keyword.get(opts, :path)
level = Keyword.get(opts, :level)
metadata = Keyword.get(opts, :metadata, [])
format_opts = Keyword.get(opts, :format, @default_format)
format = Logger.Formatter.compile(format_opts)
path = Keyword.get(opts, :path)
metadata_filter = Keyword.get(opts, :metadata_filter)
rotate = Keyword.get(opts, :rotate)
rotate = Keyword.get(opts, :rotate)

%{state | name: name, path: path, format: format, level: level, metadata: metadata, metadata_filter: metadata_filter, rotate: rotate}
%{
state
| name: name,
path: path,
format: format,
level: level,
metadata: metadata,
metadata_filter: metadata_filter,
rotate: rotate
}
end

@replacement "�"

@spec prune(IO.chardata) :: IO.chardata
@spec prune(IO.chardata()) :: IO.chardata()
def prune(binary) when is_binary(binary), do: prune_binary(binary, "")
def prune([h|t]) when h in 0..1_114_111, do: [h|prune(t)]
def prune([h|t]), do: [prune(h)|prune(t)]
def prune([h | t]) when h in 0..1_114_111, do: [h | prune(t)]
def prune([h | t]), do: [prune(h) | prune(t)]
def prune([]), do: []
def prune(_), do: @replacement

defp prune_binary(<<h::utf8, t::binary>>, acc),
do: prune_binary(t, <<acc::binary, h::utf8>>)

defp prune_binary(<<_, t::binary>>, acc),
do: prune_binary(t, <<acc::binary, @replacement>>)

defp prune_binary(<<>>, acc),
do: acc
end
25 changes: 14 additions & 11 deletions mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@ defmodule LoggerFileBackend.Mixfile do
use Mix.Project

def project do
[app: :logger_file_backend,
version: "0.0.11",
elixir: "~> 1.0",
description: description(),
package: package(),
deps: deps()]
[
app: :logger_file_backend,
version: "0.0.12",
elixir: "~> 1.0",
description: description(),
package: package(),
deps: deps()
]
end

def application do
Expand All @@ -19,13 +21,14 @@ defmodule LoggerFileBackend.Mixfile do
end

defp package do
[maintainers: ["Kurt Williams"],
licenses: ["MIT"],
links: %{"GitHub" => "https://github.com/onkel-dirtus/logger_file_backend"}]
[
maintainers: ["Kurt Williams", "Everett Griffiths"],
licenses: ["MIT"],
links: %{"GitHub" => "https://github.com/onkel-dirtus/logger_file_backend"}
]
end

defp deps do
[{:credo, "~> 1.0", only: [:dev, :test]},
{:ex_doc, "~> 0.24", only: :dev}]
[{:credo, "~> 1.0", only: [:dev, :test]}, {:ex_doc, "~> 0.24", only: :dev}]
end
end
Loading

0 comments on commit 0f37982

Please sign in to comment.