Release v0.6.0

New features

New GUI

Logsuck now uses Mantine instead of a homegrown component library, hopefully making it look better.

Support for structured logging in JSON (#7)

Logsuck can now parse structured logs which use JSON. Key-value pairs in the JSON object are automatically extracted as fields. You can configure this by setting parser.type to JSON on a fileType, or by using the configuration GUI. An example fileType configuration may look like:

{
  "name": "json-log",
  "timeLayout": "UNIX_DECIMAL_NANOS",
  "parser": {
    "type": "JSON",
    "jsonConfig": {
      "eventDelimiter": "\n",
      "timeField": "ts"
    }
  }
},

Support for UNIX timestamps

Logsuck now has support for three types of UNIX timestamps. You can access this functionality by setting timeLayout to one of the following:

• UNIX if your timestamps are in seconds from the Unix epoch
• UNIX_MILLIS if your timestamps are in milliseconds from the Unix epoch
• UNIX_DECIMAL_NANOS if your timestamps are in this format: <UNIX>.<NANOS> where <UNIX> is the number of seconds since the Unix epoch and <NANOS> are the number of nanoseconds elapsed in that second. This is the format used by zap by default.

Release v0.5.0

New features

Dynamic configuration

You can now change the Logsuck configuration from the GUI. The configuration is initially set based on command line / JSON config parameters, but after the first run of Logsuck the configuration will be saved in the database and configured through the GUI. If running in forwarder/recipient mode, the recipient will push configuration out to the forwarders so you only need to configure the recipient.

It is possible to bypass dynamic configuration and only use the JSON configuration file by setting "forceStaticConfig": true in the JSON config file. This must be done before the first run of Logsuck when the database is created and must remain in the JSON configuration file forever. There is more information in docs/Configuration.md

| surrounding pipeline step / "View context" (#24)

There is now a "View context" button on each event which will use a new pipeline step called "surrounding" which retrieves events that were close to the event in the log file, and sorts them in the order they appeared in the file. This can be useful if your search is getting result from multiple files and you are interested in seeing what happened right after a specific event in the system that logged the event. By default the 100 closest events are retrieved, this can be changed using by adding a count argument to the pipe, such as | surrounding count=200 eventId=31283.

This change required a breaking change in the database schema. logsuck.db files created before v0.5.0 will not work with v0.5.0.

Add task for deleting old events (#17)

There is now a task which deletes events based on their age. You can configure it under the "tasks" key in the configuration GUI or in the JSON configuration.

As an example you can set the configuration to

  "tasks": {
    "tasks": [
      {
        "name": "@logsuck/DeleteOldEventsTask",
        "enabled": true,
        "interval": "5m0s",
        "config": [{ "key": "minAge", "value": "4d" }]
      }
    ]
  }

To run a job every 5 minutes which deletes any events which are older than 4 days.

Bug fixes

• Fixed an issue where Logsuck would complain about tables not existing when using in-memory database (d776fba)
• Fix Logsuck not reading files which were created after Logsuck was started (#22) (40805f3)

Release v0.4.0

New features

Absolute timestamp filters (#18)

You can now filter events by absolute time spans instead of just relative ("last xx minutes") spans.

Link to a specific search result page (#19)

When you use the pagination on the search result page, the current page will be saved into the query parameters so you can send a link to the specific page of results.

Bug fixes

• #14 Logsuck will no longer log an error if you try to search for events in an empty database

Other

• Logsuck now uses the embed package from Go 1.16 instead of using vfsgen to bundle the frontend resources into the executable. This means that anyone who wants to build Logsuck from source needs to upgrade to Go 1.16.

Release v0.3.0

New features

| where command (#16)

The | where command can be used to filter events by fields which were extracted by steps in the search pipeline that aren't the initial search. Currently that just means | rex but it will probably have more uses in the future.

Here is an example search which uses | where to filter the search to only find events where the "userId" field, which was extracted by | rex, is equal to 123:

userId | rex "userId (?P<userId>\d+)" | where userId=123

Bug fixes

• #20 Starting a search from the search page now updates the query parameters

Other

• #21 A benchmarking tool called SUCKHAMMER5000 was added. SUCKHAMMER5000 benchmarks how quickly Logsuck in single mode can read logs from a file and put them into the database. Using this tool I made two performance improvements, the more noticeable one being "True batch" mode added in 82b069b which increases throughput by up to 3x.

Release v0.2.0

New features

| rex command (#13)

You can now use | rex in your searches to extract fields from events on the fly. The work on this issue also refactored how searches are parsed and executed which should make adding new commands a lot easier.

Dedicated search page (#11)

The search results portion of the GUI is now served on a separate URL from the "home" page which contains the list of recent searches. This also makes searches linkable, whereas previously the URL would always be "/" even after performing a search.

Bug fixes

• #4 Clicking a recent search on the home page now updates the time picker
• #15 The most recently added event will no longer be ignored by searches

Other

• net/http has been replaced with Gin. This should make the web backend parts of Logsuck more maintainable in the long run.

Known issues

• #20 Starting a new search does not update the query parameters if you already are on the search page. This is due to a complete brain fart.

Release v0.1.1

New features

Glob filenames (#8)

You can now use globs in file names configured via JSON. For example:

{
  "files": [
    {
      "fileName": "./log-access.txt",
      "timeLayout": "02/Jan/2006:15:04:05 -0700"
    },
    {
      "fileName": "./log-*.txt"
    }
  ]
}

If a file name is matched multiple times, the first configuration wins. So in the above example, log-access.txt will use the "02/Jan/2006:15:04:05 -0700" timelayout instead of the default timelayout.

Bug fixes

• #2 Starting a search while another is ongoing should no longer result in the results of the first search being shown
• #3 "No results found" will not be shown until a search has actually completed
• #6 The pagination buttons no longer become unclickable after the popover has been opened and closed
• #10 Fragments are now completely case insensitive

Other

• Searching should be faster in general due to a smarter SQL query being used, SQLite WAL being enabled and the max concurrent SQLite connections has been set to infinite.
• The release GitHub Action has been updated to not use ::set-env which is unsafe. That's why this release ended up being v0.1.1 instead of v0.1.0.

Release v0.0.0

This is a test of the release action! It seems to be working!