Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/v5.5.6: Merge next-patch to main #1145

Merged
merged 22 commits into from
Oct 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
33e63e6
adding in license type table
kris-watts-gravwell Sep 13, 2024
1ba825e
tweak wording a bit for unlicensed and comunity edition
kris-watts-gravwell Sep 13, 2024
be6d1cb
reworking wording
kris-watts-gravwell Sep 16, 2024
b925ce3
Merge remote-tracking branch 'upstream/dev' into dev
kris-watts-gravwell Sep 27, 2024
d9bb459
add note about hash used for resources
kris-watts-gravwell Sep 27, 2024
187d7d0
Update resources.md
ashnwade Sep 27, 2024
ea66143
Merge pull request #1125 from kris-watts-gravwell/resourceHashNote
ashnwade Sep 27, 2024
f6ee6e7
Merge remote-tracking branch 'upstream/dev' into dev
kris-watts-gravwell Sep 27, 2024
74f1ad4
add new CE advanced tier and fix ingest number on CE
kris-watts-gravwell Sep 27, 2024
7d96069
tweak wording
kris-watts-gravwell Sep 27, 2024
ef1f49f
remove AI feature, did not make release
kris-watts-gravwell Sep 30, 2024
03e2725
tweak some wording on ageout and add another warning
kris-watts-gravwell Oct 1, 2024
c9376df
make PR cleaner
kris-watts-gravwell Oct 1, 2024
1ba0052
Merge next-patch into merge/main_to_next-patch
github-actions[bot] Oct 2, 2024
cae2863
Merge pull request #1134 from gravwell/merge/main_to_next-patch
ashnwade Oct 2, 2024
b7f32cd
Update configuration/ageout.md
john-floren-gravwell Oct 2, 2024
d9923aa
Update configuration/ageout.md
john-floren-gravwell Oct 2, 2024
000e297
Merge pull request #1130 from kris-watts-gravwell/ageoutWording
ashnwade Oct 2, 2024
17744ac
changelog
ashnwade Oct 7, 2024
99a155c
Commit suggestion
ashnwade Oct 7, 2024
71645d4
review feedback
ashnwade Oct 7, 2024
adeb23d
Merge pull request #1143 from ashnwade/changelog
ashnwade Oct 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions _static/versions.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
[
{
"name": "v5.5.5 (latest)",
"version": "v5.5.5",
"name": "v5.5.6 (latest)",
"version": "v5.5.6",
"url": "https://docs.gravwell.io/",
"preferred": true
},
{
"version": "v5.5.5",
"url": "https://docs.gravwell.io/v5.5.5/"
},
{
"version": "v5.5.4",
"url": "https://docs.gravwell.io/v5.5.4/"
Expand Down
16 changes: 16 additions & 0 deletions changelog/5.5.6.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog for version 5.5.6

## Released 07 October 2024

## Gravwell

```{note}
This release contains high priority bug fixes for multifiltering acceleration.
Gravwell highly recommends that all users upgrade to 5.5.6 or above.

False negatives may occur in versions 5.3.0-5.5.5 when using multifilters with the index engine combined with the fulltext accelerator.
```

### Bug Fixes

* Fixed multifiltering accelration issue with index engine that could result in false negatives.
3 changes: 2 additions & 1 deletion changelog/list.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
maxdepth: 1
caption: Current Release
---
5.5.5 <5.5.5>
5.5.6 <5.5.6>
```

## Previous Versions
Expand All @@ -18,6 +18,7 @@ maxdepth: 1
caption: Previous Releases
---

5.5.5 <5.5.5>
5.5.4 <5.5.4>
5.5.3 <5.5.3>
5.5.2 <5.5.2>
Expand Down
2 changes: 1 addition & 1 deletion conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
project = "Gravwell"
copyright = f"Gravwell, Inc. {date.today().year}"
author = "Gravwell, Inc."
release = "v5.5.5"
release = "v5.5.6"

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
Expand Down
9 changes: 7 additions & 2 deletions configuration/ageout.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Time-based ageout manages data based on time retention requirements. For exampl
A shard will only be aged-out when its *most recent* entries exceed the duration.
```

An example well configuration using only a hot pool of data and deleting data that is more than 30 days old:
An example well configuration using only a hot pool of data and deleting data with a timestamp that is more than 30 days old:

```
[Storage-Well "syslog"]
Expand All @@ -82,8 +82,9 @@ An example configuration in which data is moved from the hot pool to the cold po
```

```{note}
In the above configuration, data will be deleted permanently when it is 90 days old, having spent 7 days in the hot pool and 83 days in the cold pool.
In the above configuration, data will be deleted permanently when the entry timestamps are 90 days old, having spent 7 days in the hot pool and 83 days in the cold pool. The `Hot-Duration` and `Cold-Duration` values are not cumulative; they specify the maximum age of entries.
```

The Time based ageout is invoked once per day, sweeping each pool for shards that can be aged out. By default the sweep happens at midnight UTC, but the execution time can be overridden in the well configuration with the Ageout-Time-Override directive. The override directive is specified in 24 hour UTC time.

An example configuration that overrides the ageout time checks to occur at 7PM UTC:
Expand All @@ -99,6 +100,10 @@ An example configuration that overrides the ageout time checks to occur at 7PM U
Ageout-Time-Override=19:00
```

```{warning}
Data age is calculated as the time difference between now and the timestamp on the entry; if an entry is ingested with a timestamp 90 days in the past, its age is considered to be 90 days. Ingesting historical data with a time based ageout may cause the data to be aged out immediately.
```

## Total Storage-Based Ageout Rules

Total storage constraints allots a specific amount of storage in a volume regardless of time spans. Storage constraints allow a Gravwell indexer to make aggressive and full use of high speed storage pools which may be of limited size (such as NVME flash). The indexer will keep entries in the storage pool, as long as the well isn't consuming more than allowed. Storage constraints allow for unexpected bursts of ingest without disrupting data storage. For example, if an indexer has 1TB of high speed flash storage which typically handles 7 days of hot storage but an unexpected data event causes 600GB of ingest in a single day, the indexer can age out the older data to the cold pool without disrupting the hot pool's ability to take in new data. Shards are prioritized by time; the oldest shards are aged out first for both hot and cold pools.
Expand Down
2 changes: 1 addition & 1 deletion quickstart/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This guide is suitable for Community Edition users as well as users with a paid

You may find the [installation checklist](checklist) and the [glossary](/glossary/glossary) useful companions to this document.

If you are interested in a complete training package, please see the [complete training PDF](https://github.com/gravwell/training/releases/download/v5.5.5/gravwell_training_v5.5.5.pdf). The Gravwell training PDF is the complete training manual which is paired with labs and exercises. The exercises are built from the open source [Gravwell Training](https://github.com/gravwell/training) repository.
If you are interested in a complete training package, please see the [complete training PDF](https://github.com/gravwell/training/releases/download/v5.5.6/gravwell_training_v5.5.6.pdf). The Gravwell training PDF is the complete training manual which is paired with labs and exercises. The exercises are built from the open source [Gravwell Training](https://github.com/gravwell/training) repository.

```{note}
Community Edition users will need to obtain their own license from [https://www.gravwell.io/download](https://www.gravwell.io/download) before beginning installation. Paid users should already have received a license file via email.
Expand Down
4 changes: 4 additions & 0 deletions resources/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ To change the actual contents of the resource, drag a file into the gray 'File'
Changes to the resource description will not be saved unless the "Save" button is clicked.
```

```{note}
All resources are validated using a one way hash. The hash displayed is a base64-encoded MD5 hash.
```

## Using resources

Resources are used by some search modules. This example will use the 'lookup' module to do a basic reference lookup of MAC address to hostname (which we originally generated from a csv file).
Expand Down