-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Loki Logstash Plugin #1822
Merged
Merged
Loki Logstash Plugin #1822
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
3159e46
Logstash plugin
adityacs 0994c44
include_labels
adityacs 727bdcb
include_labels
adityacs 88587c1
Removes binary.
cyriltovena 9c9b072
Improve documentation and remove the push path.
cyriltovena d45c9db
Move to cmd.
cyriltovena a21605a
Add more precision for jruby.
cyriltovena 87895e9
Update docs/clients/logstash/README.md
cyriltovena 913dbbb
p
cyriltovena 9b9a250
ignore
cyriltovena 1a4de4a
remove ignore file/
cyriltovena 659ca6b
More precision for installing jruby
cyriltovena c2b33e3
Rename without Grafana
cyriltovena cbef9a4
A lot of refactoring and testing.
cyriltovena d10f042
change delay logic
adityacs 26875cb
Fully tested version.
cyriltovena 5a1af98
Merge branch 'loki_logstash_plugin' of github.com:adityacs/loki into …
cyriltovena 1366ef8
Forgot to save merge.
cyriltovena 99f504d
working version.
cyriltovena 4776178
Makefile + easier docker build.
cyriltovena 425d375
adds ci to build logstash image.
cyriltovena 0d2acee
Fix build for logstash.
cyriltovena 37508d1
Adds example with helm charts.
cyriltovena a8cde47
Fix target to send 10 logs with logstash.
cyriltovena 0a266ec
Improved documentation.
cyriltovena 4ae8cc7
add missing helm add repo for external repo
cyriltovena c87581a
Review comment.
cyriltovena 3e6a5cf
Fixes loki service in Promtail.
cyriltovena 29ee215
Merge remote-tracking branch 'upstream/master' into loki_logstash_plugin
cyriltovena e9fb63a
Update loki-stack version
cyriltovena File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
*.gem | ||
.ruby-version | ||
.bundle | ||
logstash | ||
path | ||
!lib | ||
.rakeTasks |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
FROM logstash:7.6.2 | ||
|
||
USER logstash | ||
ENV PATH /usr/share/logstash/vendor/jruby/bin:/usr/share/logstash/vendor/bundle/jruby/2.5.0/bin:$PATH | ||
ENV LOGSTASH_PATH="/usr/share/logstash" | ||
ENV GEM_PATH /usr/share/logstash/vendor/bundle/jruby/2.5.0 | ||
ENV GEM_HOME /usr/share/logstash/vendor/bundle/jruby/2.5.0 | ||
|
||
RUN gem install bundler:2.1.4 | ||
|
||
COPY --chown=logstash:logstash ./cmd/logstash/ /home/logstash/ | ||
WORKDIR /home/logstash/ | ||
|
||
|
||
RUN bundle install --path=/usr/share/logstash/vendor/bundle && \ | ||
bundle exec rake vendor && \ | ||
bundle exec rspec | ||
|
||
RUN cat logstash-output-loki.gemspec | grep s.version | awk '{print $3}' | cut -d "'" -f 2 > VERSION | ||
|
||
RUN gem build logstash-output-loki.gemspec && \ | ||
PLUGIN_VERSION=$(cat VERSION); /usr/share/logstash/bin/logstash-plugin install logstash-output-loki-${PLUGIN_VERSION}.gem | ||
|
||
EXPOSE 5044 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
source 'https://rubygems.org' | ||
|
||
gemspec | ||
|
||
logstash_path = ENV["LOGSTASH_PATH"] || "./logstash" | ||
|
||
if Dir.exist?(logstash_path) | ||
gem 'logstash-core', :path => "#{logstash_path}/logstash-core" | ||
gem 'logstash-core-plugin-api', :path => "#{logstash_path}/logstash-core-plugin-api" | ||
else | ||
raise 'missing logstash vendoring' | ||
end | ||
|
||
gem "webmock", "~> 3.8" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
# Contributing to Loki Logstash Output Plugin | ||
|
||
For information about hwo to use this plugin see this [documentation](../../docs/clients/logstash/README.md). | ||
|
||
## Install dependencies | ||
|
||
First you need to setup JRuby environment to build this plugin. Refer https://github.com/rbenv/rbenv for setting up your rbenv environment. | ||
|
||
After setting up `rbenv`. Install JRuby | ||
|
||
```bash | ||
rbenv install jruby-9.2.10.0 | ||
rbenv local jruby-9.2.10.0 | ||
``` | ||
|
||
Check that the environment is configured | ||
|
||
```bash | ||
ruby --version | ||
jruby 9.2.10 | ||
``` | ||
|
||
You should use make sure you are running jruby and not ruby. If the command below still shows ruby and not jruby, check that PATH contains `$HOME/.rbenv/shims` and `$HOME/.rbenv/bin`. Also verify that you have this in your bash profile: | ||
|
||
```bash | ||
export PATH="$HOME/.rbenv/bin:$PATH" | ||
eval "$(rbenv init -)" | ||
``` | ||
|
||
Then install bundler | ||
`gem install bundler:2.1.4` | ||
|
||
Follow those instructions to [install logstash](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html) before moving to the next section. | ||
|
||
## Install dependencies and Build plugin | ||
|
||
### Install required packages | ||
|
||
```bash | ||
git clone git@github.com:elastic/logstash.git | ||
cd logstash | ||
git checkout tags/v7.6.2 | ||
export LOGSTASH_PATH=`pwd` | ||
export GEM_PATH=$LOGSTASH_PATH/vendor/bundle/jruby/2.5.0 | ||
export GEM_HOME=$LOGSTASH_PATH/vendor/bundle/jruby/2.5.0 | ||
./gradlew assemble | ||
cd .. | ||
ruby -S bundle install | ||
ruby -S bundle exec rake vendor | ||
``` | ||
|
||
### Build the plugin | ||
|
||
`gem build logstash-output-loki.gemspec` | ||
|
||
### Test | ||
|
||
`ruby -S bundle exec rspec` | ||
|
||
Alternatively if you don't want to install JRuby. Enter inside logstash-loki container. | ||
|
||
```bash | ||
docker build -t logstash-loki ./ | ||
docker run -v `pwd`/spec:/home/logstash/spec -it --rm --entrypoint /bin/sh logstash-loki | ||
bundle exec rspec | ||
``` | ||
|
||
## Install plugin to local logstash | ||
|
||
`bin/logstash-plugin install --no-verify --local logstash-output-loki-1.0.0.gem` | ||
|
||
## Send sample event and check plugin is working | ||
|
||
`bin/logstash -f loki.conf` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
require "logstash/devutils/rake" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice 👍