-
Notifications
You must be signed in to change notification settings - Fork 285
Docker
The installation and configuration procedure described here has been tested and confirmed on Debian, Fedora, CentOS (Stream 8/9), RedHat, Ubuntu LTS.
The installaton can also work on other Linux OSs that support docker but it may need local adjustments. Feedback is welcome as well.
Minimum working setup:
- 2 cores
- 10 GB of free RAM
- minimum 10 GB of free disk space (actual disk occupation will mainly depend of the number of rules and the amount of traffic on the network). 200GB+ SSD grade is recommended.
-
git,curl -
docker> 17.06.0 (will be installed during SELKS initial setup) -
docker-compose> 1.27.0 (compose v2 will be installed during SELKS initial setup ifdocker composeis not found)
NOTE: For CentOS Stream 8 users - it may be needed to remove the follwoing packages before the install - dnf remove podman buildah
NOTE: For RedHat users - docker needs to be installed via the instructions of the CentOS offical documentation
git clone https://github.com/StamusNetworks/SELKS.git
cd SELKS/docker/
./easy-setup.sh
sudo -E docker compose up -dOnce the containers are up and running, you should just point your browser to https://your.selks.IP.here/
If you chose to install Portainer during the installation, you must visit https://your.selks.IP.here:9443 to set portainer's admin password
If the setup script fails, please take a look at the Manual Docker install and Report an issue
In order to access scirius, you will need following credentials:
- user:
selks-user - password:
selks-user
The easy-setup.sh does the following :
- Checking that
dockeranddocker composeare properly installed and available to the user, and installing them if needed - Generating SSL certificates for nginx that will secure the Scirius web interface
- Generating secret key for the underlying Django
- Creating a
.envfile containing environment variables deduced from the user inputs - Pull the containers
In order to change the options you defined, just run easy-setup.sh again
For production setups Elasticsearch can be assigned to use 32/64GB RAM pending you have that available on the host. It is also recommended to automatically start the containers at boot (on host restart):
./easy-setup.sh --non-interactive -i eno1 --iA --restart-mode always --es-memory 64G
sudo -E docker compose up -d
For indexes/data retention and policies it is recommended to use the Elasticsearch's own ILM https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html
Any Suricata logs and or pcap capture data from Suricata are logged into the host as well in the current docker folder :
containers-data/suricata/logs/
It is recommended for production installs to have some sort of log rotation for those files configured.
The docker installation is design to easily read/digest pcaps too for teaching,training or research and analysis scenarios.
To get help:
./scripts/readpcap.sh -h
Pcap reading script through Suricata
Usage: scripts/readpcap.sh [-c|--(no-)cleanup] [-a|--(no-)autofp] [-s|--set-rulefile <arg>] [-S|--set-rulefile-exclusive <arg>] [-h|--help] [--] <path>
<path>: Path to the pcap file to read. If <path> specifies a directory, all files in that directory
will be processed in order of modified time maintaining flow state between files.
-c, --cleanup, --no-cleanup: Remove all previous data from elasticsearch and suricata. (off by default)
-a, --autofp, --no-autofp: Run in autofp mode instead of single mode. (off by default)
-s, --set-rulefile: Set a file with signatures, which will be loaded together with the rules set in the yaml. (no default)
-S, --set-rulefile-exclusive: Set a file with signatures, which will be loaded exclusively, regardless of the rules set in the yaml. (no default)
-h, --help: Prints help
To read a pcap:
./scripts/readpcap.sh /path/to/file.pcap
To read a pcap but clear/delete all previous data:
./scripts/readpcap.sh -c /path/to/file.pcap
To read a pcap(using all available CPUs for reading), clear/delete all previous data:
./scripts/readpcap.sh -ac /path/to/file.pcap
To read multiple pcaps into the so called tenant mode:
./scripts/readpcap.sh /path/to/file.pcap ; \
./scripts/readpcap.sh /path/to/file2.pcap ; \
./scripts/readpcap.sh /path/to/file3.pcap
Now each pcap will populate with its own file name in the Hunt interface that can be easily selected and zoomed into for investigation or teaching. That way providing an easy way to separately analyse the different cases and compare one case to another or investigate separately.
To delete and remove all data:
./scripts/cleanup.sh
SELKS comes by default with more than 400 visualizations and 28 dashboards for Suricata that serve a wide range of purposes - from network troubleshooting and visibility to threat detection and threat hunting.
To reset the Kibana dashboards use the following procedure:
- Log in to Scirius
- From the right upper corner switcher , go to
Administration. - Select
System Settingsform the left upper corner drop down menu. - Click the
Resetbutton underReset SN dashboards
Log rotation can be defined via the cron log container
dir -1 SELKS/docker/containers-data/cron-jobs/
15min
1min
daily
hourly
monthly
README.md
weekly
Daily log rotation is setup by default:
cat SELKS/docker/containers-data/cron-jobs/daily/suricata-logrotate.sh
#! /bin/sh
#
# Example of rotating the logs within the Suricata container.
#
# Add -v for verbose output.
# Add -f to force rotation.
echo "Rotating Suricata logs"
docker exec suricata logrotate -v /etc/logrotate.d/suricata $@ && echo "done." || echo "ERROR"
The log rotation config file is located here:
cat SELKS/docker/containers-data/suricata/logrotate/suricata
/var/log/suricata/*.log /var/log/suricata/*.json {
daily
missingok
rotate 3
nocompress
sharedscripts
postrotate
suricatasc -c reopen-log-files
endscript
}
PCAP logrotation is set in the helper selks.yaml file:
https://github.com/StamusNetworks/SELKS/blob/master/docker/containers-data/suricata/etc/selks6-addin.yaml#L515
In the installation directory, the file is located in the folder docker/containers-data/suricata/etc/selks6-addin.yaml
There is a default daily cronjob setup that will update any rulesets and threat intel:
cat SELKS/docker/containers-data/cron-jobs/daily/scirius-update-suri-rules.sh
#! /bin/sh
echo "Updating Suricata rules from Scirius"
docker exec scirius python /opt/scirius/manage.py updatesuricata && echo "done." || echo "ERROR"
It can be further modified or moved to another sequence - hourly for example to do it every hour.
If needed, the rulests and threat intel can be manually force updated:
In Scirius , Suricata tab -> click Ruleset actions (left hand side panel) -> select Update,Build, Push -> click on Apply
The /etc/suricata/rules/threshold.conf file inside the suricata docker contains all the alert suppression and thresholding configuration done through Scirius with respect to Suricata.
For example to list current suppression or thresholds:
docker exec suricata cat /etc/suricata/rules/threshold.config
threshold gen_id 1, sig_id 2028765, type both, track by_src, count 1, seconds 60
threshold gen_id 1, sig_id 2028766, type both, track by_src, count 1, seconds 60
threshold gen_id 1, sig_id 2028772, type both, track by_src, count 1, seconds 60
threshold gen_id 1, sig_id 2028800, type both, track by_src, count 1, seconds 60
If by mistake you have chosen to install the package maintainers version - no worries - just build,update and push the ruleset again from Scirius. (Suricata tab -> click Ruleset actions (left hand side panel) -> select Update,Build, Push -> click on Apply).
./easy-setup.sh --helpPut your existing SSL certificate and private key in SELKS/docker/containers-data/nginx/ssl as scirius.crt and scirius.key before running the easy-setup.sh script.
The script provides several command line options to avoid being prompted. This can be useful to automate SELKS deployment. Refer to the help
./easy-setup.sh --non-interactiveAnother example:
The bellow command will setup the SELKS docker instillation:
- in noninteractive mode
- install all containers (elasticsearch/suricata/ngingx/logstash/kibana/scirius/evebox/portainer)
- use interface
eno1as sniffing interface - use 6GB of RAM for Elasticsearch
- start up the installation (docker compose)
./easy-setup.sh --non-interactive -i eno1 --iA --es-memory 6G
sudo -E docker compose up -d
./easy-setup.sh --elk-version <version-number>The version will be the same for Elasticsearch, Kibana and Logstash. It is not possible (and not recommended) to set them individually with different versions each.
./easy-setup.sh --scirius-version <version>The version can be a branch name, a tag, a release number or a git sha
This case is useful for replaying specific pcap investigation cases for fast Security analysis. Sometimes it is useful to setup a dummy interface for sniffing to be sure that no other traffic will be mixed in with the specific pcap replay:
ip link add tppdummy0 type dummy && \
ip link set tppdummy0 up && \
ifconfig tppdummy0 mtu 1572
Then spin up SELKS:
./easy-setup.sh -i tppdummy0
sudo -E docker compose up -d
To check Suricata build info and version:
docker exec suricata suricata --build-info
An example for AF-PACKET Suricata IPS set up with SELKS.
NOTE: On big multi core set ups the total number of threads combined for both interfaces should not be more than the (total number of cores - 4). Ideally less than that as there is also Elasticsearch that needs CPUs.
In containers-data/suricata/etc/selks6-addin.yaml : ::
af-packet:
- interface: enp1s0f0
threads: 4 # or a number that is below half the number of cores available
defrag: no
cluster-type: cluster_flow
cluster-id: 98
copy-mode: ips
copy-iface: enp1s0f1
tpacket-v3: no
ring-size: 2048
use-mmap: yes
- interface: enp1s0f1
threads: 4 # or a number that is below half the number of cores available
cluster-id: 97
defrag: no
cluster-type: cluster_flow
copy-mode: ips
copy-iface: enp1s0f0
tpacket-v3: no
ring-size: 2048
use-mmap: yes
also make sure in /etc/network/interfaces: ::
auto lo
iface lo inet loopback
allow-hotplug eno1
iface eno1 inet static # Management interface
address 192.168.0.100
netmask 255.255.252.0
gateway 192.168.0.1
dns-nameservers 192.168.0.11 192.168.0.6
iface enp1s0f0 inet manual
pre-up ifconfig $IFACE up
post-down ifconfig $IFACE down
iface enp1s0f1 inet manual
pre-up ifconfig $IFACE up
post-down ifconfig $IFACE down
NOTE: When using the easy-setup.sh script for setup, you need to specify both sniffing interfaces.
From Scirius transform the rules you would like from alert to drop:
- Select a rule that you would like to transform into
dropaction - Under
Actionsin the left hand side panel selectTransform rule - Select type
Dropand a ruleset to be used. - Click on
Valid - From the
Suricatamain tab in Scirius - selectRuleset actionson the left hand side panel - Select
Update(optional),Build,PushandApply
If configuration adjustments are needed, please follow the guides below.
In order to tweak suricata config, edit the following file :
SELKS/docker/containers-data/suricata/etc/selks6-addin.yaml
and restart the container :
sudo docker compose restart suricata
The default suricata.yaml can be edited but it resides inside the container and any changes would be overwritten upon upgrade.
The configuration file for elasticsearch resides inside the container and should not be modified directly, as those changes would not be permanent.
Instead, you can set specifics settings in the environment subsection on the elasticsearch section of docker-compose.yml. You can set individual Elasticsearch configuration parameters using Docker environment variables. You can use the setting name directly as the environment variable name.
For example:
environment:
- discovery.type=single-node
- xpack.security.enabled=false
If you cannot do the above, due to for example because your orchestration platform forbids periods in environment variable names, then you can use an alternative style by converting the setting name as follows:
- Change the setting name to uppercase
- Prefix it with ES_SETTING_
- Escape any underscores (_) by duplicating them
- Convert all periods (.) to underscores (_)
For example, -e bootstrap.memory_lock=true becomes -e ES_SETTING_BOOTSTRAP_MEMORY__LOCK=true.
If needed , to change the logstash configuration, please refer to Elasticsearch config. It is done the exact same way just in the logstash subsection of docker-compose.yml.
Changing the default password is highly recommended!
To do so - from the Scirius Administration interface, in the right upper corner click on the selks-user user, select Account settings, on the left hand side panel click Change password.
A password change and a reset can also be done from the command line:
Attach to the Scirius docker
docker exec -it scirius bash
python3 /opt/scirius/manage.py changepassword selks-user
Most docker compose commands will have the following form docker compose COMMAND [container-name]
If no container-name is provided, it will be applied to all SELKS containers
Those commands must be run from the SELKS/docker/ directory
docker compose stop [container-name]docker compose start [container-name]docker compose restart [container-name]docker compose logs [container-name]Other application specific logs are also located in:
ls -lh containers-data/scirius/logs/
total 180K
-rw-r--r-- 1 root root 91 Sep 3 16:00 django-auth.log
-rw-r--r-- 1 root root 177 Sep 3 16:01 django-error.log
-rw-r--r-- 1 root root 158K Sep 3 17:03 elasticsearch.log
-rw-rw-rw- 1 root root 11K Sep 3 14:03 suri_reloader.log
ls -lh containers-data/suricata/logs/
total 1.6G
-rw-r--r-- 1 994 994 1.6G Sep 3 17:04 eve.json
drwxr-xr-x 2 994 994 4.0K Sep 3 16:25 fpc
-rw-r--r-- 1 994 994 11M Sep 3 17:04 stats.log
-rw-r--r-- 1 994 994 75K Sep 3 17:04 suricata.log
Also some container specific logs can be displayed like so, example:
docker exec arkime bash -c "tail -50 /opt/arkime/logs/capture.log"
docker exec arkime bash -c "tail -50 /opt/arkime/logs/viewer.log"
For going directly to the arkime viewer you can use https://your.selks.IP.here/moloch
The docker command below will display general docker container system status. For more information and other options you can reference the docker ps manual here
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
45b360771a63 elastic/logstash:7.16.1 "/usr/local/bin/dock…" 14 minutes ago Up 14 minutes (healthy) 5044/tcp, 9600/tcp logstash
b0eb5703c80c jasonish/suricata:master-amd64 "/etc/suricata/new_e…" 14 minutes ago Up 14 minutes suricata
8b4058a6b7fd ghcr.io/stamusnetworks/scirius:selks "/opt/scirius/bin/st…" 16 minutes ago Up 16 minutes (healthy) 8000/tcp scirius
599911d09abc elastic/kibana:7.16.1 "/bin/tini -- /usr/l…" 16 minutes ago Up 16 minutes (healthy) 5601/tcp kibana
163ff2e09ef4 elastic/elasticsearch:7.16.1 "/bin/tini -- /usr/l…" 16 minutes ago Up 16 minutes (healthy) 9200/tcp, 9300/tcp elasticsearch
26baf1c65e6d nginx "/docker-entrypoint.…" 16 minutes ago Up 16 minutes 80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp nginx
0844ecd5ec86 docker:latest "dockerd-entrypoint.…" 16 minutes ago Up 16 minutes 2375-2376/tcp cron
57fabc79c7f6 ghcr.io/stamusnetworks/arkimeviewer:master "/start-arkimeviewer…" 16 minutes ago Up 16 minutes 8005/tcp arkime
adfc337c2b9e jasonish/evebox:master "/docker-entrypoint.…" 16 minutes ago Up 16 minutes evebox
503ad8991aba portainer/portainer-ce "/portainer --logo h…" 17 minutes ago Up 17 minutes 8000/tcp, 9000/tcp, 0.0.0.0:9443->9443/tcp, :::9443->9443/tcp portainer
docker top [container-name]docker top scirius | head -5
UID PID PPID C STIME TTY TIME CMD
root 287364 287334 0 11:50 ? 00:00:00 /bin/bash /opt/scirius/docker/scirius/bin/start-scirius.sh
root 287973 287364 0 11:50 ? 00:00:00 python /opt/scirius/docker/scirius/suricata/scripts/suri_reloader
root 287975 287364 0 11:50 ? 00:00:00 /usr/local/bin/python /usr/local/bin/gunicorn -w 33 -t 120 -b 0.0.0.0:8000 scirius.wsgi
root 287976 287975 0 11:50 ? 00:00:00 /usr/local/bin/python /usr/local/bin/gunicorn -w 33 -t 120 -b 0.0.0.0:8000 scirius.wsgidocker compose down -vdocker compose up -d [container-name] --force-recreatedocker compose down [container-name]
docker compose pull [container-name]
docker compose up [container-name] --force-recreateIf you are already running SELKS on docker, you can upgrade SELKS and the containers to a fixed version. To do so, simply run from the docker directory:
git pull
docker compose pull
docker compose stop
sudo -E docker compose up -d
If you've encoutered an issue please let us know: Report Issue