fixed double path /opt/elastalert/elastalert on linux user elastalert #656

teochenglim · 2022-01-13T08:30:05Z

Description

The system user home is expected to be '/opt/elastalert' same as default working directory. However the Dockerfile is currently created at /opt/elastalert/elastalert. The home directory is quite important as in while user try to put AWS credential, we expected '/opt/elastalert' as home but not '/opt/elastalert/elastalert'

$ docker run -it python:3-slim-buster bash
root@7e6ea39a51b3:/# groupadd -g 1000 elastalert
root@7e6ea39a51b3:/# useradd -u 1000 -g 1000 -M -b /opt/elastalert -s /sbin/nologin -c "ElastAlert 2 User" elastalert
root@7e6ea39a51b3:/# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
_apt:x:100:65534::/nonexistent:/usr/sbin/nologin
elastalert:x:1000:1000:ElastAlert 2 User:/opt/elastalert/elastalert:/sbin/nologin
root@7e6ea39a51b3:/#

Checklist

I have reviewed the contributing guidelines.
I have included unit tests for my changes or additions.
I have successfully run make test-docker with my changes.
I have manually tested all relevant modes of the change in this PR.
I have updated the documentation.
I have updated the changelog.

Questions or Comments

Just avoid confusion for others and less troubleshooting while it should works without much surprised and troubleshooting.

For example below is expected to run it locally in docker container

$ docker run -it \
  --entrypoint bash \
  -v $(pwd)/rules:/opt/elastalert/rules \
  -v $(pwd)/config.yaml:/opt/elastalert/config.yaml \
  -v ~/.aws/:/opt/elastalert/.aws/  \
  jertel/elastalert2

jertel · 2022-01-13T14:14:07Z

Thanks for the contribution. I noticed you checked off all of the contribution guideline requirements, yet I don't see any an update to the CHANGELOG, which in this case is important since some users will have adapted their -v binds to use the original home path. Not notifying other users that this has changed is going to cause problems for them. If a contribution guideline doesn't apply to a PR that's understandable.

teochenglim · 2022-01-14T01:14:10Z

test run

$ sudo make clean; sudo make test-docker
Password:
make -C docs clean
rm -rf build/*
find . -name '*.pyc' -delete
find . -name '__pycache__' -delete
rm -rf virtualenv_run tests/.tox tests/.coverage *.egg-info docs/build
docker-compose -f tests/docker-compose.yml --project-name elastalert build tox
[+] Building 123.7s (11/11) FINISHED
 => [internal] load build definition from Dockerfile-test                                                                                                                    0.2s
 => => transferring dockerfile: 232B                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                            0.1s
 => => transferring context: 2B                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:3-slim-buster                                                                                                      0.0s
 => [internal] load build context                                                                                                                                            0.1s
 => => transferring context: 751B                                                                                                                                            0.0s
 => [1/6] FROM docker.io/library/python:3-slim-buster                                                                                                                        0.2s
 => [2/6] RUN apt update && apt upgrade -y                                                                                                                                  10.9s
 => [3/6] RUN apt install -y gcc libffi-dev                                                                                                                                 21.1s
 => [4/6] WORKDIR /home/elastalert                                                                                                                                           0.1s
 => [5/6] ADD requirements*.txt ./                                                                                                                                           0.1s
 => [6/6] RUN pip3 install -r requirements-dev.txt                                                                                                                          81.2s
 => exporting to image                                                                                                                                                       9.7s
 => => exporting layers                                                                                                                                                      9.6s
 => => writing image sha256:422285adb1bd6b8ede2eee6cfc59ee3e90ad2ccc3a5edaf1b3fd8acd3cea4db7                                                                                 0.0s
 => => naming to docker.io/library/elastalert_tox                                                                                                                            0.0s
docker-compose -f tests/docker-compose.yml --project-name elastalert run --rm tox \
        tox -c tests/tox.ini --
[+] Running 1/1
 ⠿ Network elastalert_default  Created                                                                                                                                       0.1s
GLOB sdist-make: /home/elastalert/setup.py
py310 create: /home/elastalert/tests/.tox/py310
py310 installdeps: -r../requirements-dev.txt
py310 inst: /home/elastalert/tests/.tox/.tmp/package/1/elastalert2-2.3.0.zip
py310 installed: alabaster==0.7.12,APScheduler==3.8.1,astroid==2.9.3,attrs==21.4.0,aws-requests-auth==0.4.3,Babel==2.9.1,boto3==1.20.35,botocore==1.23.35,certifi==2021.10.8,cffi==1.15.0,cfgv==3.3.1,charset-normalizer==2.0.10,coverage==6.2,croniter==1.1.0,cryptography==36.0.1,defusedxml==0.7.1,distlib==0.3.4,docopt==0.6.2,docutils==0.17.1,elastalert2 @ file:///home/elastalert/tests/.tox/.tmp/package/1/elastalert2-2.3.0.zip,elasticsearch==7.0.0,envparse==0.2.0,execnet==1.9.0,exotel==0.1.5,filelock==3.4.2,flake8==4.0.1,flake8-absolute-import==1.0.0.1,identify==2.4.4,idna==3.3,imagesize==1.3.0,importlib-metadata==4.10.0,iniconfig==1.1.1,isort==5.10.1,jeepney==0.7.1,Jinja2==3.0.3,jira==3.1.1,jmespath==0.10.0,jsonpointer==2.2,jsonschema==4.4.0,keyring==23.5.0,lazy-object-proxy==1.7.1,m2r2==0.3.2,MarkupSafe==2.0.1,mccabe==0.6.1,mistune==0.8.4,nodeenv==1.6.0,oauthlib==3.1.1,packaging==21.3,platformdirs==2.4.1,pluggy==1.0.0,pre-commit==2.16.0,prison==0.2.1,prometheus-client==0.12.0,py==1.11.0,py-zabbix==1.1.7,pycodestyle==2.8.0,pycparser==2.21,pyflakes==2.4.0,Pygments==2.11.2,PyJWT==1.7.1,pylint==2.12.2,pyparsing==3.0.6,pyrsistent==0.18.0,pytest==6.2.5,pytest-cov==3.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-dateutil==2.8.2,pytz==2021.3,PyYAML==6.0,requests==2.27.1,requests-oauthlib==1.3.0,requests-toolbelt==0.9.1,s3transfer==0.5.0,SecretStorage==3.3.1,six==1.16.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,Sphinx==4.3.2,sphinx-rtd-theme==1.0.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,statsd-tags==3.2.1.post1,stomp.py==7.0.0,tencentcloud-sdk-python==3.0.565,texttable==1.6.4,toml==0.10.2,tomli==2.0.0,tox==3.24.4,twilio==6.57.0,tzlocal==2.1,urllib3==1.26.8,virtualenv==20.13.0,wrapt==1.13.3,zipp==3.7.0
py310 run-test-pre: PYTHONHASHSEED='17958546'
py310 run-test: commands[0] | pytest --cov=../elastalert --cov-report=term-missing --cov-branch --strict-markers . -n 4
============================================================================== test session starts ===============================================================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
cachedir: .tox/py310/.pytest_cache
rootdir: /home/elastalert/tests, configfile: pytest.ini
plugins: cov-3.0.0, forked-1.4.0, xdist-2.5.0
gw0 [904] / gw1 [904] / gw2 [904] / gw3 [904]
.....................................................................................s.................................................................................... [ 18%]
....................................s.............................s........................................................................................................ [ 37%]
.........s................................................................................................................................................................ [ 56%]
.......................................................................................................................................................................... [ 75%]
.......................................................................................................................................................................... [ 94%]
.....................................................                                                                                                                      [100%]

---------- coverage: platform linux, python 3.10.1-final-0 -----------
Name                                                           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------
/home/elastalert/elastalert/__init__.py                           64     40     26      0    29%   42, 49-64, 70, 76, 82-83, 89-90, 96, 102-112, 256-266
/home/elastalert/elastalert/alerters/__init__.py                   0      0      0      0   100%
/home/elastalert/elastalert/alerters/alerta.py                    68      0     16      0   100%
/home/elastalert/elastalert/alerters/alertmanager.py              54      1     12      1    97%   40
/home/elastalert/elastalert/alerters/chatwork.py                  36      0      8      0   100%
/home/elastalert/elastalert/alerters/command.py                   36      0     14      0   100%
/home/elastalert/elastalert/alerters/datadog.py                   23      0      2      0   100%
/home/elastalert/elastalert/alerters/debug.py                     12      0      6      0   100%
/home/elastalert/elastalert/alerters/dingtalk.py                  48      0     14      0   100%
/home/elastalert/elastalert/alerters/discord.py                   52      0     10      0   100%
/home/elastalert/elastalert/alerters/email.py                    104      9     52      6    90%   53-54, 62->68, 66->68, 75-79, 107->109, 113-114
/home/elastalert/elastalert/alerters/exotel.py                    26      0      4      0   100%
/home/elastalert/elastalert/alerters/gitter.py                    25      0      2      0   100%
/home/elastalert/elastalert/alerters/googlechat.py                53      0     16      0   100%
/home/elastalert/elastalert/alerters/httppost2.py                 55      0     18      0   100%
/home/elastalert/elastalert/alerters/httppost.py                  44      0     14      0   100%
/home/elastalert/elastalert/alerters/jira.py                     245     47    142     24    76%   94-101, 112, 132, 164, 179, 186, 199, 206, 232-236, 241, 256, 259, 277-280, 299-300, 306-307, 308->303, 312-313, 315-319, 322-323, 326, 335->350, 349, 353-354, 359->364, 363, 369, 376, 381, 384->388, 390
/home/elastalert/elastalert/alerters/line.py                      27      0      8      0   100%
/home/elastalert/elastalert/alerters/mattermost.py               105      2     50      2    96%   56, 69
/home/elastalert/elastalert/alerters/opsgenie.py                 143      0     82      0   100%
/home/elastalert/elastalert/alerters/pagerduty.py                 82      6     44      2    90%   118-120, 135-137
/home/elastalert/elastalert/alerters/pagertree.py                 24      0      2      0   100%
/home/elastalert/elastalert/alerters/rocketchat.py                77      1     26      1    98%   50
/home/elastalert/elastalert/alerters/servicenow.py                31      0      8      0   100%
/home/elastalert/elastalert/alerters/ses.py                       59     16     32      6    63%   29, 34, 39, 43, 50-59, 63, 96
/home/elastalert/elastalert/alerters/slack.py                    116      1     48      1    99%   63
/home/elastalert/elastalert/alerters/sns.py                       28      4      4      1    84%   20-21, 34, 44
/home/elastalert/elastalert/alerters/stomp.py                     48      8      8      2    82%   24, 29-35, 71-73, 76
/home/elastalert/elastalert/alerters/teams.py                     45      0     16      0   100%
/home/elastalert/elastalert/alerters/telegram.py                  40      0      8      0   100%
/home/elastalert/elastalert/alerters/tencentsms.py                64      0     10      0   100%
/home/elastalert/elastalert/alerters/thehive.py                   70      0     22      0   100%
/home/elastalert/elastalert/alerters/twilio.py                    29      0      8      0   100%
/home/elastalert/elastalert/alerters/victorops.py                 33      0      4      0   100%
/home/elastalert/elastalert/alerters/zabbix.py                    58     23     14      2    60%   13-20, 26-41, 71, 87
/home/elastalert/elastalert/alerts.py                            207     10    132      9    94%   16-19, 84, 113, 155->exit, 167, 226->232, 231, 252, 287->302, 320, 325
/home/elastalert/elastalert/auth.py                               26      0      8      0   100%
/home/elastalert/elastalert/config.py                             73      8     34      5    86%   65-66, 69, 105, 135, 138-140
/home/elastalert/elastalert/create_index.py                      164    130     42      1    18%   23-118, 155-273, 277
/home/elastalert/elastalert/elastalert.py                       1319    420    651    119    65%   105, 110, 113, 118->121, 122-124, 165, 178-179, 181, 185, 222->225, 237, 244-245, 252-284, 292-306, 325, 330, 339->338, 347-348, 384, 395, 403-409, 416, 430, 470, 480, 484-490, 495-514, 539-547, 554, 565-607, 645, 647, 650-652, 663, 678, 680, 686-690, 693-697, 709->711, 721-724, 726->exit, 734-735, 777, 781-784, 790-800, 819, 835-838, 853, 868, 908, 919-920, 939, 942-943, 948, 981, 1010-1027, 1038-1042, 1048-1057, 1061-1065, 1120, 1134-1142, 1148-1149, 1151-1175, 1180->1179, 1187->1131, 1191->1212, 1196-1197, 1199, 1206->1192, 1209, 1217-1224, 1240->exit, 1245-1248, 1251, 1254->1258, 1297-1302, 1305-1307, 1311-1313, 1322, 1329, 1333-1346, 1352, 1356-1373, 1390-1395, 1408-1409, 1413-1422, 1435->1434, 1460->1459, 1466-1469, 1487->1490, 1496, 1501-1502, 1520, 1526-1533, 1539-1540, 1549, 1556-1574, 1578-1587, 1590-1592, 1595-1598, 1610-1613, 1616, 1620-1622, 1634-1636, 1662, 1665-1668, 1695, 1705-1706, 1714, 1734, 1740, 1746-1748, 1759-1761, 1765->1764, 1769, 1775->1752, 1796, 1799-1800, 1805-1814, 1829, 1837, 1840-1841, 1852, 1856, 1861-1865, 1880-1884, 1900-1901, 1950-1951, 1954-1955, 1960, 1966-1968, 1971-1972, 1993, 1997, 2006-2010, 2015-2018, 2020-2027, 2057-2058, 2060-2068, 2071, 2087-2088, 2094->2096, 2141, 2146-2148, 2152-2162, 2166
/home/elastalert/elastalert/enhancements.py                       11      0      6      0   100%
/home/elastalert/elastalert/kibana.py                             79      0     38      0   100%
/home/elastalert/elastalert/kibana_discover.py                    61      0     18      0   100%
/home/elastalert/elastalert/kibana_external_url_formatter.py      88      0     28      0   100%
/home/elastalert/elastalert/loaders.py                           390     84    198     42    74%   169-170, 172, 268, 305->307, 308, 313, 314->316, 317, 319, 321, 323, 325, 327, 329, 334-335, 359-380, 384-387, 391, 394, 407-408, 411-412, 413->416, 422, 423->425, 425->427, 428, 435->434, 441, 452-453, 458-460, 466, 474, 483-489, 498, 504, 508-509, 512->exit, 528, 533, 558-563, 575, 589, 595, 603-607, 612-613, 635-638
/home/elastalert/elastalert/prometheus_wrapper.py                 38     32     12      0    16%   9-24, 27, 31-34, 39-55
/home/elastalert/elastalert/rule_from_kibana.py                   29     29      6      0     0%   3-47
/home/elastalert/elastalert/ruletypes.py                         772    181    444     36    74%   99-102, 196->200, 251->exit, 261, 269->268, 274-282, 325, 334-345, 349-352, 356-359, 362, 367-387, 413, 432, 434-442, 454-459, 490-496, 513-514, 529-530, 544-560, 565-575, 624-633, 658-660, 664, 666, 669, 671-675, 679-681, 690, 692, 707-708, 719, 730, 735, 751-754, 756, 764-774, 912->911, 937, 986, 990-1002, 1021, 1031, 1055, 1073, 1075, 1095, 1104, 1120, 1163-1177, 1181-1186, 1194-1205, 1212-1234, 1240-1247
/home/elastalert/elastalert/test_rule.py                         302    302    159      0     0%   3-522
/home/elastalert/elastalert/util.py                              305      1    124      2    99%   295->298, 475
/home/elastalert/elastalert/yaml.py                                6      0      2      0   100%
----------------------------------------------------------------------------------------------------------
TOTAL                                                           5894   1355   2652    262    75%

=================================================================== 900 passed, 4 skipped in 129.85s (0:02:09) ===================================================================
py310 run-test: commands[1] | flake8 --config ../setup.cfg .
docs create: /home/elastalert/tests/.tox/docs
docs installdeps: -r../requirements-dev.txt
docs inst: /home/elastalert/tests/.tox/.tmp/package/1/elastalert2-2.3.0.zip
docs installed: alabaster==0.7.12,APScheduler==3.8.1,astroid==2.9.3,attrs==21.4.0,aws-requests-auth==0.4.3,Babel==2.9.1,boto3==1.20.35,botocore==1.23.35,certifi==2021.10.8,cffi==1.15.0,cfgv==3.3.1,charset-normalizer==2.0.10,coverage==6.2,croniter==1.1.0,cryptography==36.0.1,defusedxml==0.7.1,distlib==0.3.4,docopt==0.6.2,docutils==0.17.1,elastalert2 @ file:///home/elastalert/tests/.tox/.tmp/package/1/elastalert2-2.3.0.zip,elasticsearch==7.0.0,envparse==0.2.0,execnet==1.9.0,exotel==0.1.5,filelock==3.4.2,flake8==4.0.1,flake8-absolute-import==1.0.0.1,identify==2.4.4,idna==3.3,imagesize==1.3.0,importlib-metadata==4.10.0,iniconfig==1.1.1,isort==5.10.1,jeepney==0.7.1,Jinja2==3.0.3,jira==3.1.1,jmespath==0.10.0,jsonpointer==2.2,jsonschema==4.4.0,keyring==23.5.0,lazy-object-proxy==1.7.1,m2r2==0.3.2,MarkupSafe==2.0.1,mccabe==0.6.1,mistune==0.8.4,nodeenv==1.6.0,oauthlib==3.1.1,packaging==21.3,platformdirs==2.4.1,pluggy==1.0.0,pre-commit==2.16.0,prison==0.2.1,prometheus-client==0.12.0,py==1.11.0,py-zabbix==1.1.7,pycodestyle==2.8.0,pycparser==2.21,pyflakes==2.4.0,Pygments==2.11.2,PyJWT==1.7.1,pylint==2.12.2,pyparsing==3.0.6,pyrsistent==0.18.0,pytest==6.2.5,pytest-cov==3.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-dateutil==2.8.2,pytz==2021.3,PyYAML==6.0,requests==2.27.1,requests-oauthlib==1.3.0,requests-toolbelt==0.9.1,s3transfer==0.5.0,SecretStorage==3.3.1,six==1.16.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,Sphinx==4.3.2,sphinx-rtd-theme==1.0.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,statsd-tags==3.2.1.post1,stomp.py==7.0.0,tencentcloud-sdk-python==3.0.565,texttable==1.6.4,toml==0.10.2,tomli==2.0.0,tox==3.24.4,twilio==6.57.0,tzlocal==2.1,urllib3==1.26.8,virtualenv==20.13.0,wrapt==1.13.3,zipp==3.7.0
docs run-test-pre: PYTHONHASHSEED='17958546'
docs run-test: commands[0] | sphinx-build -b html -d build/doctrees -W source build/html
Running Sphinx v4.3.2
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 14 source files that are out of date
updating environment: [new config] 14 added, 0 changed, 0 removed
reading sources... [  7%] elastalert
reading sources... [ 14%] elastalert_status
reading sources... [ 21%] elasticsearch_security_privileges
reading sources... [ 28%] index
reading sources... [ 35%] recipes/adding_alerts
reading sources... [ 42%] recipes/adding_enhancements
reading sources... [ 50%] recipes/adding_loaders
reading sources... [ 57%] recipes/adding_rules
reading sources... [ 64%] recipes/exposing_rule_metrics
reading sources... [ 71%] recipes/faq
reading sources... [ 78%] recipes/signing_requests
reading sources... [ 85%] recipes/writing_filters
reading sources... [ 92%] ruletypes
reading sources... [100%] running_elastalert

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  7%] elastalert
writing output... [ 14%] elastalert_status
writing output... [ 21%] elasticsearch_security_privileges
writing output... [ 28%] index
writing output... [ 35%] recipes/adding_alerts
writing output... [ 42%] recipes/adding_enhancements
writing output... [ 50%] recipes/adding_loaders
writing output... [ 57%] recipes/adding_rules
writing output... [ 64%] recipes/exposing_rule_metrics
writing output... [ 71%] recipes/faq
writing output... [ 78%] recipes/signing_requests
writing output... [ 85%] recipes/writing_filters
writing output... [ 92%] ruletypes
writing output... [100%] running_elastalert

generating indices... genindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in build/html.
____________________________________________________________________________________ summary _____________________________________________________________________________________
  py310: commands succeeded
  docs: commands succeeded
  congratulations :)

teochenglim · 2022-01-14T01:16:48Z

Hi @jertel, Sorry I miss the change log part, and not sure if I am updating in the correct place this time too. Thank.

jertel

Looks great, and you placed the change in the correct section, thank you!

fixed double path /opt/elastalert/elastalert on linux user elastalert

b197371

update CHANGELOG.md for Dockerfile changes at user home /opt/elastalert/

b5f851d

jertel approved these changes Jan 14, 2022

View reviewed changes

jertel merged commit d6539ba into jertel:master Jan 14, 2022

github-actions bot locked as resolved and limited conversation to collaborators Feb 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixed double path /opt/elastalert/elastalert on linux user elastalert #656

fixed double path /opt/elastalert/elastalert on linux user elastalert #656

teochenglim commented Jan 13, 2022 •

edited

Loading

jertel commented Jan 13, 2022

teochenglim commented Jan 14, 2022

teochenglim commented Jan 14, 2022

jertel left a comment

fixed double path /opt/elastalert/elastalert on linux user elastalert #656

fixed double path /opt/elastalert/elastalert on linux user elastalert #656

Conversation

teochenglim commented Jan 13, 2022 • edited Loading

Description

Checklist

Questions or Comments

jertel commented Jan 13, 2022

teochenglim commented Jan 14, 2022

teochenglim commented Jan 14, 2022

jertel left a comment

Choose a reason for hiding this comment

teochenglim commented Jan 13, 2022 •

edited

Loading