Fix jira_transition_to schema bug

[toxisch]

Description

Checklist

• I have reviewed the contributing guidelines.
• I have included unit tests for my changes or additions.
• I have successfully run make test-docker with my changes.
• I have manually tested all relevant modes of the change in this PR.
• I have updated the documentation.
• I have updated the changelog.

Questions or Comments

[nsano-rururu]

Please add the corresponding content to the CHANGE LOG.
https://github.com/jertel/elastalert2/blob/master/CHANGELOG.md

Please check that you have performed the Checklist.

[toxisch]

@nsano-rururu I'am not able to test the stuff on my environment. Build does not work. I usually use the docker image.

[nsano-rururu]

@nsano-rururu I'am not able to test the stuff on my environment. Build does not work. I usually use the docker image.

When you run make test-docker, docker-compose will run all the test code currently created in the docker environment and display the results. Do you say you can't do that either?

[nsano-rururu]

I mean, I usually use docker images too

[nsano-rururu]

example

/home/user/dkwork2/es
|--docker-compose.yml
|--es
|  |--config
|  |  |--elasticsearch.yml
|  |--data
|--fluentd
|  |--dockerfiles
|  |  |--Dockerfile
|  |--etc
|  |  |--fluent.conf
|--mariadb
|  |--data
|  |--etc
|  |  |--mymariadb.cnf
|  |--log

docker-compose.yml

version: "3.7"
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - ES_JAVA_OPTS=-Xms256m -Xmx512m
      - discovery.type=single-node
    restart: always
    volumes:
      - ./es/data:/usr/share/elasticsearch/data
      - ./es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:9200 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 180s

  fluentd:
    container_name: fluentd
    build: ./fluentd/dockerfiles
    image: fluentd:1.9.3
    ports:
      - 24224:24224
      - 24224:24224/udp
    environment:
      - FLUENTD_CONF=fluent.conf
    volumes:
      - ./fluentd/etc/fluent.conf:/fluentd/etc/fluent.conf
      - ./mariadb/log:/var/log/mysql
    user: root
    restart: always

  mariadb:
    container_name: mariadb
    image: mariadb:10.4.12
    ports:
      - 3306:3306
    environment:
      - MYSQL_ROOT_PASSWORD=mariadb
    volumes:
      - ./mariadb/etc:/etc/mysql/conf.d
      - ./mariadb/log:/var/log/mysql
    restart: always

es/config/elasticsearch.yml

cluster.name: "docker-cluster"
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1

fluentd/dockerfiles/Dockerfile

FROM fluent/fluentd:v1.9.3-debian-1.0

# Use root account to use apt
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN buildDeps="sudo make gcc g++ libc-dev" \
 && apt-get update \
 && apt-get install -y --no-install-recommends $buildDeps \
 && sudo gem install fluent-plugin-elasticsearch -v 4.0.6 \
 && sudo gem install fluent-plugin-mysqlslowquery -v 0.0.9 \
 && sudo gem sources --clear-all \
 && SUDO_FORCE_REMOVE=yes \
    apt-get purge -y --auto-remove \
                  -o APT::AutoRemove::RecommendsImportant=false \
                  $buildDeps \
 && rm -rf /var/lib/apt/lists/* \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

USER fluent

fluentd/etc/fluent.conf

<source>
  @type mysql_slow_query
  path /var/log/mysql/slow.log
  pos_file /tmp/mysql/slow.pos
  tag mysqld.slow_query
   <parse>
     @type none
   </parse>
</source>
<source>
  @type tail
  format none
  path /var/log/mysql/general.log
  pos_file /tmp/mysql/general.pos
  tag mysqld.general
</source>
<source>
  @type tail
  format none
  path /var/log/mysql/error.log
  pos_file /tmp/mysql/error.pos
  tag mysqld.error
</source>

<match **.**>
  @type copy
  <store>
    @type stdout
  </store>
  <store>
    @type elasticsearch
    include_tag_key true
    tag_key @log_name
    logstash_format true
    logstash_prefix mariadb-log
    host elasticsearch
    port 9200
    index_name mysql
    flush_interval 10s
  </store>
</match>

mariadb/etc/mymariadb.cnf

[mysqld]
general_log
general_log_file=/var/log/mysql/general.log
slow_query_log
slow_query_log_file=/var/log/mysql/slow.log
long_query_time=5 
log-queries-not-using-indexes
log-error=/var/log/mysql/error.log

setting

cd /home/user/dkwork2/es
chmod 777 es/data
chmod 777 mariadb/data
chmod 777 mariadb/log
docker-compose up -d

All you have to do is execute the following command in the python virtual environment. I don't know what you're talking about.

python setup.py install
mkdir -p ruledir
touch config.yaml
vi config.yaml
vi rule.yaml
python -m elastalert.elastalert --verbose --rule ruledir/rule.yaml

[nsano-rururu]

Isn't it that difficult?

[nsano-rururu]

I don't think it is necessary to check the operation of another fix this time. However, I wanted you to know how to check the operation for this time.

[nsano-rururu]

Addition to the CHANGE LOG is mandatory.

[jertel]

Thanks for submitting the PR @toxisch! Did you manually run ElastAlert 2 with your schema change and verify that it's alerting properly, and that your change corrects the underlying issue your PR addresses? If so you can check off the 4th checkbox.

[toxisch]

So I've wasted several hours now. But I can't get the tests to run nor can I check the function manually. This is mainly because I have to work with minikube and cannot use docker desktop. also I am cut off by a VPN to test the function in our environment, I would have to deploy everything to a docker repo but that is not possible because of the VPN. This does not get along with Minikube.
I'm terribly sorry, but I can't meet the pull request requirements. I give up.

[nsano-rururu]

@toxisch

Are you at work?
Does that mean you don't have an environment at home?

By the way, the change log is added to other changes, but it is not added at the top, but it is added at the bottom of what is described because it is in the order of correspondence.

[nsano-rururu]

@jertel

The fix itself is correct, so it's the change you want to merge.
I'm not using jira so I can't do a working check instead.

[jertel]

@toxisch I've worked in corporate environments like that many years ago so I understand the frustration. If I merge this to master and have it build the elastalert2:latest image, would that be enough for you to have it pulled down into your minikube environment to validate that the JIRA functionality works? We need to be confident that JIRA continues to function before I can cut the next release. Thanks for trying to work through the challenges.

[toxisch]

@nsano-rururu: office = home it's covid time :-) I changed the order of may changlog entry.

@jertel: If I am able to pull the image directly from dockerhub I can test the Jira functionality for you.

[nsano-rururu]

@toxisch

I pushed to docker hub with my personal account. Please check the operation.

https://hub.docker.com/r/rururukenken/elastalert2

git clone https://github.com/jertel/elastalert2.git
cd elastalert2
git fetch origin pull/721/head:fix_jira_transition_to_schema
git checkout fix_jira_transition_to_schema
docker build -t elastalert2 .
docker tag elastalert2:latest rururukenken/elastalert2:latest
docker push rururukenken/elastalert2:latest

[toxisch]

@nsano-rururu Hi, with your private Docker image I was able to test the Jira feature successfully. For existing tickets, they are now successfully changed to a different status. The change works.