Bug in Elasticsearch output tests on index template test

[ghost]

The test_raise_when_no_template function was bogus as it called:

self.assertRaises(RuntimeError, self.run_bot())

self.run_bot() evaluated to None and apparently raised no RuntimeError. I fixed this in d5a288d as Python 3.8 unittests' assertRaises raises an exception itself if the callable is not a callable.

I was not able to find a solution.

@navtej @cert-lv @stone-z Can you have a look at that?

FAIL: Test that a bot raises a RuntimeError if 'rotate_index' is set, but a matching template doesn't exist in ES.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/certtools/intelmq/intelmq/tests/bots/outputs/elasticsearch/test_output.py", line 146, in test_raise_when_no_template
    self.assertRaises(RuntimeError, self.run_bot)
AssertionError: RuntimeError not raised by run_bot
-------------------- >> begin captured logging << --------------------
test-bot: INFO: ElasticsearchOutputBot initialized with id test-bot and intelmq 2.2.0.alpha.1 and python 3.8.0 (default, Oct 15 2019, 17:49:23) as process 7541.
test-bot: INFO: Bot is starting.
urllib3.util.retry: DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 127.0.0.1:9200
urllib3.connectionpool: DEBUG: http://127.0.0.1:9200 "HEAD /_template/intelmq HTTP/1.1" 200 0
elasticsearch: INFO: HEAD http://127.0.0.1:9200/_template/intelmq [status:200 request:0.003s]
elasticsearch: DEBUG: > None
elasticsearch: DEBUG: < 
test-bot: INFO: Bot initialization completed.
urllib3.util.retry: DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
urllib3.connectionpool: DEBUG: http://127.0.0.1:9200 "POST /intelmq-2019-12-10/events HTTP/1.1" 201 172
elasticsearch: INFO: POST http://127.0.0.1:9200/intelmq-2019-12-10/events [status:201 request:0.010s]
elasticsearch: DEBUG: > {"classification.type": "infected-system", "source.asn": 64496, "source.ip": "192.0.2.1", "feed.name": "Example Feed", "extra.foo.bar": "test"}
elasticsearch: DEBUG: < {"_index":"intelmq-2019-12-10","_type":"events","_id":"AW7wbb5BcUpmMLmhhcWy","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}
test-bot: INFO: Bot stopped.
--------------------- >> end captured logging << ---------------------

https://travis-ci.org/certtools/intelmq

[stone-z]

The bot's init() function raises this error. I thought init was called as part of run_bot but I now think the setup is done differently in a test case. I tried a few ways to call this but ultimately wasn't sure what the logic flow was with regard to init() from inside a test case, so the test case was removed in #1513

[ghost]

The bot's init() function raises this error. I thought init was called as part of run_bot but I now think the setup is done differently in a test case.

It is:

intelmq/intelmq/lib/test.py

Line 233 in 8ab414f

self.bot = self.bot_reference(self.bot_id)

That's part of prepare_bot which is called from run_bot.