Got an error with "TypeError: deprecated_search() got an unexpected keyword argument 'headers'" #2725

keanliu · 2020-03-29T11:43:38Z

I was trying to have a try with a simple alert, but go an error, searched with goole, without getting any result, the exception stack as following:

[root@elasticsearch ~]# elastalert-test-rule --config config.yaml /root/alert_rules/cpu_too_high.yaml
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
Error running your filter:
TypeError("search() got multiple values for argument 'body'",)
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
1 rules loaded
INFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts
Traceback (most recent call last):
File "/usr/local/bin/elastalert-test-rule", line 11, in
load_entry_point('elastalert==0.2.1', 'console_scripts', 'elastalert-test-rule')()
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 445, in main
test_instance.run_rule_test()
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 437, in run_rule_test
self.run_elastalert(rule_yaml, conf, args)
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 318, in run_elastalert
client.run_rule(rule, endtime, starttime)
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 883, in run_rule
if not self.run_query(rule, rule['starttime'], tmp_endtime):
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 630, in run_query
data = self.get_hits_aggregation(rule, start, end, index, rule.get('query_key', None))
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 564, in get_hits_aggregation
body=query, size=0, ignore_unavailable=True)
File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
TypeError: deprecated_search() got an unexpected keyword argument 'headers'

Environment information:

OS version: CentOS Linux release 7.7.1908 (Core)
Elasticsearch version: 7.6.1
Python version: 3.6.8

lessmian · 2020-03-30T14:29:29Z

The same here with elastalert 0.2.1 and elasticsearch 7.6. Pip freeze:
APScheduler==3.6.3 attrs==19.3.0 aws-requests-auth==0.4.2 blist==1.3.6 boto3==1.12.31 botocore==1.15.31 certifi==2019.11.28 cffi==1.14.0 chardet==3.0.4 configparser==5.0.0 croniter==0.3.31 defusedxml==0.6.0 docopt==0.6.2 docutils==0.15.2 elastalert==0.2.1 elasticsearch==7.6.0 envparse==0.2.0 exotel==0.1.5 future==0.18.2 idna==2.9 importlib-metadata==1.6.0 jira==1.0.14 jmespath==0.9.5 jsonschema==2.6.0 mock==4.0.2 oauthlib==3.1.0 pbr==5.4.4 prison==0.1.3 py-zabbix==1.1.3 pycparser==2.20 PyJWT==1.7.1 pyrsistent==0.16.0 PySocks==1.7.1 PyStaticConfiguration==0.10.4 python-dateutil==2.6.1 python-magic==0.4.15 pytz==2019.3 PyYAML==5.3.1 requests==2.23.0 requests-oauthlib==1.3.0 requests-toolbelt==0.9.1 s3transfer==0.3.3 six==1.14.0 stomp.py==6.0.0 texttable==1.6.2 thehive4py==1.6.0 twilio==6.0.0 tzlocal==2.0.0 urllib3==1.25.8 zipp==3.1.0

keanliu · 2020-03-31T02:51:54Z

Thanks in advance, is there some input upon how to solve the problem?

samz-github · 2020-03-31T14:27:16Z

I was trying to have a try with a simple alert, but go an error, searched with goole, without getting any result, the exception stack as following:

[root@elasticsearch ~]# elastalert-test-rule --config config.yaml /root/alert_rules/cpu_too_high.yaml
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
Error running your filter:
TypeError("search() got multiple values for argument 'body'",)
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
1 rules loaded
INFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts
Traceback (most recent call last):
File "/usr/local/bin/elastalert-test-rule", line 11, in
load_entry_point('elastalert==0.2.1', 'console_scripts', 'elastalert-test-rule')()
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 445, in main
test_instance.run_rule_test()
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 437, in run_rule_test
self.run_elastalert(rule_yaml, conf, args)
File "/usr/local/lib/python3.6/site-packages/elastalert/test_rule.py", line 318, in run_elastalert
client.run_rule(rule, endtime, starttime)
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 883, in run_rule
if not self.run_query(rule, rule['starttime'], tmp_endtime):
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 630, in run_query
data = self.get_hits_aggregation(rule, start, end, index, rule.get('query_key', None))
File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 564, in get_hits_aggregation
body=query, size=0, ignore_unavailable=True)
File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
TypeError: deprecated_search() got an unexpected keyword argument 'headers'

Environment information:

OS version: CentOS Linux release 7.7.1908 (Core)
Elasticsearch version: 7.6.1
Python version: 3.6.8

I also encounter same issue.

kvskjava · 2020-04-02T07:41:12Z

Hi Team,

I also got the same error. Couldn't find what's going wrong. Even the example rule in elastalert git folder(example_rules/example_percentage_match.yaml) itself is not working.

Seems there is a bug in elastalert. Please let me know if any one found the problem/solution to proceed further.

bleizg · 2020-04-02T11:58:33Z

Same error, with the last version of elastalert:

elastalert_error - {'message': "Uncaught exception running rule <removed>: deprecated_search() got an unexpected keyword argument 'headers'", 'traceback':
['Traceback (most recent call last):', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1440, in alert', '    return self.send_alert(match
es, rule, alert_time=alert_time, retried=retried)', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1475, in send_alert', '    counts = s
elf.get_top_counts(rule, start, end, keys, qk=qk)', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1978, in get_top_counts', '    hits_t
erms = self.get_hits_terms(rule, starttime, endtime, index, key, qk, number)', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 517, in ge
t_hits_terms', '    body=query, size=0, ignore_unavailable=True)', '  File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped',
 '    return func(*args, params=params, headers=headers, **kwargs)', "TypeError: deprecated_search() got an unexpected keyword argument 'headers'"], 'data': {'rule': '<removed>'}}

kvskjava · 2020-04-02T14:10:32Z

Has anyone found any previous version which works fine without this error?

Please let me know if anyone finds some way to proceed further. I'm also trying to find out a way to proceed further and will let you know if I find something.

ThinkMo · 2020-04-03T02:05:37Z

use elasticsearch==7.0.0, elasticsearch==7.6.1 will cause this error!

kvskjava · 2020-04-03T08:05:49Z

use elasticsearch==7.1.0, elasticsearch==7.6.1 will cause this error!

Hi ThinkMo,

We didn't get you what to use. Earlier I was using elasticsearch-7.6.1, after your comment I've downloaded elasticsearch-7.1.0 and started it.

When I try to check existing example rule by below command:
elastalert-test-rule --config config.yaml example_rules/example_percentage_match.yaml

still I'm getting same error(TypeError: deprecated_search() got an unexpected keyword argument 'headers')

Am I missing anything or doing anything wrong? Please help.

ThinkMo · 2020-04-03T08:32:52Z

use elasticsearch==7.1.0, elasticsearch==7.6.1 will cause this error!

Hi ThinkMo,

We didn't get you what to use. Earlier I was using elasticsearch-7.6.1, after your comment I've downloaded elasticsearch-7.1.0 and started it.

When I try to check existing example rule by below command:
elastalert-test-rule --config config.yaml example_rules/example_percentage_match.yaml

still I'm getting same error(TypeError: deprecated_search() got an unexpected keyword argument 'headers')

Am I missing anything or doing anything wrong? Please help.

sorry, try this
pip install elasticsearch==7.0.0

i first pip install elasticsearch==7.1.0, same error as you
then pip install elasticsearch==7.0.0, it's ok for me, i forgot update

perceptron01 · 2020-04-03T09:39:13Z

I also got the same error with combination of elastalert==0.2.1 and elasticsearch==7.6.0.

So I version downed python elasticsearch library as below.
pip install elasticsearch==7.5.1

After that, the problem was solved with combination of elastalert==0.2.1 and elasticsearch==7.5.1.

I guess there seems to be something wrong with combination of library elasticsearch==7.6.0 and elastalert.

kvskjava · 2020-04-03T09:50:05Z

use elasticsearch==7.1.0, elasticsearch==7.6.1 will cause this error!

Hi ThinkMo,
We didn't get you what to use. Earlier I was using elasticsearch-7.6.1, after your comment I've downloaded elasticsearch-7.1.0 and started it.
When I try to check existing example rule by below command:
elastalert-test-rule --config config.yaml example_rules/example_percentage_match.yaml
still I'm getting same error(TypeError: deprecated_search() got an unexpected keyword argument 'headers')
Am I missing anything or doing anything wrong? Please help.

sorry, try this
pip install elasticsearch==7.0.0

i first pip install elasticsearch==7.1.0, same error as you
then pip install elasticsearch==7.0.0, it's ok for me, i forgot update

Hi ThinkMo,

Thank you. After trying with pip install elasticsearch=7.0.0, I didn't get that error after executing rules. Thanks a lot.

Seems there is no link with the elasticsearch server we are running(whether it's 7.6.1 or 7.1.0). Only pip install elasticsearch=7.0.0 has to be installed and it worked.

kvskjava · 2020-04-03T11:11:00Z

Hi All,

I'm able to proceed further but when I try to use alert type as stomp, it's not working as expected.
Is it something related to our error? I've created a issue for that with #2731
Please let me know if anyone finds anything to proceed further.

JoshuaSmeda · 2020-04-07T17:01:25Z

Running into the same error. I had some success in manually installing elasticsearch==7.0.0 but the errors still comes through now and then,

bleizg · 2020-04-14T11:14:31Z

Tried with elasticsearch==7.0.0, I still have some errors.
It seems the elasticsearch==7.5.1 is more stable.

jfcantu · 2020-04-15T22:27:23Z

It appears that there were some breaking changes in elasticsearch-py in v7.5.0. I wound up here specifically because of the TypeError("search() got multiple values for argument 'body'",) error, which was caused by this change to client.search() in elasticsearch-py:

Old definition: def search(self, index=None, body=None, params=None):
New definition: def search(self, body=None, index=None, doc_type=None, params=None):

This specifically affects test_rule.py in ElastAlert, which passes index to client.search() as a positional rather than a named parameter, and causes Python to think that it's the body parameter and then fail when it encounters the actual body parameter:

elastalert/elastalert/test_rule.py

Line 86 in 4ebdabf

res = es_client.search(index, size=1, body=query, ignore_unavailable=True)

elastalert/elastalert/test_rule.py

Line 156 in 4ebdabf

 res = es_client.search(index, size=args.max_query_size, body=query, ignore_unavailable=True) 

I'd guess that similar breaking changes were the reason behind the other errors referenced in this issue.

LazarenkoA · 2020-06-17T12:50:31Z

I'd guess that similar breaking changes were the reason behind the other errors referenced in this issue.

hi so what to do in this case? The repository is not fixed, can send PR?

StefanSa · 2020-09-08T13:59:26Z

any news on this case ?

rabiashaikabdulkader · 2020-09-28T10:39:02Z

Has anyone resolved this? I get the same error with the combination of elastalert : 0.2.1 and elastic search 7.5.1

nsano-rururu · 2020-11-18T15:13:33Z

@jfcantu

Please let me know if you know the situation.
Did anyone make a pull request for the problem with elasticsearch-py v7.5.0 and test_rule.py?
Or are you doing nothing in particular?

nsano-rururu · 2020-11-19T16:03:33Z

I tried it with elasticsearch-py 7.10.0 and ElastAlert 0.2.4.

The correction method is as follows.

test_rules.py

Line86

res = es_client.search(index, size=1, body=query, ignore_unavailable=True)
↓
res = es_client.search(index=index, size=1, body=query, ignore_unavailable=True)

Line112

res = es_client.count(index, doc_type=doc_type, body=count_query, ignore_unavailable=True)
↓
res = es_client.count(index=index, doc_type=doc_type, body=count_query, ignore_unavailable=True)

Line 156

res = es_client.search(index, size=args.max_query_size, body=query, ignore_unavailable=True)
↓
res = es_client.search(index=index, size=args.max_query_size, body=query, ignore_unavailable=True)

nsano-rururu · 2020-11-19T16:05:02Z

elasticsearch-py	Test Run	Send real alerts
7.0.0	〇	〇
7.10.0	✖	✖
7.10.0	〇 (fix test_rules.py )	✖
7.10.0	〇 (fix test_rules.py )	〇 (Fix initializing self.thread_data.alerts_sent for running elastalert-test-rule #2991)

When I created a new index, the following warning was output, but it did not affect the operation of Elast Alert. No warning was output in the environment where the index of ElastAlert has already been created.

16:13:49.571Z ERROR elastalert-server:
    ProcessController:  /home/node/.local/lib/python3.8/site-packages/elasticsearch/connection/base.py:190: ElasticsearchDeprecationWarning: Camel case format name dateOptionalTime is deprecated and will be removed in a future version. Use snake case name date_optional_time instead.
      warnings.warn(message, category=ElasticsearchDeprecationWarning)
    /home/node/.local/lib/python3.8/site-packages/elasticsearch/connection/base.py:190: ElasticsearchDeprecationWarning: [types removal] Using include_type_name in put mapping requests is deprecated. The parameter will be removed in the next major version.
      warnings.warn(message, category=ElasticsearchDeprecationWarning)

ashishkaransingh · 2021-03-16T13:13:05Z

Hello,
Has this been resolved, any workaround?
Only getting this error when using rule type: Percentage Match

kvskjava mentioned this issue Apr 2, 2020

Unable to execute rule with metric_aggregation type #2729

Closed

jfcantu mentioned this issue Apr 15, 2020

Errno 99 when testing rule with "Send real alerts" johnsusek/praeco#196

Closed

LazarenkoA mentioned this issue Jun 17, 2020

TypeError("search() got multiple values for argument 'body'",) #2811

Closed

anismiles pushed a commit to rubik-ai/elastalert that referenced this issue Jun 24, 2020

froze elasticsearch version to 7.0.0; ref: Yelp#2725

faa22ec

This was referenced Nov 17, 2020

CVE-2020-7009 nnovaes/elastalert-docker#2

Open

Please don't let this die. ElastAlert usecase is very strong. #2947

Open

nsano-rururu mentioned this issue Nov 22, 2020

Fix elasticsearch-py test_rule.py error #3046

Closed

ashishkaransingh mentioned this issue Mar 16, 2021

TypeError: deprecated_search() got an unexpected keyword argument 'headers' #3151

Closed

nsano-rururu mentioned this issue Mar 29, 2021

Some Fixes #3158

Closed

robrankin mentioned this issue Apr 26, 2021

ES Version revert to 7.0.0 jertel/elastalert2#90

Merged

nsano-rururu mentioned this issue Sep 26, 2021

ElastAlert 0.2.5 #3214

Closed

nsano-rururu mentioned this issue Mar 12, 2022

⚠️ ElastAlert is no longer maintained. We strongly recommend migrating to ElastAlert2. #3241

Closed

CristiRO added a commit to CristiRO/elastalert that referenced this issue Nov 17, 2022

Fix ElastAlert issue Yelp#2725

eada5c2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Got an error with "TypeError: deprecated_search() got an unexpected keyword argument 'headers'" #2725

Got an error with "TypeError: deprecated_search() got an unexpected keyword argument 'headers'" #2725

keanliu commented Mar 29, 2020 •

edited

Loading

lessmian commented Mar 30, 2020 •

edited

Loading

keanliu commented Mar 31, 2020

samz-github commented Mar 31, 2020

kvskjava commented Apr 2, 2020

bleizg commented Apr 2, 2020

kvskjava commented Apr 2, 2020

ThinkMo commented Apr 3, 2020 •

edited

Loading

kvskjava commented Apr 3, 2020

ThinkMo commented Apr 3, 2020 •

edited

Loading

perceptron01 commented Apr 3, 2020

kvskjava commented Apr 3, 2020 •

edited

Loading

kvskjava commented Apr 3, 2020

JoshuaSmeda commented Apr 7, 2020

bleizg commented Apr 14, 2020

jfcantu commented Apr 15, 2020 •

edited

Loading

LazarenkoA commented Jun 17, 2020 •

edited

Loading

StefanSa commented Sep 8, 2020

rabiashaikabdulkader commented Sep 28, 2020

nsano-rururu commented Nov 18, 2020

nsano-rururu commented Nov 19, 2020

nsano-rururu commented Nov 19, 2020 •

edited

Loading

ashishkaransingh commented Mar 16, 2021 •

edited

Loading

Got an error with "TypeError: deprecated_search() got an unexpected keyword argument 'headers'" #2725

Got an error with "TypeError: deprecated_search() got an unexpected keyword argument 'headers'" #2725

Comments

keanliu commented Mar 29, 2020 • edited Loading

lessmian commented Mar 30, 2020 • edited Loading

keanliu commented Mar 31, 2020

samz-github commented Mar 31, 2020

kvskjava commented Apr 2, 2020

bleizg commented Apr 2, 2020

kvskjava commented Apr 2, 2020

ThinkMo commented Apr 3, 2020 • edited Loading

kvskjava commented Apr 3, 2020

ThinkMo commented Apr 3, 2020 • edited Loading

perceptron01 commented Apr 3, 2020

kvskjava commented Apr 3, 2020 • edited Loading

kvskjava commented Apr 3, 2020

JoshuaSmeda commented Apr 7, 2020

bleizg commented Apr 14, 2020

jfcantu commented Apr 15, 2020 • edited Loading

LazarenkoA commented Jun 17, 2020 • edited Loading

StefanSa commented Sep 8, 2020

rabiashaikabdulkader commented Sep 28, 2020

nsano-rururu commented Nov 18, 2020

nsano-rururu commented Nov 19, 2020

nsano-rururu commented Nov 19, 2020 • edited Loading

ashishkaransingh commented Mar 16, 2021 • edited Loading

keanliu commented Mar 29, 2020 •

edited

Loading

lessmian commented Mar 30, 2020 •

edited

Loading

ThinkMo commented Apr 3, 2020 •

edited

Loading

ThinkMo commented Apr 3, 2020 •

edited

Loading

kvskjava commented Apr 3, 2020 •

edited

Loading

jfcantu commented Apr 15, 2020 •

edited

Loading

LazarenkoA commented Jun 17, 2020 •

edited

Loading

nsano-rururu commented Nov 19, 2020 •

edited

Loading

ashishkaransingh commented Mar 16, 2021 •

edited

Loading