opensearchpy.exceptions.RequestError: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens

[mmansour]

Hi,

On v 2.11.0 I am getting this error when upserting to latest using opensearch.py... When I downgrade to version 2.9.0 it works fine.

 traceback (most recent call last):
  File "/app/manage.py", line 22, in <module>
    main()
  File "/app/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/richlist/management/commands/get_sgb_explorer_data.py", line 47, in handle
    obj.save()
  File "/app/richlist/models.py", line 65, in save
    super(SongBirdAccount, self).save(*args, **kwargs)
  File "/app/richlist/models.py", line 20, in save
    super(AccountAbstract, self).save(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save
    self.save_base(
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 892, in save_base
    post_save.send(
  File "/usr/local/lib/python3.11/site-packages/django/dispatch/dispatcher.py", line 176, in send
    return [
           ^
  File "/usr/local/lib/python3.11/site-packages/django/dispatch/dispatcher.py", line 177, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/richlist/signals.py", line 9, in save_document_sgb_accounts
    instance.to_elastic(instance.address)
  File "/app/richlist/models.py", line 53, in to_elastic
    client.update(
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/client/utils.py", line 179, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/client/__init__.py", line 1769, in update
    return self.transport.perform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/transport.py", line 409, in perform_request
    raise e
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/transport.py", line 370, in perform_request
    status, headers_response, data = connection.perform_request(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/connection/http_urllib3.py", line 266, in perform_request
    self._raise_error(
  File "/usr/local/lib/python3.11/site-packages/opensearchpy/connection/base.py", line 301, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.RequestError: RequestError(400, 'json_parse_exception', 'Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: (org.opensearch.core.common.io.stream.InputStreamStreamInput); line: 1, column: 2]')

[dblock]

(Edited the issue to be titled with the actual error, which is "'Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens").

@mmansour do you have a step-by-step repro, please? what are you sending to OpenSearch?

[mmansour]

Sorry I don't. It would be hard to create one in this case. It kept happening on every rerun of a script that pulls data from external resources. The upserted JSON looked valid on the offending records. Not sure what was happening. But in my scenario it made sense to just downgrade to the version of opensearch that was working last month.

…

On Tue, Oct 24, 2023 at 9:38 AM Daniel (dB.) Doubrovkine < ***@***.***> wrote: @mmansour <https://github.com/mmansour> do you have a step-by-step repro, please? — Reply to this email directly, view it on GitHub <#10747 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGRODQU5QCEPT4HSWWX2ZLYA7VITAVCNFSM6AAAAAA6HLP7ROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXGYYTMMJVHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dblock]

I think #10802 is the same thing and has a lot of detail. I'm going to close this one. Can you please confirm they are the same problem @mmansour?

[mmansour]

Yeah that looks like the one. Thank you!

…

On Tue, Oct 24, 2023 at 9:52 AM Daniel (dB.) Doubrovkine < ***@***.***> wrote: I think #10802 <#10802> is the same thing and has a lot of detail. I'm going to close this one. Can you please confirm they are the same problem @mmansour <https://github.com/mmansour>? — Reply to this email directly, view it on GitHub <#10747 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGRODVTYXZTQJ5VG2ZGIA3YA7W5PAVCNFSM6AAAAAA6HLP7ROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXGY2DGOBZGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>