signature_method_rejected returned with 3.5.1 but not 3.5.0

[merickso]

Bug summary

I just upgraded to 3.5.1 and am now seeing a 400 response with signature_method_rejected in the body. I can see the following change was added to this latest result.
#1643

Does this have dependencies on the server configuration?

Is there an existing issue for this?

• I have searched the existing issues

Jira Instance type

Jira Server or Data Center (Self-hosted)

Jira instance version

8.20.10

jira-python version

3.5.1

Python Interpreter version

3.8.13

Which operating systems have you used?

• Linux
• macOS
• Windows

Reproduction steps

# 1. Given a Jira client instance
    JIRA()
# 2. When I call the function with argument x
    JIRA(server=constants['JIRA_URL'], oauth=jira_oauth, options={'verify': constants['VALIDATE_JIRA_URL']})

# 3.
The server returns a 400 with a response text of oauth_problem=signature_method_rejected

Stack trace

File \"/home/runner/.local/lib/python3.8/site-packages/jira/client.py\", line 579, in __init__",
    "    si = self.server_info()",
    "  File \"/home/runner/.local/lib/python3.8/site-packages/jira/client.py\", line 3119, in server_info",
    "    j = self._get_json(\"serverInfo\")",
    "  File \"/home/runner/.local/lib/python3.8/site-packages/jira/client.py\", line 3831, in _get_json",
    "    r = self._session.get(url, params=params)",
    "  File \"/home/runner/.local/lib/python3.8/site-packages/requests/sessions.py\", line 600, in get",
    "    return self.request(\"GET\", url, **kwargs)",
    "  File \"/home/runner/.local/lib/python3.8/site-packages/jira/resilientsession.py\", line 246, in request",
    "    elif raise_on_error(response, **processed_kwargs):",
    "  File \"/home/runner/.local/lib/python3.8/site-packages/jira/resilientsession.py\", line 71, in raise_on_error",
    "    raise JIRAError(",
    "jira.exceptions.JIRAError: JiraError HTTP 400 url: https://xxx/rest/api/2/serverInfo",
    "\ttext: oauth_problem=signature_method_rejected",

Expected behaviour

response 200

Additional Context

No response

[merickso]

I rolled back to 3.5.0 and the server no longer returned an error so the problem appears to be specific to 3.5.1

[studioj]

@merickso thank you for reporting. It seems to be related to #1643 provided by @traylenator

[traylenator]

See you added #1663 to make a configuration. Was tested against a random jira server - issues.puppetlabs.com but guess older ones have problems.

[adehad]

@traylenator , could you confirm whether we would get an import error if we tried to import the legacy sha? I merged a PR regarding a config option, but I want to try and make the default behaviour more seamless for users, eg perhaps we can catch the ImportError or JiraError

[traylenator]

Hi @adehad
I should have included that in the orignal report:

During package of python-jira an attempt is made to import all modules which is guarenteed to work only against packaged dependencies. Error with 3.5.0 is:

+ /usr/bin/python3 -s /usr/lib/rpm/redhat/import_all_modules.py -f /builddir/build/BUILD/python-jira-3.5.0-2.el9.x86_64-pyproject-modules
Check import: jira
Check import: jira.client
Check import: jira.config
Check import: jira.exceptions
Check import: jira.jirashell
Traceback (most recent call last):
  File "/usr/lib/rpm/redhat/import_all_modules.py", line 171, in <module>
    main()
  File "/usr/lib/rpm/redhat/import_all_modules.py", line 167, in main
    import_modules(modules)
  File "/usr/lib/rpm/redhat/import_all_modules.py", line 100, in import_modules
    importlib.import_module(module)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/builddir/build/BUILDROOT/python-jira-3.5.0-2.el9.x86_64/usr/lib/python3.9/site-packages/jira/jirashell.py", line 17, in <module>
    from oauthlib.oauth1 import SIGNATURE_RSA
ImportError: cannot import name 'SIGNATURE_RSA' from 'oauthlib.oauth1' (/usr/lib/python3.9/site-packages/oauthlib/oauth1/__init__.py)

[traylenator]

In case of confusion 3.5.0-2 is 3.5.0 + patch that was added to 3.5.1. Above error though is without that patch so pure 3.5.0.

(will release a new EPEL9 package once this settles down.)

[adehad]

Excellent, thanks for this.

@traylenator and @merickso (and anyone else!) could I request you give pip install git+https://github.com/pycontribs/jira.git@bugfix/default_oauth1_sha a go and see if this is a suitable compromise between safety and user experience?

[jefft]

I am experiencing the same problem, authenticating against an old Jira 7.11.0.

@adehad, I tried your patch, and it didn't help. The patch tries SIGNATURE_HMAC_SHA1, then falling back to SIGNATURE_RSA_SHA1. Shouldn't it rather fall back to SIGNATURE_RSA, which 3.5.0 uses? If I alter your patch to import and fall back to SIGNATURE_RSA, it works for me.

@adehad thanks, your patch does work for me, but only after I upgraded my oauthlib library.

The reason is that oauthlib-3.1.0 defined:

SIGNATURE_RSA = "RSA-SHA1"

and then by oauthlib-3.2.2 (currently the latest) it was renamed:

SIGNATURE_RSA_SHA1 = "RSA-SHA1"
SIGNATURE_RSA = SIGNATURE_RSA_SHA1  # deprecated variable for RSA-SHA1

Would you consider tweaking your patch to use the earlier SIGNATURE_RSA name? Then the patch will work with old and new oauthlib.

[adehad]

Of course, no problem at all. I have updated the patch, if you have the chance to give it a try that would be great, I'll then include it in a patch release

[jefft]

Thanks, I updated to the latest patch (gfc6ffd7) and it successfully falls back even with oauthlib-1.1.0.

The code does now generate an error stacktrace on first auth attempt, which is ugly and alarming (leaves me thinking: did something silently fail??). What do you think about emitting a warning instead (Edit: a _logging.warning omits the stacktrace and wrapped JIRAError, which is needed if the error is genuine) adding a warning log after the fallback error, advising the user to make 'signature_method' explicit to make the error go away.

A diff against your patched code:

--- /tmp/clean-client.py        2023-06-27 13:50:46.828564162 +1000
+++ ./venv/lib/python3.10/site-packages/jira/client.py  2023-06-27 14:10:34.468003736 +1000
@@ -3716,6 +3716,8 @@
                 _logging.exception(f"Failed to create OAuth session with {sha_type}")
                 if sha_type is FALLBACK_SHA:
                     raise  # We have exhausted our options, bubble up exception
+                else:
+                    _logging.warning(f"Trying fallback {FALLBACK_SHA}. Note: if fallback succeeds, consider adding 'signature_method': '{FALLBACK_SHA}' to oauth options to avoid the previous error.")
 
     def _create_kerberos_session(
         self,

and in full:

          for sha_type in (oauth.get("signature_method"), DEFAULT_SHA, FALLBACK_SHA):
              if sha_type is None:
                  continue
              oauth_instance = OAuth1(
                  oauth["consumer_key"],
                  rsa_key=oauth["key_cert"],
                  signature_method=sha_type,
                  resource_owner_key=oauth["access_token"],
                  resource_owner_secret=oauth["access_token_secret"],
              )
              self._session.auth = oauth_instance
              try:
                  self.myself()
                  return  # successful response, return with happy session
              except JIRAError:
                  _logging.exception(f"Failed to create OAuth session with {sha_type}")
                  if sha_type is FALLBACK_SHA:
                      raise  # We have exhausted our options, bubble up exception
                  else:
                      _logging.warning(f"Trying fallback {FALLBACK_SHA}. Note: if fallback succeeds, consider adding 's
  

[adehad]

Thanks for this feedback, I want to avoid too much conditional logic so I've made the debug logs a bit chatty so someone can temporarily use a more verbose log level to pick up what signature_method succeeded and then go from there.

I also aimed to address the issue about a silent failure by highlighting in the log that we are retrying.

[jefft]

Works nicely, thanks.