Can't fetch password from system

[rgardam]

I am seeing this error on Yosemite (10.10.5 (14F27))

keyring.backends._OS_X_API.Error: (-25293, "Can't fetch password from system")

edit: it works if I enable all applications to use the password from the keychain.

[wlbentley]

Exactly the same error here on El Capitan (10.11.4). Python 2.7 running in a virtualenv.

What does "enable all applications to use the password from the keychain" mean (from @rgardam comment)?

Config root and data root are both reported as /Users/_myusername_/.local/share/python_keyring, but after setting a password with keyring set ..., there is no change to this directory, and no files are created by keyring.

Any suggestions?

[Edit:] When I tried running keyring outside the virtualenv, it works as expected. This is a virtualenv issue.

However, the config and data roots are still being reported as above, and still contain nothing from keyring.

[jaraco]

Keyring doesn't store any passwords in the config or data root for the OS X backend, but instead stores the passwords in the default keychain. After adding a password, you should see changes in the login keychain when running the Keychain Access application.

As for the error message reported or the issues with virtualenv, I'm unable to replicate them.

Last login: Wed May  4 12:41:40 on ttys002
$ python -m virtualenv --version
15.0.1
$ python -m virtualenv --python python2.7 issue219
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in /Users/jaraco/issue219/bin/python
Installing setuptools, pip, wheel...done.
$ issue219/bin/pip install -q -U pip
$ issue219/bin/pip install keyring
Collecting keyring
  Downloading keyring-9.0-py2.py3-none-any.whl
Installing collected packages: keyring
Successfully installed keyring-9.0
$ issue219/bin/keyring set 'test issue 219' 'test user 219'                
Password for 'test user 219' in 'test issue 219': 
$ issue219/bin/keyring get 'test issue 219' 'test user 219'
some password
$ issue219/bin/keyring del 'test issue 219' 'test user 219'
Deleting password for 'test user 219' in 'test issue 219': 

Before deleting the password, I could find it in Keychain Access.

I did not have to do anything special like /enable all applications to use the password from the keychain/.

I'm using OS X El Capitan and Keyring 9.0 on Python 2.7.10.

Can you provide any additional information on what's different about your environment that's different from mine?

[jaraco]

Until I have more information, I don't know what else I can do to help resolve the issue. I do have a clean Mac OS X VM, so if there are steps I can take in that VM to replicate the issue, I'm willing to try. I'm closing the issue for now, but I'm happy to re-open it as new information comes to light.

[n8henrie]

I am getting this error with Python2.7 when using a virtualenv (tox). It works okay with Python2.7 if the venv is deactivated.

I am not getting the error in Python3.4 or Python3.5 in the venv (again, tox) -- they seem to work okay. Strange.

EDIT: Also on OS X 10.11(.5)

EDIT: For each password request, I'm seeing in Console.app (no such message for python3):

2016-05-27 10:17:31.000 PM kernel[0]: CODE SIGNING: cs_invalid_page(0x10454c000): p=52336[python2.7] final status 0x2000000, allowing (remove VALID) page

EDIT: I'm getting it with my pyenv installed 2.7.11:

mkdir tmp
cd tmp
virtualenv -p python2 venv
venv/bin/pip install keyring
venv/bin/python2 -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))'

As well as OS X's built-in 2.7.10:

mkdir tmp
cd tmp
virtualenv -p /usr/bin/python venv
venv/bin/pip install keyring
venv/bin/python -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))'

But running fine with my pyenv 3.5.1:

mkdir tmp
cd tmp
virtualenv -p python3 venv
venv/bin/pip install keyring
venv/bin/python -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))'

[jeremydw]

Data point: this is affecting our program too (https://github.com/grow/grow) and I can confirm it occurs when run in a virtualenv. I can also confirm that downgrading keyring to 8.7 seems to have fixed the issue.

Here's the relevant traceback:

  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/oauth2client/keyring_storage.py", line 84, in locked_get
    content = keyring.get_password(self._service_name, self._user_name)
  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/keyring/core.py", line 42, in get_password
    return _keyring_backend.get_password(service_name, username)
  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/keyring/backends/OS_X.py", line 44, in get_password
    return api.find_generic_password(self.keychain, service, username)
  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 92, in find_generic_password
    NotFound.raise_for_status(status, msg)
  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 43, in raise_for_status
    Error.raise_for_status(status, msg)
  File "/Users/jeremydw/git/pygrow/env/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 35, in raise_for_status
    raise cls(status, msg)
keyring.backends._OS_X_API.Error: (-25293, "Can't fetch password from system")

[jaraco]

Sorry, but I'm still unable to replicate the issue. I created a fresh install of OS X in a virtual machine. Python didn't even have pip installed. This machine is stock. Yet, when I install all the prereqs, keyring works as advertised without any issue:

Jasons-MacBook-Pro:~ jaraco$ sudo python2.7 -m ensurepip

WARNING: Improper use of the sudo command could lead to data loss
or the deletion of important system files. Please double-check your
typing when using sudo. Type "man sudo" for more information.

To proceed, enter your password, or type Ctrl-C to abort.

Password:
The directory '/Users/jaraco/Library/Logs/pip' or its parent directory is not owned by the current user and the debug log has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Ignoring indexes: https://pypi.python.org/simple
The directory '/Users/jaraco/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): setuptools in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
Collecting pip
Installing collected packages: pip
Successfully installed pip-6.1.1
Jasons-MacBook-Pro:~ jaraco$ python2.7 -m pip install --user virtualenv
You are using pip version 6.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting virtualenv
  Downloading virtualenv-15.0.2-py2.py3-none-any.whl (1.8MB)
    100% |████████████████████████████████| 1.8MB 263kB/s 
Installing collected packages: virtualenv
Successfully installed virtualenv
Jasons-MacBook-Pro:~ jaraco$ python2.7 -m virtualenv issue219
New python executable in /Users/jaraco/issue219/bin/python
Installing setuptools, pip, wheel...done.
Jasons-MacBook-Pro:~ jaraco$ issue219/bin/pip install keyring
Collecting keyring
  Downloading keyring-9.0-py2.py3-none-any.whl
Installing collected packages: keyring
Successfully installed keyring-9.0
Jasons-MacBook-Pro:~ jaraco$ issue219/bin/keyring set "Some Service" "Some Account"
Password for 'Some Account' in 'Some Service': 
Jasons-MacBook-Pro:~ jaraco$ issue219/bin/keyring get "Some Service" "Some Account"
some password

It sure seems like whatever is triggering this API error is not universal (and is specific to only some users' systems). Perhaps there's a security setting enabled or a legacy settings lingering from an older build of the OS. I still am aware of no actions that keyring can take to work around the issue, short of bringing back the command-based invocation. I'd consider that, if it's a separate package and not in the default implementation.

I'd prefer, though, if we could understand the underlying cause of the error and correct for it, so any details you might be able to provide as to why your systems are failing and a stock system is not would be most helpful in advancing this issue.

[n8henrie]

Slight update: If I request a password that doesn't exist (by changing a letter of either the service or account), I get None. Only getting the error if requesting an existing password.

[jeremydw]

@jaraco thanks for working through this with us. I just reproduced the error inside virtualenv following your commands exactly (minus the initial ensurepip). See below. Let me know if there's anything else you'd like me to try.

Here's my OS version, if it helps:
Mac OS X El Capitan
10.11.4 (15E65)

$ virtualenv --version
15.0.2

~$ python2.7 -m virtualenv issue219
New python executable in issue219/bin/python
Installing setuptools, pip, wheel...done.
~$ issue219/bin/pip install keyring
Collecting keyring
  Using cached keyring-9.0-py2.py3-none-any.whl
Installing collected packages: keyring
Successfully installed keyring-9.0
You are using pip version 7.1.2, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
~$ issue219/bin/keyring set "Some Service" "Some Account"
Password for 'Some Account' in 'Some Service': 
~$ issue219/bin/keyring get "Some Service" "Some Account"
Traceback (most recent call last):
  File "issue219/bin/keyring", line 11, in <module>
    sys.exit(main())
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/cli.py", line 101, in main
    return cli.run(argv)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/cli.py", line 54, in run
    password = get_password(service, username)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/core.py", line 42, in get_password
    return _keyring_backend.get_password(service_name, username)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/backends/OS_X.py", line 44, in get_password
    return api.find_generic_password(self.keychain, service, username)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 92, in find_generic_password
    NotFound.raise_for_status(status, msg)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 43, in raise_for_status
    Error.raise_for_status(status, msg)
  File "/Users/jeremydw/issue219/lib/python2.7/site-packages/keyring/backends/_OS_X_API.py", line 35, in raise_for_status
    raise cls(status, msg)
keyring.backends._OS_X_API.Error: (-25293, "Can't fetch password from system")
~$ 

[jeremydw]

Worked with keyring==8.7, same system and sequence of commands:

~$ python2.7 -m virtualenv issue219-8.7
New python executable in issue219-8.7/bin/python
Installing setuptools, pip, wheel...done.
~$ issue219-8.7/bin/pip install keyring==8.7
Collecting keyring==8.7
  Using cached keyring-8.7-py2.py3-none-any.whl
Installing collected packages: keyring
Successfully installed keyring-8.7
You are using pip version 7.1.2, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
~$ issue219-8.7/bin/keyring set "Some Service" "Some Account"
Password for 'Some Account' in 'Some Service': 
~$ issue219-8.7/bin/keyring get "Some Service" "Some Account"
foo
~$ 

[jialongl]

I get this error too after I upgraded to OS X 10.11.5 today, which may or may not have to do with the error. I am not sure.

Quoting @n8henrie :

If I request a password that doesn't exist (by changing a letter of either the service or account), I get None. Only getting the error if requesting an existing password.

And I will not get an error (exception) if I "allow all application to access this item" in OS X Keychain Access.app

So I guess it is a permission thing. That is to say we need to find the correct application to allow access to the item.

Unfortunately, none of those worked...
But hopefully this sheds some light.

[jialongl]

I second @jeremydw . Version 8.7 works.

Version 8.7 invokes the shell command security find-generic-password -g -a <username> -s <service> to get the password (in backends/OS_X.py), while version 9.0 gets the password by loading the system's shared library /System/Library/Frameworks/Security.framework/Versions/A/Security (in backends/_OS_X_API.py) and making a foreign function call (C, I guess).

[bobzilladev]

I ran into this as well on 10.11.5 and keyring 9.0, the system's python.app would pop up the dialog asking for permission for the keyring item correctly, but a python in a virtualenv would not. Adding the allow-access for the virtualenv python also did not help. I can set new keys with the virtualenv binary but not read them back out. Moving to a 'security' exec'ing version works as expected, in this case 5.4.

[n8henrie]

I'm still working to get this figured out.

To that end, I've made a test keychain credential keyring_test, with user: foo and pass: bar.

I'm still getting error 25293 / errSecAuthFailed under Python 3.5.2, but I'm able to retrieve the test password under 2.7.12.

I've simplified the new ctypes OSX API to the following test script:

import ctypes

sec_path = "/System/Library/Frameworks/Security.framework/Versions/A/Security"
kc = ctypes.CDLL(sec_path).SecKeychainFindGenericPassword

kc.argtypes = (
    ctypes.c_void_p,
    ctypes.c_uint32,
    ctypes.c_char_p,
    ctypes.c_uint32,
    ctypes.c_char_p,
    ctypes.POINTER(ctypes.c_uint32),  # passwordLength
    ctypes.POINTER(ctypes.c_void_p),  # passwordData
    ctypes.POINTER(ctypes.c_void_p),  # itemRef
)

length = ctypes.c_uint32()
data = ctypes.c_void_p()

kc(None, len("keyring_test"), "keyring_test".encode("utf-8"), len("foo"),
        "foo".encode("utf-8"), length, data, None,)

password = ctypes.create_string_buffer(length.value)
ctypes.memmove(password, data.value, length.value)

print(password.raw.decode("utf-8"))

Interestingly, I found that I do get the popup to "allow" and subsequently can access it under the following Python versions (with keyring 9.3.1):

1. python 2.7.12, installed by homebrew

2. python 2.7.9, installed by pyenv

3. either of the above python2, in a virtualenv

4. python 3.5.1, installed by homebrew (real path Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/bin/python3.5)

5. python 3.5.2, installed by pyenv

6. 3.5.2 from homebrew, in a virtualenv created by virtualenv

   I am not able to get it to work (get the error from above) in the following settings:

7. python 3.5.2, installed by homebrew (real path Cellar/python3/3.5.2/Frameworks/Python.framework/Versions/3.5/bin/python3.5)

8. python 3.5.2 from homebrew, in a virtualenv created by pyvenv using --symlinks

9. python 3.5.2 from homebrew, in a virtualenv created by pyvenv using --copies

---

This is really confusing at this point. Specifically,

• I'm surprised that #4 works whereas #7 doesn't (both homebrew installed, similar path with frameworks, the python3 formula doesn't seem to have crazy changes in the interim)
• I'm surprised that #6 works while #9 doesn't (at first I thought it was because pyvenv uses symlinks by default, but the --copies flag didn't make a difference)

It clearly doesn't seem to be a problem with Python, since 3.5.2 from pyenv works, as well as 3.5.2 from homebrew in a virtualenv installed virtualenv (but not a pyvenv installed virtualenv).

Any other ideas?

[jaraco]

Perhaps there's something about the security settings on the Python 3.5.2 installed by homebrew, something that might be getting copied to the pyvenv.

That's quite interesting, though. It sounds like you're on to something, and this really gives us an opportunity to perhaps reproduce the issue, so I'll give it another go.

[jaraco]

I've installed Python on homebrew:

brew install python3

And added the credential as you did:

security add-generic-password -p bar -a foo -s keyring_test

I then created a virtualenv.

python3 -m venv env

I then saved the script you created as test-keychain.py and ran it with success:

$ env/bin/python test-keychain.py
bar

It did prompt me to Always Allow, Allow, or Deny the request, and I clicked allow.

I wiped and recreated env with --symlinks and --copies with the same results. I also tried with a raw homebrew installation, still with success.

$ python3 test-keychain.py
bar

So it seems that there's still something unique about your environment that I can't replicate.

I haven't done anything special with this machine - it's the same stock machine I referenced above, and I installed homebrew from https://brew.sh today with no special steps.

May I suggest you try the same? Create an OS X virtual machine using Virtualbox and an installer image. Then, see if you can replicate the failure in that clean environment. I suspect you cannot, at which point you might then compare the security settings and file system permissions against that machine and the failing system to perhaps suss out the cause.

[jialongl]

I give up finding out why somehow my environment (the python that comes along with Xcode, I guess) complains Can't fetch password from system --- I bet it has to do with my screwed-up system configurations.

However, I have got a work-around lousy hack. Hope it helps the late comers:

• Go to https://www.python.org/downloads/ and download python-2.7.12-macosx10.6.pkg
• Install it. It should have installed two directories:
  • /Applications/Python2.7
  • /Library/Frameworks/Python.framework/
• Backup the current installation folder, whatever that is. In my case, I just gave it another name: mv /System/Library/Frameworks/Python.framework/Versions/2.7{,.10}
• mv /Library/Frameworks/Python.framework/Versions/2.7 /System/Library/Frameworks/Python.framework/Versions
• (Optional) Remove the Python2.7 app from /Applications

... and yes, it works:

$ /usr/bin/python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> sys.executable
'/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python'
>>> import keyring
>>> keyring.__version__
'9.3.1'
>>> keyring.set_password('keyring_test', 'yum', 'yummy')
>>> keyring.get_password('keyring_test', 'yum')
u'yummy'

My guess is that Keychain Access.app looks at apps' Info.plist to keep track of which apps have access to which items --- what I did above effectively changed /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/Info.plist.

identifies and distinguishes apps by "runtime libraries" --- the output of otool -L /path/to/executable, for example:

$ otool -L /System/Library/Frameworks/Python.framework/Versions/2.7.10/Resources/Python.app/Contents/MacOS/Python
/System/Library/Frameworks/Python.framework/Versions/2.7.10/Resources/Python.app/Contents/MacOS/Python:
    /System/Library/Frameworks/Python.framework/Versions/2.7.10/Python (compatibility version 2.7.0, current version 2.7.10)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.12.0)

My hack (and virtual environments, as mentioned above) works because I effectively used another runtime library to get passwords from Keychain Access.app (after clicking "Allow" or "Always Allow").

So, all in all, IMHO this is not a bug per se, but as a result of (unintendedly) having messed up system-wise configurations and OS X no longer pops the "Do you want xxx to access your keychains?" dialog.

[jaraco]

Thanks @jialongl . Sorry to hear you didn't have a proper fix, but you seem to have gotten awfully close to the underlying cause. If anyone else is able to trace a solution to the underlying problem, perhaps by clearing a .properties file or similar, do please report back here. It would be great to be able to provide a better error message.

In the meantime, I don't see any pending action, so I'm closing this ticket, but anyone is welcome to revive the conversation.

[n8henrie]

I got access to another Macbook Air on El Capitan and installed 3.5.2 with homebrew -- keyring worked appropriately with or without a venv, no issues. So it doesn't look like a homebrew problem necessarily either.

On my personal MBA I've tried uninstalling, cleaning the homebrew cache, updating to the latest formula, and reinstalling, and still no dice.

^^ Sorry, thought I had submitted that a few days ago before I left on a little camping trip, came back to find it still in the comment box, unsubmitted.

Sounds fair, I'll come back to the thread if I ever figure this out. Thanks for the consideration and recommendations.

[n8henrie]

It's also working for me with a fresh Virtualbox installation of homebrew's 3.5.2. Still trying to figure out what's going on in my system (I have a number of python3 scripts that rely on keyring, which is why I'm being so persistent about this).

I found that homebrew was leaving /usr/local/lib/python3.5 and /usr/local/opt/python3 on my system after uninstalling, but removing them didn't change anything.

Selecting the option to allow access to all apps does let keyring access the password. Once it's changed back, it's back to the same error.

I created a new keychain ~/Library/Keychains/asdf.keychain and added some test credentials -- same error, even on a brand new keychain. I moved that keychain to my Virtualbox environment, and keyring was able to access it there.

So looks my problem isn't intrinsic to the keychain, either.

[n8henrie]

Something seems to have fixed it in in Homebrew/homebrew-core@84897ff .

[tommyblue]

I use python 2.7.10 (provided with mac) and the problem persists

[jialongl]

@tommyblue If you get the error message keyring.backends._OS_X_API.Error: (-25293, "Can't fetch password from system") (Note the error code). -25293 is the same error code if you are prompted the dialog:

Python wants to use your confidential information stored in "KEYRING_NAME" in your keychain The authenticity of "Python" cannot be verified. Do you want to allow access to this item?

and clicked Deny

Thus, my point is, your system is somehow configured to deny this request automatically for you (though without your awareness / intention).
Maybe you could try my hack (and use newer python :p ).

[maludwig]

I also got this problem with a fresh install of keyring on High Sierra today. Only in the venv.

...
keyring.errors.KeyringError: Can't get password from keychain: (-67030, 'Security Auth Failure: make sure python is signed with codesign util')

So I tried checking out the signature of the venv python:

$ codesign -v `which python`
/Users/mitchell.ludwig/.virtualenvs/builings/bin/python: invalid Info.plist (plist or signature have been modified)
In architecture: x86_64

Re-signing it fixed it:

$ codesign -f -s - `which python`
/Users/mitchell.ludwig/.virtualenvs/builings/bin/python: replacing existing signature

[jaraco]

I encountered this issue today. At one moment, keyring was working just fine. Less than two hours later, it spontaneously started failing with keyring.errors.KeyringError: Can't get password from keychain: (-25293, 'Security Auth Failure: make sure python is signed with codesign util'). Looking at the binary, I could see it was unsigned:

~ $ codesign -v $(which python3.7)                                                                                                      
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7: code object is not signed at all
In architecture: x86_64

I was able to add a signature for it:

~ $ codesign -f -s - $(which python3.7)                                                                                                 
~ $ codesign -v $(which python3.7)                                                                                                      
~ $ codesign -v $(which python3)                                                                                                        
~ $ codesign -v ~/.local/pipx/venvs/xonsh/bin/python

Yet somehow still the keyring continues to fail with the same error.

[rohithrpai]

@jaraco Do you intend to investigate this issue?

[karanmagdani]

I was facing the same issue on macOS Mojave after changing password, I fixed it by doing by following this jdamata/k8vault#3 (comment)

Not sure why this fixed it though

[jaraco]

@rohithrpai: I did investigate, but what I found was that the problem went away on its own. I was working with my internal IT department as the issue was isolated to my work macbook. The behavior was very strange. I installed Python 3.8b4 and the keyring seemed to be working there, but not on 3.7.4. Here are some excerpts from my interaction then.

2019-10-07

Okay, this is bizarre. Today, I installed Python 3.7.5rc1 over 3.7.4 and I made one other change--for the foo/bar test credential, I with to Get Info and Access Control and configured it to "Allow all applications to access this item", and I was able to query it. I then went back to the original credential that was giving me trouble and I was able to access it without the error. It seems that mysteriously, whatever was causing the problem the week before last has gone away as spontaneously as it occurred.

2019-10-10

Even more bizarre, I'm finding now that the Python 3.8rc1 is no longer signed and is now being rejected by the keychain calls but Python 3.7 (still unsigned) is not rejected by the keychain.

2019-10-11

And as I encounter even more weirdness, I realize I'm simply out of my depth. After struggling for most of a day with the issue, I made several surprising findings:

• I tried adjusting the settings on the credential in question but to no avail.
• restarting, changing first-to-query (between Python 3.7 and Python 3.8) did not affect the rejected calls
• I reinstalled Python 3.8b4 (possibly over 3.8rc1 or maybe over 3.8b4), but that had no effect.
• The reinstalled Python 3.8b4 was not signed (confirmed with codesign -v $(which python3.8)).
• I wrote some additional code to extract the error message from the failed API call and the message was "bad username or password". May be a red-herring.
• I re-downloaded and installed Python 3.8rc1 over Python 3.8b4 and things started working again. And still, python3.8 is unsigned.

So I'm very confused. I don't understand what caused the access to stop nor why reinstalling a later Python allowed the access to be restored. I'm beginning to think there's some executable whitelist and that executable whitelist only includes a few select recent Python versions.

Status

I'm currently not experiencing the issue in any Python environment on any system, but when I was, the issue was elusive and inscrutable. The behavior would seemingly change without any explicit alteration to the system. I just don't know what else to do.

[jaraco]

Note, I continue to use keyring on my macOS systems without any signed Python executable.

[satyagv557]

@jaraco : This issue is happing on my mac,

I am using python 3.7, keyring==18.0.1, with same setup in other Mac this is working fine.
here is the log

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/OS_X.py", line 49, in get_password
    return api.find_generic_password(self.keychain, service, username)
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/_OS_X_API.py", line 103, in find_generic_password
    Error.raise_for_status(status)
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/_OS_X_API.py", line 50, in raise_for_status
    raise cls(status, "Unknown Error")
keyring.backends._OS_X_API.Error: (-25308, 'Unknown Error')

[lemoinem]

I've just hit that.
This started after install some Profile now required by our IT dept. This seems to be something related to Device Management. However, I couldn't see any king of settings that would restrict access to the keychain.

After removing the profile, I still had to delete my login keychain and then reset it (which would create a new login keychain). (Thankfully, I didn't have anything important in my keychain).

Now it's working again.

[jaraco]

Yesterday, this problem mysteriously and spontaneously re-emerged on my workstation. Steps I tried unsuccessfully:

• restart the system
• lock and unlock the keychain
• Check access control for the item and ensure it allows access (confirm before)
• Allow all applications to access the item

[jaraco]

I've confirmed that while keyring is failing under Python 3.8.1, it's succeeding under Python 3.7.5rc1, and neither are signed:

~ $ codesign -v $(which python3.8)                                                                                                      
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: code object is not signed at all
In architecture: x86_64
~ $ codesign -v $(which python3.7)                                                                                                      
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7: code object is not signed at all
In architecture: x86_64

[jaraco]

Since Python 3.7.5 was an RC, I decided to update it to 3.7.6. After upgrading it to 3.7.6, the issue now manifests in Python 3.7 as well, leading me to believe there's some setting in the system that's preventing new apps from being authorized to the keychain.

[jaraco]

I should have mentioned - this environment is using a keychain that's not the default keychain. It's using a separate file and setting KEYCHAIN_PATH to indicate to the keychain to use this non-default keychain.

I've also observed that when logging in, I was previously prompted to enter the password to unlock the keychain (for some macOS application), but macOS has not been prompting me there either. This leads me to believe that applications other than keyring are affected (and suppressing the error when querying the keyring).

[jaraco]

@jaraco : This issue is happing on my mac,

I am using python 3.7, keyring==18.0.1, with same setup in other Mac this is working fine.
here is the log

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/OS_X.py", line 49, in get_password
    return api.find_generic_password(self.keychain, service, username)
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/_OS_X_API.py", line 103, in find_generic_password
    Error.raise_for_status(status)
  File "/usr/local/lib/python3.7/site-packages/keyring/backends/_OS_X_API.py", line 50, in raise_for_status
    raise cls(status, "Unknown Error")
keyring.backends._OS_X_API.Error: (-25308, 'Unknown Error')

@satyagv557 The error you're getting is different from mine. I encountered a similar error (-25307, 'Unknown Error') when I attempted to use sudo to access the keychain.

[jaraco]

I've found some references to the -25293 error:

• Someone using Fetch
• Someone using SequelPro
• Someone coding direct to the API

[satyagv557]

Issue fixed, sometime even your keychain is unlocked, keyring is not able to grab the password, I did added a shell build in my pipeline to unlock in runtime, that did solve this issue, thanks for your comments and suggestions.

[jaraco]

A few other things I tried today to no avail:

• Copied the keychain file from another system where it's working fine.
• Deleted the zero-bytes file sitting alongside the keyring.
• Ensure no other Python scripts are running prior to querying keyring.
• Uninstall, clean, and reinstall Python 3.8.
• Restart the computer.

I did find that while Python 3.8 and 3.7 from python.org are currently failing, Python 3.7.6 from homebrew is able to query the keychain.

[jaraco]

So here's something interesting: I rebooted the system. After logging in, when fmfd and Reminders prompted for the password on this password file, instead of entering the password, I went to keyring to query the keychain. When I did, the Python routine blocked. So I entered the password for fmfd, then was prompted for the password for Reminders, then unexpectedly I was prompted for the password for Python.

After entering the password for that keychain, Python was unblocked but still emitted the -25293 Security Auth Failure.

[jaraco]

I extracted some "python" logs from the Console when the error occurs and here's what I see:

default	14:32:28.012794-0500	mDNSResponder	[R1977] DNSServiceCreateConnection START PID[2743](Python)
default	14:32:28.012909-0500	mDNSResponder	[R1978] DNSServiceQueryRecord(1D000, 0, <private>, Addr) START PID[2743](Python)
default	14:32:28.013488-0500	mDNSResponder	[R1979] DNSServiceQueryRecord(1D000, 0, <private>, AAAA) START PID[2743](Python)
default	14:32:28.014352-0500	mDNSResponder	[R1978] DNSServiceQueryRecord(1D000, 0, <private>, Addr) STOP PID[2743](Python)
default	14:32:28.014434-0500	mDNSResponder	[R1979] DNSServiceQueryRecord(1D000, 0, <private>, AAAA) STOP PID[2743](Python)
default	14:32:28.837054-0500	symptomsd	defusing ticker tickerFatal having seen progress by flow for org.python.python, rxbytes 4738 duration 0.819 seconds started at time: Fri Feb  7 14:32:28 2020
default	14:32:28.891334-0500	mDNSResponder	[R1977] DNSServiceCreateConnection STOP PID[2743](Python)
default	14:32:29.076834-0500	Python	UNIX error exception: 17
default	14:32:29.083203-0500	Python	UNIX error exception: 17
default	14:32:29.087646-0500	Python	UNIX error exception: 17
default	14:32:29.139852-0500	securityd	suppressing keychain prompt /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app(2745); code signing check failed rc=-67030
default	14:32:29.150214-0500	Python	CSSM Exception: -2147416032 CSSMERR_CSP_OPERATION_AUTH_DENIED
default	14:32:29.151314-0500	Python	CSSM Exception: -2147416032 CSSMERR_CSP_OPERATION_AUTH_DENIED
fault	14:32:49.655900-0500	python3	Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.
default	14:32:49.657445-0500	python3	Received configuration update from daemon (initial)
fault	14:33:49.656145-0500	python3	Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.
default	14:33:49.656501-0500	python3	Received configuration update from daemon (initial)

[jaraco]

Okay, so I've found something that seems to have worked:

~ $ codesign -f -s - /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app                                             
/Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app: replacing existing signature
~ $ python3.8 -m pip-run keyring -- -m keyring get 'GitLab MFA' jaraco > /dev/null
Collecting keyring
  Using cached keyring-21.1.0-py2.py3-none-any.whl (31 kB)
Installing collected packages: keyring
Successfully installed keyring-21.1.0
~ $

(no output as the error did not occur)

And yet the python executable is still unsigned.

~ $ codesign -v $(which python3.8)                                                                                                      
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: code object is not signed at all
In architecture: x86_64

It seems that $(which python) wasn't the right way for me to identify the offending Python binary that needed to be signed.

What I really don't understand is why these binaries are apparently signed but not with a valid signature and only sometimes.

[abohne]

We had a user receive keyring.backends._OS_X_API.SecAuthFailure: (-25293, 'Security Auth Failure: make sure python is signed with codesign util')

The application is using python 3.7 installed from homebrew.

Locking and unlocking the login keychain solved the issue. This leads me to believe the codesigning error is a complete red herring.

[rxvt]

Just some more info that might help, I've discovered that if there is no entry in MacOS Keychain for my app then I do not receive the make sure python is signed with codesign util error, however the next time I run the app the error will appear. If I then remove the entry in Keychain the app runs again successfully. No idea why currently.

Update: Discovered that forcing a resign of the PyInstaller created executable fixes the Keychain codesign issue: codesign -f -s <identity> <executable>

[jellespijker]

It isn't immediately clear to me if this issue has been fixed. But we are still experiencing this crash with keyring version 23.0.1

We have included your keyring library in Cura 4.9.0 (https://github.com/Ultimaker/Cura/blob/master/cura/OAuth2/KeyringAttribute.py) to store our auth tokens. But now that Cura 4.9.0 is released in the wild, I noticed multiple crashes coming in on our Sentry server. With users that are experiencing this crash: Can't get password from keychain: (-25293, 'Security Auth Failure: make sure python is signed with codesign util')

This only on occurs for the Mac's Darwin OS versions: 17.7.0, 18.7.0, 19.6.0, 20.2.0, 20.3.0, 20.4.0
We use Python version 3.8 with cx_freeze v6.5.3 and the Darwin image is signed by Apple.

[jaraco]

@jellespijker From keyring's perspective, there's little that can be done. The keyring library just invokes Apple's APIs under whatever security regime that application (be it Python or some bundler like cx_freeze or pyinstaller) was built. Apple then sometimes rejects requests with opaque error codes.

Of one thing you can be pretty confident, if you see that error message, then Apple rejected the security API call with one of two codes:

keyring/keyring/backends/macOS/api.py

Lines 42 to 47 in bfa0148

	if status == error.sec_auth_failed or status == error.plist_missing:
	raise SecAuthFailure(
	status,
	"Security Auth Failure: make sure "
	"python is signed with codesign util",
	)

The best recommendation I can make is to ensure the executable image is signed and if you've already done that, maybe consider breaking down the problem into smaller pieces:

• Can you replicate the issue locally?
• Does the error occur using Cura outside cx_freeze (using an installed Python)?
• Does the issue occur in a minimal cx_freeze app that only invokes keyring?

To be sure, though, I use macOS daily and haven't encountered this issue in a long time, so I consider it solved. If you have concerns that involve new and different factors, please consider distilling the concerns and then opening a new issue.

[jellespijker]

@jaraco thanks for the reply. Our MacOS executable is signed by Apple. Our developers and testers did not encounter the issue running from source or image. Luckily there aren't any issues reported by our users on GitHub. Just multiple Sentry crash reports. Which could be an indication that it is a single, non-recuring event for an individual user.

We will keep an eye out and if we find the root-cause report it here.

@evtrados and @Vandresc FYI

[Ghostkeeper]

Here is a stack trace, although it probably doesn't give much more information. A bunch of it is filtered out as personally identifiable information of our users: https://sentry.io/share/issue/cadfe0c6dee14b5d8e5dd119e22407a8/

It's occurred for several hundred people, but not for any of our developers or testers so I think it's safe to say that it's computer-dependent. On average each user has encountered it 3-4 times as far as we can see (we can't identify individual users) so it's recurring for them too.

We can't reproduce the issue locally, so we can't verify the other two questions. It really seems to be depending on the configuration on the computer. A configuration that we don't have. We're seeing it in all MacOS versions Cura supports (at least most of them from what I can see) so it's not a specific MacOS version. It must be something else in the configuration.

Our application is signed by Apple with an enterprise account, and also notarized (which unpacks and repacks the binaries).

[jaraco]

That's consistent with what I've seen in some situations too - that the behavior is system-dependent and somewhat transient. One thing I'd like to do is provide more context in the error so that it's apparent which error code is actually returned.

[jaraco]

That's consistent with what I've seen in some situations too - that the behavior is system-dependent and somewhat transient. One thing I'd like to do is provide more context in the error so that it's apparent which error code is actually returned.

Oh, I see the actual code is already included in the error message, and it's -25293 (sec_auth_failed).

So unfortunately, that's the full context that Apple gives us in this situation. Perhaps you could reach out to Apple (or its community) to see if they can shed any light onto what factors might cause this error or what more diagnoses could be performed to gather more insight. Perhaps the systems in question somehow have corrupt signing keys (so even though the apps are properly signed, they appear unsigned on those systems).