Enables using requests
without having to configure the CERN Root certificates or getting an API access token manually.
Inspired by certifi
, requests-kerberos
, cern-sso-python
and api-access-examples
.
The Root certificate bundle is copied from the linuxsoft cern page and can also be created manually by downloading the CERN Grid Certification Authority files from cafiles.cern.ch/cafiles.
Warning We are no longer supporting Python 2.7.
Requires Python 3.6+.
pip install cernrequests
Request a Grid User Certificate (with password) and convert into public and private key:
mkdir -p ~/private
openssl pkcs12 -in myCertificate.p12 -clcerts -nokeys -out ~/private/usercert.pem # Will ask for the certificate password
openssl pkcs12 -in myCertificate.p12 -nocerts -nodes -out ~/private/userkey.pem # Will ask for the certificate password
The .pem
certificates have to be passwordless.
You will need to have a plaintext file named .env
at the same directory where you are running your script from, with the following contents:
SSO_CLIENT_ID=<your client id>
SSO_CLIENT_SECRET=<your client secret>
(You can use the .env_sample
found in this repository as an example).
Warning Automatic PaaS registrations (i.e. those whose id starts with
webframeworks-paas-
) will not work; You will have to create a new one. Thanks to Tongguang Cheng for his report.
To request a client id and a client secret, you will need to register your application at CERN's Application Portal:
-
Create an SSO registration for your application on the CERN Application Portal:
-
Add an application identifier and description:
The
Application Identifier
can be anything, it's like a username for your application. It has to be unique, so, if the name you chose is rejected, try another one. The description can be anything, and it doesn't affect your registration. ClickSubmit
. -
Go back to the Application Portal and edit the SSO application (green button). Then, go to the
SSO Registration
tab and click the plus button: -
Fill out the form of the new SSO registration as follows:
- You can put any value in the
Redirect URI(s)
, e.g.http://localhost/*
- Same for the
Base URL
- Make sure you click
My application will need to get tokens using its own client ID and secret
.
- You can put any value in the
-
Submit the form:
Note the
client id
andclient secret
that the form will show you.
import cernrequests
url = "https://<your-cern-website>"
response = cernrequests.get(url)
If you want to access a website which requires a (""new"") CERN Single Sign-on token you can do the following:
import cernrequests
url = "https://<your-cern-website-url>"
reponse = cernrequests.get_with_token(url, target_audience="<the SSO id of the target URL>")
Note The
target_audience
depends on the SSO registration name of the target application. E.g. if you want to access the development instance of Run Registry,target_audience
should bedev-cmsrunregistry-sso-proxy
. In case of doubt, communicate with the app's developers directly.
The get_with_token
method also accepts an api_token
argument, in case you want to reuse an already issued and non-expired one that you got from get_api_token
.
This is a method that needs a target_application
parameter and will try and get an API token, using the SSO_CLIENT_ID
and SSO_CLIENT_SECRET
provided as environment variables.
A tuple
is returned: api_token
(str
) and the expiration_datetime
(datetime
).
Note The
expiration_datetime
is not used internally, so it's returned to the user, if they need to check when the token expires.
If you want to use requests
directly without the CERN wrapper you can get the exact same functionality by doing:
import requests
from cernrequests import certs
url = "https://<your-cern-website>"
cert = certs.default_user_certificate_paths()
ca_bundle = certs.where()
response = requests.get(url, cert=cert, verify=ca_bundle)
The default user certificate paths are first ~\private\
and ~\.globus\
for fallback. The default public key file name is usercert.pem
and the default private key file name is userkey.pem
You can configure the default grid user certificate path by setting the CERN_CERTIFICATE_PATH
environment variable.
For example:
export CERN_CERTIFICATE_PATH=${HOME}/my_custom_folder
This will still assume that your filenames are usercert.pem
and userkey.pem
Write this line in your .bashrc
to make the configuration persistent.
Alternatively you can also specify the paths directly in your code:
import cernrequests
url = "https://<your-cern-website>"
cert = "my/custom/path/cert.pem" # Public key path
key = "my/custom/path/key.pem" # Private key path
cernrequests.get(url, cert=(cert,key))
This way you can even use custom names such as cert.pem
and key.pem
python -m venv venv
source venv/bin/activate
pip install -e .
pip install -r testing-requirements.txt
pytest
To run the tests on GitHub, we have uploaded a set of userkey.pem
and usercert.pem
files, encrypted with GPG. Those are decrypted, using a secret and then used normally.
See the yaml file
for more details.
Note If the actions are failing, make sure that the Grid Certificates are still valid. You may need to recreate them. For instructions, see here. Create new grid certificates, encrypt them with a password and replace
tests/usercert.pem.gpg
andtests/userkey.pem.gpg
. You will also need to update theGPG_ENC_PASSWORD
secret with the password you used to encrypt them.
The cernrequests/cern-cacerts.pem
file has expired, and will need to be updated by the library maintainer.
-
This will create a
git clone https://gitlab.cern.ch/linuxsupport/rpms/cern-ca-certs/ cd cern-ca-certs/src make
CERN-bundle.pem
file. - Rename it to
cern-cacerts.pem
and replace the original.pem
certificate chain.
Verify that the certs work by running pytest
.
I'm getting 403 Client Error: Forbidden for url: https://login.cern.ch/adfs/ls/auth/sslclient
errors!1 What should I do?
- Your grid certificate may have expired. Try creating a new one.
- You may be trying to access a CERN webpage using a grid certificate, but this method may be deprecated. Make sure that the web page allows SSL certificate authentication.
- http://docs.python-requests.org/en/master/
- https://certifi.io/en/latest/
- https://github.com/cerndb/cern-sso-python
- https://linux.web.cern.ch/linux/docs/cernssocookie.shtml
- http://linuxsoft.cern.ch/cern/centos/7/cern/x86_64/repoview/CERN-CA-certs.html
- https://ca.cern.ch/ca/
- https://auth.docs.cern.ch