API request to dbpedia gives [SSL: CERTIFICATE_VERIFY_FAILED]

[neostrange]

Hi,

I have utilized the spacy-dbpedia-spotlight plugin without any issues for a considerable period of time. However, my code encountered an error yesterday.

2023-03-08 11:44:56.603 | ERROR    | spacy_dbpedia_spotlight.entity_linker:get_remote_response:245 - Endpoint unreachable, please check your connection. Document not updated.
                HTTPSConnectionPool(host='api.dbpedia-spotlight.org', port=443): Max retries exceeded with url: /en/annotate (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)')))

Can you please look into this why it suddenly start appearing?

I am running spacy in a python venv virtual environment in ubuntu.

[MartinoMensio]

Hi @neostrange,
The problem is on the DBpedia end. They have an expired certificate from 8 days. You can (temporarily) ignore certificate verification by editing the following lines and adding verify=False:

spacy-dbpedia-spotlight/spacy_dbpedia_spotlight/entity_linker.py

Lines 218 to 219 in 5c5802c

	return requests.post(
	f'{endpoint}/{self.process}', headers={'accept': 'application/json'}, data=params)

        return requests.post(
            f'{endpoint}/{self.process}', headers={'accept': 'application/json'}, verify=False, data=params)

I could an option in the configuration of the pipeline to allow ignoring verification more easily.

Best,
Martino

[MartinoMensio]

You can now install the new release v0.2.6 with

pip install --upgrade spacy-dbpedia-spotlight

The following is a snippet of the configuration to skip SSL verifiication:

import spacy
nlp = spacy.blank('en')
# during the pipeline instantiation (e.g. custom dbpedia_rest_endpoint with HTTPS but self-signed certificate)
nlp.add_pipe('dbpedia_spotlight', config={'verify_ssl': False})
# or afterwards
nlp.get_pipe('dbpedia_spotlight').verify_ssl = False
# this will generate a warning, but will not break your processing (e.g. in a loop)
doc = nlp('Google LLC is an American multinational technology company.')
print(doc.ents)

# you can suppress warnings with this
import requests
from urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
# and now no warnings
doc = nlp('Google LLC is an American multinational technology company.')
print(doc.ents)

Let me know if this works for you.

Best,
Martino

[neostrange]

Hi @neostrange, The problem is on the DBpedia end. They have an expired certificate from 8 days. You can (temporarily) ignore certificate verification by editing the following lines and adding verify=False:

spacy-dbpedia-spotlight/spacy_dbpedia_spotlight/entity_linker.py

Lines 218 to 219 in 5c5802c

	return requests.post(
	f'{endpoint}/{self.process}', headers={'accept': 'application/json'}, data=params)

        return requests.post(
            f'{endpoint}/{self.process}', headers={'accept': 'application/json'}, verify=False, data=params)

I could an option in the configuration of the pipeline to allow ignoring verification more easily.

Best, Martino

Hi Martino,

Thank you so much for your response.

As per your solution, how can I install the plugin using pip?
Though I did fork your project and updated the code as you instructed and installed it using pip git+https but still, it did not work. Any tips to make it work as a plugin?

It would be great if you add a configuration parameter to turn SSL validation disabled.

Regards,
Ali

[MartinoMensio]

Hi @neostrange ,
I guess we were editing this at the same time! 😂

Try with the upgrade and then config parameter as in my last message and let me know if it works correctly!

Best,
Martino

[neostrange]

You can now install the new release v0.2.6 with

pip install --upgrade spacy-dbpedia-spotlight

The following is a snippet of the configuration to skip SSL verifiication:

import spacy
nlp = spacy.blank('en')
# during the pipeline instantiation (e.g. custom dbpedia_rest_endpoint with HTTPS but self-signed certificate)
nlp.add_pipe('dbpedia_spotlight', config={'verify_ssl': False})
# or afterwards
nlp.get_pipe('dbpedia_spotlight').verify_ssl = False
# this will generate a warning, but will not break your processing (e.g. in a loop)
doc = nlp('Google LLC is an American multinational technology company.')
print(doc.ents)

# you can suppress warnings with this
import requests
from urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
# and now no warnings
doc = nlp('Google LLC is an American multinational technology company.')
print(doc.ents)

Let me know if this works for you.

Best, Martino

Hi Martino,

I tried but it give me the following error:

ConfigValidationError:

Config validation error
dbpedia_spotlight -> verify_ssl	extra fields not permitted
{'nlp': , 'name': 'dbpedia_spotlight', 'confidence': None, 'dbpedia_rest_endpoint': None, 'debug': False, 'language_code': None, 'overwrite_ents': True, 'policy': None, 'process': 'annotate', 'raise_http_errors': True, 'span_group': 'dbpedia_spotlight', 'sparql': None, 'support': None, 'types': None, 'verify_ssl': False, '@factories': 'dbpedia_spotlight'}

Regards,

[neostrange]

Hi @neostrange , I guess we were editing this at the same time! 😂

Try with the upgrade and then config parameter as in my last message and let me know if it works correctly!

Best, Martino

Yes, I think we are 👍

[MartinoMensio]

Have you tried upgrading the package with pip install --upgrade spacy-dbpedia-spotlight?

What is the result of the following?

import spacy_dbpedia_spotlight
print(spacy_dbpedia_spotlight.__version__)

If it is not 0.2.6, it means that for some reason you are not getting the updated library from pypi: https://pypi.org/project/spacy-dbpedia-spotlight/

[neostrange]

Have you tried upgrading the package with pip install --upgrade spacy-dbpedia-spotlight?

What is the result of the following?

import spacy_dbpedia_spotlight
print(spacy_dbpedia_spotlight.__version__)

If it is not 0.2.6, it means that for some reason you are not getting the updated library from pypi: https://pypi.org/project/spacy-dbpedia-spotlight/

Yes, you are right, the pip couldn't upgrade the library in the python virtual environment at my side. I am trying to fix it and will post here the result.
Thanks
Ali