Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

harvester sees SSLError SSLCertVerificationError when harvesting DOI source #4526

Closed
FuhuXia opened this issue Nov 10, 2023 · 5 comments
Closed
Labels
bug Software defect or bug O&M Operations and maintenance tasks for the Data.gov platform

Comments

@FuhuXia
Copy link
Member

FuhuXia commented Nov 10, 2023

This error started a week ago when harvesting url https://datainventory.doi.gov/data.json

Harvester reports error:

ConnectionError getting json source: HTTPSConnectionPool(host='datainventory.doi.gov', port=443):
Max retries exceeded with url: /data.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
certificate (_ssl.c:1129)'))).

curl shows the same kind of error:

$ curl -I  https://datainventory.doi.gov/data.json
HTTP/1.1 200 OK
Server: Caddy
Content-Length: 0

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

SSL certificate is fine for Chrome browser:
image

How to reproduce

Check harvest source /harvest/test-new-doi-datajson

@FuhuXia FuhuXia added the bug Software defect or bug label Nov 10, 2023
@FuhuXia
Copy link
Member Author

FuhuXia commented Nov 10, 2023

It appears the issue is on DOI side, their Digicert SSL certificate is incorrectly installed.

The server is not sending the required intermediate certificate.

https://www.digicert.com/help/
https://www.sslshopper.com/ssl-checker.html#hostname=datainventory.doi.gov

@hkdctol
Copy link
Contributor

hkdctol commented Nov 16, 2023

Awaiting answer from DOI

@FuhuXia FuhuXia mentioned this issue Nov 20, 2023
10 tasks
@FuhuXia FuhuXia added the O&M Operations and maintenance tasks for the Data.gov platform label Nov 20, 2023
@jbrown-xentity jbrown-xentity moved this to 📔 Product Backlog in data.gov team board Nov 21, 2023
@jbrown-xentity
Copy link
Contributor

In checking the cert via @FuhuXia 's link above, it's reported as valid now. So we should be able to re-harvest this source...

@jbrown-xentity jbrown-xentity moved this from 📔 Product Backlog to 📟 Sprint Backlog [7] in data.gov team board Nov 21, 2023
@jbrown-xentity jbrown-xentity moved this from 📟 Sprint Backlog [7] to 📔 Product Backlog in data.gov team board Nov 21, 2023
@jbrown-xentity
Copy link
Contributor

Fuhu is correct, there is still a broken piece in the chain. Check curl (as per the description), or see here:

Image

@FuhuXia FuhuXia moved this from 📔 Product Backlog to 🏗 In Progress [8] in data.gov team board Nov 22, 2023
@FuhuXia FuhuXia self-assigned this Nov 22, 2023
@FuhuXia FuhuXia moved this from 🏗 In Progress [8] to 📡 Blocked in data.gov team board Nov 22, 2023
@FuhuXia
Copy link
Member Author

FuhuXia commented Nov 28, 2023

All checks are green now. curl command is successful. datainventory.doi.gov has its SSL certificate fixed.

@FuhuXia FuhuXia closed this as completed Nov 28, 2023
@github-project-automation github-project-automation bot moved this from 📡 Blocked to ✔ Done in data.gov team board Nov 28, 2023
@hkdctol hkdctol moved this from ✔ Done to 🗄 Closed in data.gov team board Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Software defect or bug O&M Operations and maintenance tasks for the Data.gov platform
Projects
Archived in project
Development

No branches or pull requests

3 participants