OpenFDA should enforce HTTPS for its API

[konklone]

The OpenFDA API currently doesn't enforce HTTPS, because of concerns over SNI support:

We encourage you to use https://api.fda.gov for all queries to ensure secure communication. We use Server Name Indication (SNI) External link icon to support SSL. As not all clients support SNI External link icon, we do not enforce HTTPS access. If your application does not support SNI, you can use http://api.fda.gov.

FDA should review this decision, as enforcing HTTPS for its API is important for its users (and is required by OMB's M-15-13 HTTPS-only policy).

Things to consider:

• The latest Python 2.7.X versions support SNI, which was not the case when OpenFDA first launched.
• For clients using the OpenFDA API in-browser, IE+XP use is now very low, much lower than when OpenFDA first launched. On analytics.usa.gov the amount is < 3% (since not all IE8 users are XP), but most sites I've checked into have lower %'s than even the government-wide average.
• Now that OpenFDA has been up for a while, it should have analytics showing how many clients currently using the API don't support SNI (or couldn't easily be upgraded to a version that does).
• OpenFDA can announce that HTTP support will be shut off at some point in the future (such as by the end of 2016) and should have time to proactively reach out to non-SNI-compatible client users and warn them of the change.

FDA should enforce HTTPS for its APIs, and help move its userbase (and the federal userbase generally) into the world of SNI, now that the tools to do so are easy and common.

[mattmo]

Thanks for reporting this. We recently moved from having api.fda.gov served by Cloudfront (which then proxied to API Umbrella) to having api.fda.gov served directly by API Umbrella. The SNI issue should be a non-issue now since the SSL cert is served by the ELB. I'll confirm this with the API Umbrella team and also speak with them about the ability to redirect users to https

[GUI]

@mattmo: Yup, this should be pretty easy for you to do within the api.data.gov admin. Under Configuration > API Backends, select your API backend to edit. Under the "Global Request Settings," you'll find a "HTTPS Requirements" option where you can change your settings. After making changes, just remember to publish them via Configuration > Publish Changes.

There's several options here, so you'll need to figure out which option best suites your needs. The tooltip for this field in the admin provides more detailed documentation that hopefully helps:

Choose whether HTTPS is required to access this API. HTTPS is encouraged to protect the API keys.

• Required & return message: HTTPS is required to access the API. HTTP requests will return an error message instructing the user to use an HTTPS URL instead. This is the recommended and default strategy.
• Required & return redirect: HTTPS is required to access the API. HTTP requests will return a redirect to the HTTPS URL.
• Transitionary & return message: New API keys that signup after choosing this setting will be forced to use HTTPS. Existing API keys may continue to use either HTTP or HTTPS. New API keys using HTTP will return an error message instructing the user to use an HTTPS URL instead.
• Transitionary & return redirect: New API keys that signup after choosing this setting will be forced to use HTTPS. Existing API keys may continue to use either HTTP or HTTPS. New API keys using HTTP will return a redirect to the HTTPS URL.
• Optional: HTTPS is optional and either HTTP or HTTPS may be used.

Notes on redirects:

• Not all API clients will automatically follow redirects, so be careful if using a redirect strategy for existing APIs (since existing calls may break).
• If API clients rely on the redirect for HTTPS access, this strategy does not secure the API keys, since the client may still be making an insecure initial HTTP request with their API key.
• For GET requests a 301 Moved Permanently redirect will be returned. For all other HTTP methods a 307 Temporary Redirect redirect will be returned (to instruct the client to retry using the same HTTP method).

However, I'd be careful with redirects. Here's some further discussion on some of the pitfalls of redirects: 18F/api.data.gov#34 (comment) and https://https.cio.gov/apis/

The "transitionary & return message" option is probably the best option that should have no impact on users. However, the ultimate goal should be the "required & return message" option, but that may require a bit more coordination with previous users that are still making calls insecurely (which you can determine using the api.data.gov analytics).

And regarding SNI, you're correct--with the SSL setup provided by api.data.gov, you're no longer dependent on SNI support for HTTPS access. (FYI: on api.data.gov's end, we are beginning to explore requiring SNI for new agency subdomains, so this may change in the future. However, right now we're taking a gradual approach, and there are no immediate plans for requiring this on existing agency domains. If our future plans ever change on this front, we'll be sure to reach out with much more details.)

[konklone]

@GUI You continue to post the most rigorous technical analysis of any public servant I know.

Also, as the chief maintainer of https://https.cio.gov/apis/ I would love to make it the most helpful thing possible, to you and other agencies. Suggestions and pull requests welcome.

[mattmo]

We just posted a notice at the top of our site indicating access via HTTP is deprecated. We will switch over to the "transitionary & return message" recommended by @GUI on April 4th. Please let me know if you have any questions.

[konklone]

👏 👏 👏 🎉

That's terrific! And an example for other federal APIs. Please do send any lessons learned or suggested language for https://https.cio.gov/apis/ if you can think of anything that would improve it!

[konklone]

How'd the transition go?

[mattmo]

Given the executive order and that we have posted the notice for a month at the top of all open.fda.gov pages, we have switched to "require & return". Please let me know if you have any questions.