Hash request URIs before using them as a cache key #181
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iMacTia
requested changes
Sep 19, 2018
There was a problem hiding this comment.
@timrogers thanks for quickly turning this in!
Appreciate you listing the trade-offs of this approach, I believe we should be good in both cases but happy to reconsider at a future date if any issue arises.
There's just a small nit that makes Travis complain, I've left a comment below.
| @@ -66,7 +67,8 @@ def cache_key(env) | |||
| url.query = params.any? ? build_query(params) : nil | |||
| end | |||
| url.normalize! | |||
| url.request_uri | |||
|
|
|||
There was a problem hiding this comment.
There appears to be a trailing whitespace here according to Travis:
lib/faraday_middleware/response/caching.rb:70 Line contains trailing whitespace
|
@timrogers just touching base to know if you might have some time to close this off? It's basically finished except for that small typo 👍 |
|
Yes - sure. I’ll do this shortly.
|
Issue lostisland#132 notes that the caching middleware can generate keys which are too long when the input is a complex URL, leading to errors like `Errno::ENAMETOOLONG: File name too long @ rb_sysopen`. (The effective length limit depends on what cache driver you're using.) At the moment, we use the normalised request URI as the cache key. The fixes the issue by using `Digest::SHA1.hexdigest` to hash the URLs, giving us a constant 20-byte result. The trade-off of using a hash is that: * there is the potential for collisions (as rare as they will realistically be!) * performing a hash takes time
timrogers
force-pushed
the
hash-cache-keys
branch
from
da6ba29 to
8922555
Compare
iMacTia
approved these changes
Nov 26, 2018
|
Thanks @timrogers! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #132 notes that the caching middleware can generate keys which are too long when the input is a complex URL, leading to errors like
Errno::ENAMETOOLONG: File name too long @ rb_sysopen. (The effective length limit depends on what cache driver you're using.)At the moment, we use the normalised request URI as the cache key. The fixes the issue by using
Digest::SHA1.hexdigestto hash the URLs, giving us a constant 20-byte result.The trade-off of using a hash is that: