Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Caching middleware can generate cache keys with #cache_key which are too long #132

Closed
timrogers opened this issue Mar 16, 2016 · 4 comments
Closed

Caching middleware can generate cache keys with #cache_key which are too long #132

timrogers opened this issue Mar 16, 2016 · 4 comments
Labels
bug

Comments

@timrogers
Copy link
Contributor

#cache_key in FaradayMiddleware::Caching (https://github.com/lostisland/faraday_middleware/blob/master/lib/faraday_middleware/response/caching.rb#L61) can generate cache keys that are too long for complex URLs.

This can lead to errors like Errno::ENAMETOOLONG: File name too long @ rb_sysopen.

I'm not really sure how best to go about fixing this, as obviously the uniqueness is important as it makes the cache per-identical request. Perhaps we could hash the keys in some way to get them to a uniform, shorter length?
@timrogers timrogers mentioned this issue Mar 16, 2016
Closed
@timrogers
Copy link
Contributor Author

Anyone there? ❤️

@iMacTia
Copy link
Member

iMacTia commented Sep 19, 2018

@timrogers sorry for being late on this one.

Hashing the url.request_uri before returning it seems like a reasonable solution!
Appreciate a lot of time has passed so you might not be interested in this anymore, but if you still are, I'd happily review a PR

@iMacTia iMacTia added the bug label Sep 19, 2018
@timrogers
Copy link
Contributor Author

Happy to look at this. Shouldn't be too complicated.

timrogers pushed a commit to timrogers/faraday_middleware that referenced this issue Sep 19, 2018
Hash request URIs before using them as a cache key
da6ba29 
Issue lostisland#132 notes that the caching middleware can generate keys
which are too long when the input is a complex URL, leading to
errors like
`Errno::ENAMETOOLONG: File name too long @ rb_sysopen`. (The
effective length limit depends on what cache driver you're using.)

At the moment, we use the normalised request URI as the cache key.
The fixes the issue by using `Digest::SHA1.hexdigest` to hash
the URLs, giving us a constant 20-byte result.

The trade-off of using a hash is that:

* there is the potential for collisions (as rare as they will
realistically be!)
* performing a hash takes time
@timrogers timrogers mentioned this issue Sep 19, 2018
Merged
timrogers pushed a commit to timrogers/faraday_middleware that referenced this issue Nov 26, 2018
Hash request URIs before using them as a cache key
8922555 
Issue lostisland#132 notes that the caching middleware can generate keys
which are too long when the input is a complex URL, leading to
errors like
`Errno::ENAMETOOLONG: File name too long @ rb_sysopen`. (The
effective length limit depends on what cache driver you're using.)

At the moment, we use the normalised request URI as the cache key.
The fixes the issue by using `Digest::SHA1.hexdigest` to hash
the URLs, giving us a constant 20-byte result.

The trade-off of using a hash is that:

* there is the potential for collisions (as rare as they will
realistically be!)
* performing a hash takes time
@iMacTia
Copy link
Member

iMacTia commented Feb 24, 2020

Fixed in #181 🎉!
Thanks again!

@iMacTia iMacTia closed this as completed Feb 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@timrogers @iMacTia