-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
set IV after the encryption key to actually use IV in AES-*-GCM algorithms #22
Conversation
Just got bit by this as well; it's breaking attr_encrypted gem attr-encrypted/attr_encrypted#203 |
@borama thank you for opening this issue. The intention of using aes-*-gcm was to improve security. It looks like this bug prevented that effort. It looks like the only indication that the IV should be set after the key is in the example here: http://ruby-doc.org/stdlib-2.3.0/libdoc/openssl/rdoc/OpenSSL/Cipher.html#class-OpenSSL::Cipher-label-Authenticated+Encryption+and+Associated+Data+-28AEAD-29 However, it's not obvious that this ordering is required for the algorithm to work correctly. I believe you are correct in your assessment regarding the breaking changes it will introduce. Do you have any recommendations on how to move forward with this? I'm not sure if this requires another major version change, but since it will indeed break existing code using aes-*-gcm I'm leaning to the fact that yes it should. There is a path forward for those that have been bitten by this bug, but it's not pretty. They can upgrade, and implement code to decrypt the old value and then re-encrypt using the new code that utilizes the IV. It's not pretty but it's a path forward. Any suggestions that you have would be very much appreciated. |
I would tend to agree with you about the major version change - a careless update could be catastrophic. Perhaps the data manipulations required by the update could be automated in a rake task, or something, to help people do it correctly... |
Hello guys, I came to about the same conclusions as you. I was also thinking about a rake task that would help users to decrypt (without IVs) and reencrypt (with IVs) their data. There is a risk I see in the fact that there is no way to automatically recognize if an encrypted value was calculated using or not using IV (unless you know the plain text data), so I'd be worried that someone ran this task more than once, for example. Perhaps the task should be interactive, showing a sample decrypted data and forcing the user to confirm that the decrypted data is ok before actually running the process? Another problem I see is that it would be advisable for I think I'd also agree with a major update, with big warnings everywhere about the consequences of the upgrade... |
Agreed. But you want to make super sure this bug is really squashed; you don't want to have several major version releases in a single week.
Seconded.
Suggestion for interim point version: leave the -gcm algorithms broken for backwards compatibility but (a) add a deprecation warning when they are invoked then (b) implement fixed versions suffixed like -gcm-fixed - yeah I know that's yucky but in order to migrate you need both version working so you can decrypt from the bad algorithm and encrypt with the good one. |
I like the idea of a rake task, but I think there's too many variables to consider. Perhaps at the very least an example of how to do it is in order. For users of attr_encrypted, the path forward is a bit easier, as you can pass in the encryption library to use. An affected app can copy the broken implementation and use that for decryption, and use the new version of encryptor for encrypting. I agree with @borama's suggestion that this data migration should happen offline to prevent the different implementations from being used otherwise the migration becomes significantly more complex. @trak3r I think the test that @borama provided is fairly conclusive, but point taken. Please feel free to review the tests and offer any suggestions. |
I think this test proves that this is only a problem for the GCM algorithm. https://github.com/attr-encrypted/encryptor/blob/master/test/compatibility_test.rb#L44 |
I've pushed a new version of Encryptor. https://rubygems.org/gems/encryptor/versions/3.0.0 The bug that @borama pointed out is fixed. I've also added an option to allow the IV to be set in the way that it was in Encryptor v2.0.0, so that decryption of data encrypted using an AES-*-GCM algorithm is possible. This should facilitate migrating data that was incorrectly encrypted. |
@saghaulor thanks for the speedy update, great work! |
FYI, I made some further tests and came to believe this is either bug or an undocumented feature in the |
FYI, it was recently confirmed that this had actually been an issue in the underlying ruby-openssl library and the bug has been fixed today. So, in the future, it is possible that |
@borama I saw that, thanks for the update. The problem is, if people use v2 and upgrade to the fix, they're going to have records encrypted with varying implementations. It will be a mess to sort out. |
@saghaulor thanks, you are correct, this did not occur to me and I can see now that the fix may actually complicate things even more :( |
Hopefully there won't be too much fallout. 🙏 |
Hello,
a stack overflow user noticed a weird behavior when using
attr_encrypted
(see the SO question and discussion). The problem is that when encrypting with one of theaes-*-gcm
algorithms while using initialization vector (IV), the encrypted result is the same for any IV, i.e. the init. vector is not taken into account at all. This seems to be an important security problem, especially considering that theaes-256-gcm
is the default encryption algorithm in this gem.Further investigation showed that the problem is only with the
-gcm
algorithms, other algorithms seem to be OK, which is btw also proved by thetest_should_crypt_with_the_#{algorithm}_algorithm_with_iv
test. Unfortunately this test does not verify the-gcm
algorithms.The problem can be fixed simply by setting the IV after setting the encryption key (current code sets IV before setting the key).
It is unclear to me where exactly this problem originates from, but the fact that it affects only the
-gcm
algorithms and not others suggests that it is a bug somewhere in the OpenSSL library code forAES-*-GCM
algorithms. I could not find anything related to the order of setting IVs and keys in any documentation.Anyway, this pull request moves setting the IV after setting the key and adds an accompanying test. This test fails in the current master branch but will run OK after merging.
It is very unfortunate that the default encryption algorithm is
aes-256-gcm
and thus the encryption is less secure for everybody who is not using random salt. A sad thing to note is also that this fix will probably make all these setups undecipherable as suddenly the IV would be taken into account after upgrading this gem.Testing the issue:
Tested under ruby 2.3.0 compiled against OpenSSL version "1.0.1f 6 Jan 2014".
Running the
test_encryption
gives:Test results 1-4 show that for CBC it is irrelevant if IV is set before or after the encryption key and that IV is always taken into account (the results differ for different IVs).
The results 5-8 on the other hand prove that the encrypted results are the same for two different IVs if IV set before the key, whereas the results are different when IV set after key.
If you need anything else from me regarding this issue, I'll be happy to help.