Added decryption of resources and patches. Refactored SOPS test data. #1286
Conversation
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
|
@stefanprodan tagging you because you have reviewed #1283 for me 😅 |
stefanprodan
added
area/sops
internal/decryptor/decryptor.go
Outdated
| // Iterate over all resources in the Kustomization file and attempt to decrypt them if they are encrypted. | ||
| for _, res := range kus.Resources { | ||
| // Determine the format for the resource, defaulting to YAML if not specified. | ||
| format := formatForPath(res) | ||
| // Visit the resource reference and attempt to decrypt it. | ||
| if err := visitRef(res, format, true); err != nil { | ||
| return err | ||
| } | ||
| } | ||
| // Iterate over all patches in the Kustomization file and attempt to decrypt their paths if they are encrypted. | ||
| for _, patch := range kus.Patches { | ||
| if patch.Path == "" { | ||
| continue | ||
| } | ||
| // Determine the format for the patch, defaulting to YAML if not specified. | ||
| format := formatForPath(patch.Path) | ||
| // Visit the patch reference and attempt to decrypt it. | ||
| if err := visitRef(patch.Path, format, false); err != nil { | ||
| return err | ||
| } | ||
| } |
There was a problem hiding this comment.
Why is this needed? Besides the secret/configMap generators, all the other YAML encrypted resources are decrypted in-memory before apply.
There was a problem hiding this comment.
I will give a short example that will be close to the case I have faced.
We have victoria-metrics HelmRelease in bases. It contains many services, also it is partially encrypted, because of basicAuth embedded in externalWrite URL.
In overlays, we have many values blocks for the same HelmRelease defined but in separate merge patch files. One of them is alert-manager.sops.yaml that has Slack token and it is also encrypted.
When flux runs kustomization and merges them before the decryption then it is fails on sops decryption stage, because SOPS metadata fields are mixed/overwriten etc. and hashes do not match anymore.
That is why I needed patches to be decrypted before the kustomization processing.
There was a problem hiding this comment.
There is an example https://github.com/vlasov-y/kustomize-controller/tree/main/internal/controller/testdata/sops/patches of such overlays, I have included one in tests.
There was a problem hiding this comment.
Ok but for resources this will fail the moment you use remote bases no?
There was a problem hiding this comment.
I thought about that and that Is why I have added extra arg ignoreNotRegular to visitRef
https://github.com/vlasov-y/kustomize-controller/blob/aefd7aa811cc7afa19a9005504393f03e3734bb8/internal/decryptor/decryptor.go#L418
and used it
https://github.com/vlasov-y/kustomize-controller/blob/aefd7aa811cc7afa19a9005504393f03e3734bb8/internal/decryptor/decryptor.go#L479
There was a problem hiding this comment.
We load all resources to memory during the kustomization build anyway, do not we?
Exactly, and this PR doubles the memory usage and the Go GC CPU usage.
Can we decrypt resources basing on filename?
I'm not for changing the API to cope with a SOPS upstream issue, I'm Ok with decrypting the patches but after merge SOPS should handle mixed mode.
There was a problem hiding this comment.
I'm not for changing the API to cope with a SOPS upstream issue, I'm Ok with decrypting the patches but after merge SOPS should handle mixed mode.
That is not a SOPS issues. SOPS decrypts the file that it encrypts without any issue, but we alter that file, we may adding keys that match encrypted_regex. Merging this PR with patches decryption only won't resolve mix test case completely, but will allow to easily bypass such limitation moving encrypted values to patch files. I think that will make life much easier already.
And I do not like changing API for this case either, that resourcesRegexMatch feels like a "dirty" solution anyway.
There was a problem hiding this comment.
@vlasov-y the next dev meeting is in 2 days, feel free to join https://fluxcd.io/community/#meetings
I understand now how SOPS fails, it's due to the regex. I think the best way forward is to decrypt the patches only and document this workaround, where the secret will have an empty stringData entry will all keys coming from patches.
There was a problem hiding this comment.
@vlasov-y the next dev meeting is in 2 days, feel free to join https://fluxcd.io/community/#meetings
Thanks!
I think the best way forward is to decrypt the patches only and document this workaround
Thanks, agree. I have tried to benchmark decryption of resources and to find a better algorithm, tried streaming reading of files, but it is slower. Anyway, making patches being decrypted and document the regex issue case will be more than enough to live with Kustomization + SOPS with ease.
There was a problem hiding this comment.
Updated documentation describing the case.
Removed decryption of resources.
Removed incompatible sops tests.
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Signed-off-by: Yuriy <yuriy@vlasov.pro>
Hi ✋
In addition to the previous PR, I've added decryption for the resource and patch files before the build process. I also found the test data for SOPS to be quite messy, so I restructured it entirely and streamlined the controller test code.