config_map_files and generic_secret_data support

docs/_docs/helpers.md

@@ -2,18 +2,9 @@
 title: Helpers
 ---
 
-Kubes provides some helper methods to help write Kubernetes YAML files.  Here's a list of the helper methods. These are available whether you write your resources in YAML or DSL.
-
-Helper | Description
---- | ---
-decode64 | Base64 decode a string.
-docker_image | Method refers to the latest Docker image built by Kubes. This spares you from having to update the image manually in the deployment resource. Note, this can be overridden with the `--image` cli option or the `Kubes.config.image` setting. See: [Docker Image]({% link _docs/intro/docker-image.md %})
-dockerfile_port	| Exposed port extracted from the Dockerfile of the project.
-encode64 | Base64 encode a string. Also available as `base64` method.
-extra | The `KUBES_EXTRA` value.
-with_extra | Appends the `KUBES_EXTRA` value to a string if it's set. It's covered in the [Extra Env Docs]({% link _docs/extra-env.md %}).
-
-Here's also the source code with most of the helpers: [helpers.rb](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/compiler/shared/helpers.rb).
+## Built-In Helpers
+
+Kubes provides core helper methods to help write Kubernetes YAML files. Docs: [Built-In Helpers]({% link _docs/helpers/builtin.md %}).
 
 ## DSL Specific Methods
 

docs/_docs/helpers/aws/secret_data.md

@@ -0,0 +1,55 @@
+---
+title: AWS Secrets
+nav_text: Secrets Data
+categories: helpers-aws
+---
+
+The `aws_secret_data` helper fetches secret data that is one single file from AWS Secrets.
+
+## Example
+
+For example if you have these secret values stored as one file with multiple values separated by `=`.
+
+    $ aws secretsmanager get-secret-value --secret-id demo-dev-secret-data | jq '.SecretString'
+    KEY1=secretvalue1
+    KEY2=secretvalue2
+
+Kubes can fetch the secret data and base64 encode the values properly.  Example:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<%= aws_secret_data("demo-dev-secret-data") %>
+```
+
+Notice how the text is idented properly by 2 spaces and the values are automatically base64 encoded.
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  KEY1: c2VjcmV0dmFsdWUx
+  KEY2: c2VjcmV0dmFsdWUy
+```
+
+## Options
+
+Here's an example of the available options for `aws_secret_data`.
+
+```ruby
+aws_secret_data("demo-#{Kubes.env}-secret-data", base64: true, ident: 2)
+```

docs/_docs/helpers/builtin.md

@@ -0,0 +1,16 @@
+---
+title: Core Built-In Helpers
+---
+
+Kubes provides some helper methods to help write Kubernetes YAML files.  Here's a list of the helper methods. These are available whether you write your resources in YAML or DSL.
+
+Helper | Description
+--- | ---
+decode64 | Base64 decode a string.
+docker_image | Method refers to the latest Docker image built by Kubes. This spares you from having to update the image manually in the deployment resource. Note, this can be overridden with the `--image` cli option or the `Kubes.config.image` setting. See: [Docker Image]({% link _docs/intro/docker-image.md %})
+dockerfile_port	| Exposed port extracted from the Dockerfile of the project.
+encode64 | Base64 encode a string. Also available as `base64` method.
+extra | The `KUBES_EXTRA` value.
+with_extra | Appends the `KUBES_EXTRA` value to a string if it's set. It's covered in the [Extra Env Docs]({% link _docs/extra-env.md %}).
+
+Here's also the source code with most of the helpers: [helpers.rb](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/compiler/shared/helpers.rb).

docs/_docs/helpers/builtin/config-map-files.md

@@ -0,0 +1,63 @@
+---
+title: Config Map Files
+nav_text: Config Map Files
+---
+
+The `config_map_files` helper allows you to add config map data from a list of files. The files support layerying.
+
+## Example
+
+Here's how to use it.
+
+.kubes/resources/shared/config_map.yaml
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<%= config_map_files %>
+```
+
+You can conveniently set multiple configmap values in files like so:
+
+.kubes/resources/shared/config_map/base.txt
+
+    KEY1=cmvalue1
+    KEY2=cmvalue2
+
+.kubes/resources/shared/config_map/dev.txt
+
+    KEY2=cmvalue2-dev-override
+    KEY3=cmvalue3
+
+The resulting generated ConfigMap will be:
+
+```yaml
+---
+metadata:
+  namespace: demo-dev
+  labels:
+    app: demo
+  name: demo-928146dd24
+apiVersion: v1
+kind: ConfigMap
+data:
+  KEY1: cmvalue1
+  KEY2: cmvalue2-dev-override
+  KEY3: cmvalue3
+```
+
+## Layering Details
+
+Layering for Config Map Files and also supports app-scoped layers.
+
+Name | Example
+--- | ---
+configmap root | .kubes/resources/shared/config_map/{base,dev}.txt
+configmap app-scoped | .kubes/resources/shared/config_map/app1/{base,dev}.txt
+
+So if `KUBES_APP=app1`, then the app-scoped layer is also used. This handles the [Central Deployer Pattern]({% link _docs/patterns/central-deployer.md %}).

docs/_docs/helpers/google/secret_data.md

@@ -0,0 +1,55 @@
+---
+title: Google Secrets
+nav_text: Secrets Data
+categories: helpers-google
+---
+
+The `google_secret_data` helper fetches secret data that is one single file from Google Secrets.
+
+## Example
+
+For example if you have these secret values stored as one file with multiple values separated by `=`.
+
+    $ gcloud secrets versions access latest --secret demo-dev-secret-data
+    KEY1=secretvalue1
+    KEY2=secretvalue2
+
+Kubes can fetch the secret data and base64 encode the values properly.  Example:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<%= google_secret_data("demo-dev-secret-data") %>
+```
+
+Notice how the text is idented properly by 2 spaces and the values are automatically base64 encoded.
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  KEY1: c2VjcmV0dmFsdWUx
+  KEY2: c2VjcmV0dmFsdWUy
+```
+
+## Options
+
+Here's an example of the available options for `google_secret_data`.
+
+```ruby
+google_secret_data("demo-#{Kubes.env}-secret-data", base64: true, ident: 2)
+```

docs/_includes/sidebar.html

@@ -153,7 +153,11 @@ <h2><a href="{% link docs.md %}">Docs</a></h2>
       </li>
       <li><a href="{% link _docs/helpers.md %}">Helpers</a>
         <ul>
-          <li><a href="{% link _docs/helpers/custom.md %}">Custom</a></li>
+          <li><a href="{% link _docs/helpers/builtin.md %}">Built-In</a>
+            <ul>
+              <li><a href="{% link _docs/helpers/builtin/config-map-files.md %}">Config Map Files</a>
+            </ul>
+          </li>
           <li><a href="{% link _docs/helpers/aws.md %}">AWS</a>
             <ul>
               {% assign docs = site.docs | where: "categories","helpers-aws" %}
@@ -192,6 +196,7 @@ <h2><a href="{% link docs.md %}">Docs</a></h2>
               {% endfor %}
             </ul>
           </li>
+          <li><a href="{% link _docs/helpers/custom.md %}">Custom</a></li>
         </ul>
       </li>
       <li><a href="{% link _docs/patterns.md %}">Patterns</a>

lib/kubes/compiler/shared/helpers.rb

@@ -5,6 +5,8 @@ module Kubes::Compiler::Shared
   module Helpers
     extend Kubes::Compiler::Dsl::Core::Fields
     fields "name"
+
+    include ConfigMapHelper
     include DockerHelper
     include ExtraHelper
     include SecretHelper

lib/kubes/compiler/shared/helpers/config_map_helper.rb

@@ -0,0 +1,35 @@
+module Kubes::Compiler::Shared::Helpers
+  module ConfigMapHelper
+    def config_map_files(options={})
+      indent = options[:indent] || 2
+
+      shared_config_map = "#{Kubes.root}/.kubes/resources/shared/config_map"
+      layers = [
+        [shared_config_map, "base.txt"],
+        [shared_config_map, "#{Kubes.env}.txt"],
+      ]
+      if Kubes.app
+        layers += [
+          [shared_config_map, Kubes.app, "base.txt"],
+          [shared_config_map, Kubes.app, "#{Kubes.env}.txt"],
+        ]
+      end
+      layers.map! { |layer| layer.compact.join('/') }
+      data = {}
+      layers.each do |path|
+        next unless File.exist?(path)
+        lines = IO.readlines(path)
+        lines.each do |line|
+          key, value = line.split('=').map(&:strip)
+          data[key] = value
+        end
+      end
+
+      spacing = " " * indent
+      lines = data.map do |key,value|
+        "#{spacing}#{key}: #{value}"
+      end
+      lines.join("\n")
+    end
+  end
+end

lib/kubes/compiler/shared/helpers/secret_helper.rb

@@ -1,6 +1,22 @@
-# Including ConfigMap here also
 module Kubes::Compiler::Shared::Helpers
   module SecretHelper
+    # Meant to be used by plugins. IE:
+    # google_secret_data and aws_secret_data
+    def generic_secret_data(plugin_secret_method, name, options={})
+      indent = options[:indent] || 2
+      base64 = options[:base64].nil? ? true : options[:base64]
+
+      full_data = send(plugin_secret_method, name, base64: false)
+      spacing = " " * indent
+      lines = full_data.split("\n")
+      new_lines = lines.map do |line|
+        key, value = line.split('=')
+        value = encode64(value) if base64
+        "#{spacing}#{key}: #{value}"
+      end
+      new_lines.join("\n")
+    end
+
     def encode64(v)
       Base64.strict_encode64(v.to_s).strip
     end