Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impl use of content of the service account key file in the environment variable YC_SERVICE_ACCOUNT_KEY_FILE #82

Merged
merged 3 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .web-docs/components/builder/yandex/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ can also be supplied to override the typical auto-generated key:

- `source_image_family` (string) - The source image family to create the new image
from. You can also specify source_image_id instead. Just one of a source_image_id or
source_image_family must be specified. Example: `ubuntu-1804-lts`.
source_image_family must be specified. Example: `ubuntu-2204-lts`.

<!-- End of code generated from the comments of the SourceImageConfig struct in builder/yandex/common_config.go; -->

Expand All @@ -130,9 +130,9 @@ can also be supplied to override the typical auto-generated key:

- `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.

- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
`YC_SERVICE_ACCOUNT_KEY_FILE`.
- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).

- `max_retries` (int) - The maximum number of times an API request is being executed.

Expand Down Expand Up @@ -181,6 +181,8 @@ can also be supplied to override the typical auto-generated key:

- `instance_cores` (int) - The number of cores available to the instance.

- `instance_core_fraction` (int) - The vCPU performance level (core fraction) of the instance

- `instance_gpus` (int) - The number of GPU available to the instance.

- `instance_mem_gb` (int) - The amount of memory available to the instance, specified in gigabytes.
Expand Down
8 changes: 5 additions & 3 deletions .web-docs/components/post-processor/yandex-export/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ Also, you should configure [ssh communicator](/packer/docs/communicators/ssh). D

- `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.

- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
`YC_SERVICE_ACCOUNT_KEY_FILE`.
- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).

- `max_retries` (int) - The maximum number of times an API request is being executed.

Expand Down Expand Up @@ -131,6 +131,8 @@ Also, you should configure [ssh communicator](/packer/docs/communicators/ssh). D

- `instance_cores` (int) - The number of cores available to the instance.

- `instance_core_fraction` (int) - The vCPU performance level (core fraction) of the instance

- `instance_gpus` (int) - The number of GPU available to the instance.

- `instance_mem_gb` (int) - The amount of memory available to the instance, specified in gigabytes.
Expand Down
6 changes: 3 additions & 3 deletions .web-docs/components/post-processor/yandex-import/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ file.

- `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.

- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
`YC_SERVICE_ACCOUNT_KEY_FILE`.
- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).

- `max_retries` (int) - The maximum number of times an API request is being executed.

Expand Down
2 changes: 1 addition & 1 deletion GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ dev: build
test:
@go test -race -count $(COUNT) $(TEST) -timeout=3m

install-packer-sdc: ## Install packer sofware development command
install-packer-sdc: ## Install packer software development command
@go install github.com/hashicorp/packer-plugin-sdk/cmd/packer-sdc@${HASHICORP_PACKER_PLUGIN_SDK_VERSION}

plugin-check: install-packer-sdc build
Expand Down
32 changes: 27 additions & 5 deletions builder/yandex/access_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
package yandex

import (
"encoding/json"
"errors"
"fmt"
"os"
Expand All @@ -15,6 +16,14 @@ import (
"github.com/yandex-cloud/go-sdk/iamkey"
)

type keyType int

const (
Undefined keyType = iota
File
Content
)

const (
defaultEndpoint = "api.cloud.yandex.net:443"
defaultMaxRetries = 3
Expand All @@ -24,9 +33,9 @@ const (
type AccessConfig struct {
// Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
Endpoint string `mapstructure:"endpoint" required:"false"`
// Path to file with Service Account key in json format. This
// is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
// `YC_SERVICE_ACCOUNT_KEY_FILE`.
// Contains either a path to or the contents of the Service Account file in JSON format.
// This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
// You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
ServiceAccountKeyFile string `mapstructure:"service_account_key_file" required:"false"`
// [OAuth token](https://cloud.yandex.com/docs/iam/concepts/authorization/oauth-token)
// or [IAM token](https://cloud.yandex.com/docs/iam/concepts/authorization/iam-token)
Expand All @@ -35,6 +44,8 @@ type AccessConfig struct {
Token string `mapstructure:"token" required:"true"`
// The maximum number of times an API request is being executed.
MaxRetries int `mapstructure:"max_retries"`

saKeyType keyType
}

func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
Expand Down Expand Up @@ -66,8 +77,19 @@ func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
}

if c.ServiceAccountKeyFile != "" {
if _, err := iamkey.ReadFromJSONFile(c.ServiceAccountKeyFile); err != nil {
errs = append(errs, fmt.Errorf("fail to read service account key file: %s", err))
// if ServiceAccountKeyFile is file path
if _, err := os.Stat(c.ServiceAccountKeyFile); err == nil {
if _, err := iamkey.ReadFromJSONFile(c.ServiceAccountKeyFile); err != nil {
errs = append(errs, fmt.Errorf("fail to read service account key file: %s", err))
}
c.saKeyType = File
} else {
// else check for a valid json data value
var f map[string]interface{}
if err := json.Unmarshal([]byte(c.ServiceAccountKeyFile), &f); err != nil {
errs = append(errs, fmt.Errorf("JSON in %q are not valid: %s", c.ServiceAccountKeyFile, err))
}
c.saKeyType = Content
}
}

Expand Down
67 changes: 67 additions & 0 deletions builder/yandex/access_config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
package yandex

import (
"errors"
"os"
"testing"

"github.com/hashicorp/packer-plugin-sdk/template/interpolate"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestAccessConfig_Prepare(t *testing.T) {
bytes, err := os.ReadFile(TestServiceAccountKeyFile)
require.NoErrorf(t, err, "failed to read file %s", TestServiceAccountKeyFile)

var TestServiceAccountKeyFileContent = string(bytes)

type fields struct {
Endpoint string
ServiceAccountKeyFile string
Token string
MaxRetries int
saKeyType keyType
}
tests := []struct {
name string
fields fields
want []error
}{
{
name: "sa_key_as_file", fields: fields{
Endpoint: "",
ServiceAccountKeyFile: TestServiceAccountKeyFile,
Token: "",
},
want: nil,
},
{
name: "sa_key_as_json_content", fields: fields{
ServiceAccountKeyFile: TestServiceAccountKeyFileContent,
Token: "",
},
want: nil,
},
{
name: "both_identities", fields: fields{
ServiceAccountKeyFile: TestServiceAccountKeyFileContent,
Token: "t1.super-token",
},
want: []error{errors.New("one of token or service account key file must be specified, not both")},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx := &interpolate.Context{}
c := &AccessConfig{
Endpoint: tt.fields.Endpoint,
ServiceAccountKeyFile: tt.fields.ServiceAccountKeyFile,
Token: tt.fields.Token,
MaxRetries: tt.fields.MaxRetries,
saKeyType: tt.fields.saKeyType,
}
assert.Equalf(t, tt.want, c.Prepare(ctx), "Prepare(%v)")
})
}
}
2 changes: 1 addition & 1 deletion builder/yandex/common_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ func (c *InstanceConfig) Prepare(errs *packersdk.MultiError) *packersdk.MultiErr
type SourceImageConfig struct {
// The source image family to create the new image
// from. You can also specify source_image_id instead. Just one of a source_image_id or
// source_image_family must be specified. Example: `ubuntu-1804-lts`.
// source_image_family must be specified. Example: `ubuntu-2204-lts`.
SourceImageFamily string `mapstructure:"source_image_family" required:"true"`
// The ID of the folder containing the source image.
SourceImageFolderID string `mapstructure:"source_image_folder_id" required:"false"`
Expand Down
13 changes: 11 additions & 2 deletions builder/yandex/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
package yandex

import (
"io/ioutil"
"os"
"strings"
"testing"
Expand All @@ -15,9 +14,14 @@ import (
const TestServiceAccountKeyFile = "./testdata/fake-sa-key.json"

func TestConfigPrepare(t *testing.T) {
tf, err := ioutil.TempFile("", "packer")
tf, err := os.CreateTemp("", "packer")
require.NoError(t, err, "create temporary file failed")

bytes, err := os.ReadFile(TestServiceAccountKeyFile)
require.NoErrorf(t, err, "failed to read file %s", TestServiceAccountKeyFile)

var TestServiceAccountKeyFileContent = string(bytes)

defer os.Remove(tf.Name())
tf.Close()

Expand Down Expand Up @@ -47,6 +51,11 @@ func TestConfigPrepare(t *testing.T) {
TestServiceAccountKeyFile,
false,
},
{
"service_account_key_file",
TestServiceAccountKeyFileContent,
false,
},

{
"folder_id",
Expand Down
19 changes: 17 additions & 2 deletions builder/yandex/driver_yc.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package yandex

import (
"context"
"errors"
"fmt"
"log"
"strings"
Expand Down Expand Up @@ -60,8 +61,7 @@ func NewDriverYC(ui packersdk.Ui, ac *AccessConfig) (Driver, error) {
sdkConfig.Credentials = ycsdk.OAuthToken(ac.Token)
}
case ac.ServiceAccountKeyFile != "":
log.Printf("[INFO] Use Service Account key file %q for authentication", ac.ServiceAccountKeyFile)
key, err := iamkey.ReadFromJSONFile(ac.ServiceAccountKeyFile)
key, err := iamKeyFromSAKey(ac)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -106,6 +106,21 @@ func NewDriverYC(ui packersdk.Ui, ac *AccessConfig) (Driver, error) {

}

func iamKeyFromSAKey(ac *AccessConfig) (*iamkey.Key, error) {
switch ac.saKeyType {
case File:
log.Printf("[INFO] Use Service Account key (file %q) for authentication", ac.ServiceAccountKeyFile)
return iamkey.ReadFromJSONFile(ac.ServiceAccountKeyFile)
case Content:
log.Printf("[INFO] Use Service Account key (as raw JSON) for authentication")
return iamkey.ReadFromJSONBytes([]byte(ac.ServiceAccountKeyFile))
case Undefined:
return nil, errors.New("`undefined` SA key type - something goes wrong")
default:
return nil, errors.New("failed to determine SA key type; perhaps skipped the preparation stage")
}
}

func (d *driverYC) GetImage(imageID string) (*Image, error) {
image, err := d.sdk.Compute().Image().Get(context.Background(), &compute.GetImageRequest{
ImageId: imageID,
Expand Down
6 changes: 3 additions & 3 deletions docs-partials/builder/yandex/AccessConfig-not-required.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

- `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.

- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
`YC_SERVICE_ACCOUNT_KEY_FILE`.
- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).

- `max_retries` (int) - The maximum number of times an API request is being executed.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

- `source_image_family` (string) - The source image family to create the new image
from. You can also specify source_image_id instead. Just one of a source_image_id or
source_image_family must be specified. Example: `ubuntu-1804-lts`.
source_image_family must be specified. Example: `ubuntu-2204-lts`.

<!-- End of code generated from the comments of the SourceImageConfig struct in builder/yandex/common_config.go; -->
2 changes: 1 addition & 1 deletion website/content/docs/builders/yandex.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ To get more information about this kind of authentication check [documentaion](h
"type": "yandex",
"token": "YOUR OAUTH TOKEN",
"folder_id": "YOUR FOLDER ID",
"source_image_family": "ubuntu-1804-lts",
"source_image_family": "ubuntu-2204-lts",
"ssh_username": "ubuntu",
"use_ipv4_nat": "true"
}
Expand Down