Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Terraform CLI logs support #248

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Terraform CLI logs support #248

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
352ba15
Added writing terraform CLI logs to a file support
Feb 20, 2024
79ffa3a
Support log file rollover based on state change
rvnyk Feb 26, 2024
009ef82
enableLogging changed to a boolean option to providerconfig
amirbavand Feb 27, 2024
d6329d9
added the logic to configuring cleanup of archived terraform log files
amirbavand Feb 29, 2024
2f406bb
changed logConfigs to logConfig
Feb 29, 2024
ad979cf
applied some formatting and added comments
Feb 29, 2024
42aa432
applied some formatting and added comments
Feb 29, 2024
dcbcbaf
added description for new config parameters
Feb 29, 2024
0094b75
fixed lint issues
Mar 1, 2024
75d8444
Unit tests added for writeCLILogs and minor code changes
rvnyk Mar 7, 2024
7e1ba76
Bug fix, Added additional test and documentation
Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions apis/v1beta1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ type ProviderConfigSpec struct {
// +optional
// +kubebuilder:default=true
PluginCache *bool `json:"pluginCache,omitempty"`

// LogConfig configures terraform cli logging
// +optional
LogConfig *LogConfig `json:"logConfig,omitempty"`
}

// LogConfig configures terraform cli logging
type LogConfig struct {
// EnableLogging enables terraform cli logging
// +optional
EnableLogging *bool `json:"enableLogging,omitempty"`
// BackupLogFilesCount is the number of archived log files to retain
// +optional
BackupLogFilesCount *int `json:"backupLogFilesCount,omitempty"`
}

// ProviderCredentials required to authenticate.
Expand Down
30 changes: 30 additions & 0 deletions apis/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions docs/Configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,3 +355,25 @@ spec:
At Vault side configuration is also needed to allow the write operation, see
[example](https://docs.crossplane.io/knowledge-base/integrations/vault-as-secret-store/)
here for inspiration.

## Enable Terraform CLI logs

Terraform CLI logs can be written to a log file to assist with debugging and to view detailed information about Terraform operations.
To enable it, the `ProviderConfig` spec has an **optional** `LogConfig` field.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To enable it, the `ProviderConfig` spec has an **optional** `LogConfig` field.
To enable it, the `ProviderConfig` spec has an **optional** `logConfig` field.

nit: LogConfig -> logConfig


```yaml
apiVersion: tf.upbound.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
logConfig:
enableLogging: True
backupLogFilesCount: 1
...
```

- `enableLogging`: Specifies whether logging is enabled (`true`) or disabled (`false`). When enabled, Terraform CLI command logs will be written to a file. Default is `false`
- `backupLogFilesCount`: Specifies the number of archived log files to retain. When a new log file is created due to a change detected by `terraform diff`, the previous log file is archived and renamed with a timestamp. This parameter controls how many archived log files are kept before older ones are deleted. Default is `0`

By default, Terraform CLI stores log files in the workspace directory. The default log file name is `terraform.log`. When a backup is taken (e.g., due to a new change detected by `terraform diff`), the current log file is renamed to `terraform.log.<timestamp>`, where `<timestamp>` is a placeholder for the actual timestamp of the backup.
16 changes: 16 additions & 0 deletions examples/providerconfig-with-logConfig.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: tf.upbound.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
configuration: |
terraform {
backend "kubernetes" {
secret_suffix = "providerconfig-aws-eu-west-1"
namespace = "upbound-system"
in_cluster_config = true
}
}
logConfig:
backupLogFilesCount: 1
enableLogging: True
30 changes: 26 additions & 4 deletions internal/controller/workspace/workspace.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,8 @@ func Setup(mgr ctrl.Manager, o controller.Options, timeout, pollJitter time.Dura
usage: resource.NewProviderConfigUsageTracker(mgr.GetClient(), &v1beta1.ProviderConfigUsage{}),
logger: o.Logger,
fs: fs,
terraform: func(dir string, usePluginCache bool) tfclient {
return terraform.Harness{Path: tfPath, Dir: dir, UsePluginCache: usePluginCache}
terraform: func(dir string, usePluginCache bool, logConfig v1beta1.LogConfig) tfclient {
return terraform.Harness{Path: tfPath, Dir: dir, UsePluginCache: usePluginCache, LogConfig: logConfig}
},
}

Expand Down Expand Up @@ -166,7 +166,7 @@ type connector struct {
usage resource.Tracker
logger logging.Logger
fs afero.Afero
terraform func(dir string, usePluginCache bool) tfclient
terraform func(dir string, usePluginCache bool, logConfig v1beta1.LogConfig) tfclient
}

func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.ExternalClient, error) { //nolint:gocyclo
Expand Down Expand Up @@ -284,7 +284,29 @@ func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.E
pc.Spec.PluginCache = new(bool)
*pc.Spec.PluginCache = true
}
tf := c.terraform(dir, *pc.Spec.PluginCache)

// disable logging by default
if pc.Spec.LogConfig == nil {
pc.Spec.LogConfig = &v1beta1.LogConfig{
EnableLogging: new(bool),
BackupLogFilesCount: new(int),
}
*pc.Spec.LogConfig.EnableLogging = false
*pc.Spec.LogConfig.BackupLogFilesCount = 0
} else
// if logging is not null, then set the value of EnableLogging and BackupLogFilesCount if it is not set
{
if pc.Spec.LogConfig.EnableLogging == nil {
pc.Spec.LogConfig.EnableLogging = new(bool)
*pc.Spec.LogConfig.EnableLogging = false
}
if pc.Spec.LogConfig.BackupLogFilesCount == nil {
pc.Spec.LogConfig.BackupLogFilesCount = new(int)
*pc.Spec.LogConfig.BackupLogFilesCount = 0
}
}

tf := c.terraform(dir, *pc.Spec.PluginCache, *pc.Spec.LogConfig)
if cr.Status.AtProvider.Checksum != "" {
checksum, err := tf.GenerateChecksum(ctx)
if err != nil {
Expand Down
30 changes: 15 additions & 15 deletions internal/controller/workspace/workspace_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ func TestConnect(t *testing.T) {
kube client.Client
usage resource.Tracker
fs afero.Afero
terraform func(dir string, usePluginCache bool) tfclient
terraform func(dir string, usePluginCache bool, logConfig v1beta1.LogConfig) tfclient
}

type args struct {
Expand Down Expand Up @@ -216,7 +216,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -255,7 +255,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join(tfDir, string(uid), tfCreds): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -294,7 +294,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join(tfDir, string(uid), "subdir", tfCreds): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -338,7 +338,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join("/tmp", tfDir, string(uid), ".git-credentials"): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -381,7 +381,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join("/tmp", tfDir, string(uid)): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -422,7 +422,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join(tfDir, string(uid), tfConfig): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -463,7 +463,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join(tfDir, string(uid), "subdir", tfConfig): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -499,7 +499,7 @@ func TestConnect(t *testing.T) {
errs: map[string]error{filepath.Join(tfDir, string(uid), tfMain): errBoom},
},
},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
}
Expand Down Expand Up @@ -529,7 +529,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{MockInit: func(_ context.Context, _ ...terraform.InitOption) error { return errBoom }}
},
},
Expand All @@ -553,7 +553,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
MockWorkspace: func(_ context.Context, _ string) error { return errBoom },
Expand All @@ -579,7 +579,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockGenerateChecksum: func(ctx context.Context) (string, error) { return "", errBoom },
}
Expand Down Expand Up @@ -613,7 +613,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockGenerateChecksum: func(ctx context.Context) (string, error) { return tfChecksum, nil },
MockWorkspace: func(_ context.Context, _ string) error { return nil },
Expand Down Expand Up @@ -649,7 +649,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error { return nil },
MockGenerateChecksum: func(ctx context.Context) (string, error) { return tfChecksum, nil },
Expand Down Expand Up @@ -688,7 +688,7 @@ func TestConnect(t *testing.T) {
},
usage: resource.TrackerFn(func(_ context.Context, _ resource.Managed) error { return nil }),
fs: afero.Afero{Fs: afero.NewMemMapFs()},
terraform: func(_ string, _ bool) tfclient {
terraform: func(_ string, _ bool, _ v1beta1.LogConfig) tfclient {
return &MockTf{
MockInit: func(ctx context.Context, o ...terraform.InitOption) error {
args := terraform.InitArgsToString(o)
Expand Down
Loading
Loading