Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for variable substitution in the logs configuration beyond the prebuilt variables #1406

Open
chubbard opened this issue Nov 1, 2024 · 0 comments

Comments

@chubbard
Copy link

chubbard commented Nov 1, 2024

Is your feature request related to a problem? Please describe.
I want to simplify my log configuration for different environments (production, staging, etc) so I can share a single cloudwatch agent config and swap out the little pieces I need per environment through existing variable substitution mechanisms. But the variable substitution facility is pre-defined and fixed preventing me from simplifying my configs.

Describe the solution you'd like
In the logs configuration section you can use {instanceId} or other variables to substitute in values from the environment, but these are fixed values controlled by the agent. They are not extensible in any way. These fixed values don't cover things like environment variables, EC2 metadata, or tags which could expand what is possible to cover the situation where I want to differ values by environment.

I want the ability to substitute in either environment variables, tags, or EC2 metadata into these because I want to define things log groups or log streams that differ per environment so that I don't have to write multiple configs or roll your own bash scripts to accomplish this.

So for example I want to define a couple of log groups for my different environments then I can share the log configuration if I was able to do something like this:

"log_group_name": "myApp.{env:ENV_VAR}.sys"

Where ENV_VAR is an environment variable in linux I could substitute into that expression for the log group. Then my log configuration could be shared between production and staging without needing to copy them.

Or possibly using EC2 metadata that I can add properties too like so:

"log_stream_name": "myApp.{meta:region}.cron"

Other features like AWS Backup use tags to signify which assets to backup. Using tags allows the user to define this in one place and have it be applicable across lots of AWS services:

"log_stream_name": "myApp.{tag:Environment}.secure"

The tag Environment have a value of "production", "staging", "testing", etc. Tags are like cloud environment variables.

Describe alternatives you've considered
The alternative is a complex architecture of bash scripts, and deployment code to modify and carefully deploy copies to parameter store using aws cli. The other option was to write a gradle script to "build" the config files at deployment time. Since committing these configs to git makes sense for long term management then treating them like yet another build also makes sense if I'm willing to accept the additional overhead.

Additional context

If I had this feature I wouldn't have to build devops scripts and it would simplify my management of our dev ops which if I'm truthful devops == me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant