Skip to content

Latest commit

 

History

History
75 lines (42 loc) · 1.76 KB

README.md

File metadata and controls

75 lines (42 loc) · 1.76 KB

Data Generator

A simple data generator for easy validations of Logs & Metrics ingestion.

img.png

Quick Start

Require Go version 1.22+

  • Clone the repository
  • Copy config.sample.yaml and edit as needed.
  • Start the data generator with config file as the only parameter go run main.go --configFile ./config.yaml

Configurations

Check config.sample.yaml as a reference configuration.

Data sources

Logs

ECS (Elastic Common Schema) formated logs based on zap. Check ecs-logging-go-zap for adoption

Metrics

Generate metrics similar to a CloudWatch metrics entry. Limited support and tobe improved

Exporters

Important

AWS specific exporters require aws.profile to configure with a valid credential profile name. And make sure you have sufficient permissions to use the exporter with these credentials.

File

Generates a file with new line as data point delimiter.

Available options,

  • file_location : Optional file location to write the output to. Default to ./out
S3

Write generated data to a S3 bucket.

Available options,

  • s3Bucket : Bucket name (required)
  • bucketSeconds: Period between two bucket entries. Optional and default to 120seconds
  • bucketPrefix: Optional prefix for the entry. Default to logFile-
FIREHOSE

Push data to an AWS firehose data stream.

Available options,

  • firehoseStreamName : firehose stream name (required)
CLOUDWATCH_LOG

Push data to AWS CloudWatch log group or stream name.

Available options,

  • cloudwatchLogGroup : Cloudwatch log group name
  • cloudwatchLogStreamName : Define a specific log stream name

Important

One of these options must be set to use the exporter