fix test config

fix yaml test config and also format the code fix wrong usage log.get, now all tests are passing Ran cargo vdev fmt Fix reference in to_metrics_metadata in log_to_metric.rs Convert type errors to single error with type. Added test for multiple metrics but it is failing. Additional attempts to get the multiple_metadata_metrics working. fix(playground): fix playground vrl version and link (vectordotdev#19119) * fix(playground): fix playground vrl version and link * Ran cargo vdev fmt * remove redundant line * replace clone() with as_ref() whenever possible * Ran cargo vdev fmt chore(website): Fix commenting step on workflow (vectordotdev#19134) * fix: extra env vars into comment step * fix: add sleep to allow branch to connect * fix: update where branch name and sanitized branch name are used Remove multiple metrics test. chore(ci): Bump bufbuild/buf-setup-action from 1.27.2 to 1.28.0 (vectordotdev#19137) Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.27.2 to 1.28.0. - [Release notes](https://github.com/bufbuild/buf-setup-action/releases) - [Commits](bufbuild/buf-setup-action@v1.27.2...v1.28.0) --- updated-dependencies: - dependency-name: bufbuild/buf-setup-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> chore(deps): Bump the clap group with 1 update (vectordotdev#19127) Bumps the clap group with 1 update: [clap](https://github.com/clap-rs/clap). - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@v4.4.7...v4.4.8) --- updated-dependencies: - dependency-name: clap dependency-type: direct:production update-type: version-update:semver-patch dependency-group: clap ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> chore(releasing): Add known issue for Datadog Metrics sink in v0.34.0 (vectordotdev#19122) * chore(releasing): Add known issue for Datadog Metrics sink in v0.34.0 Ref: vectordotdev#19110 Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com> * Update website/cue/reference/releases/0.34.0.cue Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> --------- Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com> Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com> chore(deps): Bump proptest from 1.3.1 to 1.4.0 (vectordotdev#19131) Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.3.1 to 1.4.0. - [Release notes](https://github.com/proptest-rs/proptest/releases) - [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md) - [Commits](proptest-rs/proptest@v1.3.1...v1.4.0) --- updated-dependencies: - dependency-name: proptest dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> chore(deps): Bump env_logger from 0.10.0 to 0.10.1 (vectordotdev#19130) Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.10.0 to 0.10.1. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.10.0...v0.10.1) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> chore(docs): Add alpha to traces and beta to metrics in descriptions (vectordotdev#19139) add alpha to traces and beta to metrics in descriptions Update README.md (vectordotdev#19142) chore(deps): Bump hdrhistogram from 7.5.2 to 7.5.3 (vectordotdev#19129) Bumps [hdrhistogram](https://github.com/HdrHistogram/HdrHistogram_rust) from 7.5.2 to 7.5.3. - [Release notes](https://github.com/HdrHistogram/HdrHistogram_rust/releases) - [Changelog](https://github.com/HdrHistogram/HdrHistogram_rust/blob/main/CHANGELOG.md) - [Commits](HdrHistogram/HdrHistogram_rust@v7.5.2...v7.5.3) --- updated-dependencies: - dependency-name: hdrhistogram dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> enhancement(file sink, aws_s3 sink, gcp_cloud_storage): configurable filename timezone (vectordotdev#18506) * add TzOffset as File Sink configuration * integrate TzOffset into File Sink * apply tz offset to all log event in render_timestamp. * added TzOffset tests * adding chrono-tz for parsing timezones * rename tz_offset to path_tz. timezones are safer than offsets * update tz_offset references to path_tz * cargo fmt * remove unnecessary commented out code. fmt and generate-component-docs * clippy suggestions and remove TryFrom<&str> - serde handles converting to String * rename Template config option `path_tz` to `timezone` * move `path_tz.rs` to `src/config` preparing for applying the same to `aws_s3` sink for filename timezone * update doc configuration description for path_tz * fix wrong method name * AWS and GCS filename timezone support * remove custom tz config * use VRL's timezone config * pass around SinkContext * use TzOffset to pass down to request builders. VRL's TimeZone can't be hash derived * make key_prefix timezone aware and use Option `or` syntax * move tz to offset conversion codes to sink util * remove empty line * update timezone docs in vector-config * get timezone and convert to offset in one go in FileSink * just pass the sinkconfig directly * vector_common to vector_lib * configurable_component is in vector_lib now * lookup to vector_lib * fix aws s3 integration test. pass the context to build_processor in tests * formatting * add sinkcontext to FileSink in file tests * key_prefix is expected to be a template. no need for into Update documentation Updated documentation to include example enhancement(networking, sinks): add full jitter to retry backoff policy (vectordotdev#19106) * enhancement(networking, sinks): add full jitter to retry backoff policy * fmt * fix tests * add test * fix * force ci fix(file source, kubernetes_logs source, file sink): make file internal metric tag opt-in (vectordotdev#19145) * fix(file source, kubernetes_logs source, file sink): make file internal metric tag opt-in * update cue * fix tests fix(datadog_metrics sink): evaluate series v1 env var at runtime (vectordotdev#19148) fix(datadog_metrics sink): evaluate v1 env var at runtime chore(website): WEB-4247 | Update references from s3 to setup.vector.dev (vectordotdev#19149) feat: update references from s3 to setup.vector.dev fix(ARC, networking): improve request settings (vectordotdev#19101) * fix(ARC, networking): improve request settings * fix spelling * change defaults * refactor * self-review * clippy * update default overrides * fmt nit * add upgrade guide entry Update src/transforms/log_to_metric.rs Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> Tweak and generate docs Tweak and generate docs Update website/cue/reference/components/transforms/base/log_to_metric.cue Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Update website/cue/reference/components/transforms/base/log_to_metric.cue Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Update website/cue/reference/components/transforms/base/log_to_metric.cue Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Update src/transforms/log_to_metric.rs Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Update src/transforms/log_to_metric.rs Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Update src/transforms/log_to_metric.rs Co-authored-by: Heston Hoffman <hestonhoffman@gmail.com> Fix styling in example for all_metrics. Minor tweaks to formatting for documentation Fix minor issues with formatting that was causing tests to fail.
dygfloyd · Nov 15, 2023 · 3b931e6 · 3b931e6
1 parent 15930cf
commit 3b931e6
Show file tree

Hide file tree

Showing 134 changed files with 2,767 additions and 1,203 deletions.
diff --git a/.github/workflows/create_preview_sites.yml b/.github/workflows/create_preview_sites.yml
@@ -52,8 +52,9 @@ jobs:
       run: |
         unzip pr.zip -d pr
         BRANCH_NAME=$(cat ./pr/branch)
-        SANITIZED_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed 's/\./-/g')
+        SANITIZED_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed 's/[\/\.]/-/g')
         echo "SANITIZED_BRANCH_NAME=$SANITIZED_BRANCH_NAME" >> $GITHUB_ENV
+        echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
 
     # Kick off the job in amplify
     - name: Deploy Site
@@ -64,12 +65,13 @@ jobs:
         REQUEST_MESSAGE: ${{ secrets.REQUEST_MESSAGE }}
         ENDPOINT: ${{ secrets.ENDPOINT }}
       run: |
+        sleep 20
         HMAC_KEY=$(echo -n $REQUEST_MESSAGE | openssl dgst -sha256 -hmac "$REQUEST_TOKEN" -binary | od -An -tx1 | tr -d ' \n'; echo)
         SIGNATURE="sha256=$HMAC_KEY"
         RESPONSE_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
              -H "Content-Type: application/json" \
              -H "X-Hub-Signature: $SIGNATURE" \
-             -d "{\"app_id\": \"$APP_ID\", \"branch_name\": \"$SANITIZED_BRANCH_NAME\"}" \
+             -d "{\"app_id\": \"$APP_ID\", \"branch_name\": \"$BRANCH_NAME\"}" \
           "$ENDPOINT")
 
         # check the response code and fail if not 200
@@ -91,6 +93,7 @@ jobs:
           const fs = require('fs');
           const prNumber = fs.readFileSync('./pr/number', 'utf8');
           const issueNumber = parseInt(prNumber);
+          const { APP_ID, APP_NAME, SANITIZED_BRANCH_NAME } = process.env
 
           await github.rest.issues.createComment({
             owner: context.repo.owner,

diff --git a/.github/workflows/protobuf.yml b/.github/workflows/protobuf.yml
@@ -20,7 +20,7 @@ jobs:
       # Run `git checkout`
       - uses: actions/checkout@v3
       # Install the `buf` CLI
-      - uses: bufbuild/buf-setup-action@v1.27.2
+      - uses: bufbuild/buf-setup-action@v1.28.0
       # Perform breaking change detection against the `master` branch
       - uses: bufbuild/buf-breaking-action@v1.1.3
         with:

diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -242,8 +242,9 @@ bollard = { version = "0.15.0", default-features = false, features = ["ssl", "ch
 bytes = { version = "1.5.0", default-features = false, features = ["serde"] }
 bytesize = { version = "1.3.0", default-features = false }
 chrono = { version = "0.4.31", default-features = false, features = ["serde"] }
+chrono-tz = { version = "0.8.3", default-features = false }
 cidr-utils = { version = "0.5.11", default-features = false }
-clap = { version = "4.4.7", default-features = false, features = ["derive", "error-context", "env", "help", "std", "string", "usage", "wrap_help"] }
+clap = { version = "4.4.8", default-features = false, features = ["derive", "error-context", "env", "help", "std", "string", "usage", "wrap_help"] }
 colored = { version = "2.0.4", default-features = false }
 csv = { version = "1.3", default-features = false }
 derivative = { version = "2.2.0", default-features = false }
@@ -351,7 +352,7 @@ criterion = { version = "0.5.1", features = ["html_reports", "async_tokio"] }
 itertools = { version = "0.11.0", default-features = false, features = ["use_alloc"] }
 libc = "0.2.150"
 similar-asserts = "1.5.0"
-proptest = "1.3"
+proptest = "1.4"
 quickcheck = "1.0.3"
 reqwest = { version = "0.11", features = ["json"] }
 rstest = {version = "0.18.2"}

diff --git a/README.md b/README.md
@@ -18,7 +18,7 @@
 Vector is a high-performance, end-to-end (agent & aggregator) observability data
 pipeline that puts you in control of your observability data.
 [Collect][docs.sources], [transform][docs.transforms], and [route][docs.sinks]
-all your logs, metrics, and traces to any vendors you want today and any other
+all your logs and metrics to any vendors you want today and any other
 vendors you may want tomorrow. Vector enables dramatic cost reduction, novel
 data enrichment, and data security where you need it, not where it is most
 convenient for your vendors. Additionally, it is open source and up to 10x
@@ -31,7 +31,7 @@ Vector**][docs.installation].
 
 * **Reliable** - Built in [Rust][urls.rust], Vector's primary design goal is reliability.
 * **End-to-end** - Deploys as an [agent][docs.roles#agent] or [aggregator][docs.roles#aggregator]. Vector is a complete platform.
-* **Unified** - [Logs][docs.data-model.log], [metrics][docs.data-model.metric], and traces (coming soon). One tool for all of your data.
+* **Unified** - [Logs][docs.data-model.log], [metrics][docs.data-model.metric] (beta), and traces (coming soon). One tool for all of your data.
 
 ### Use cases
 

diff --git a/docs/DEPRECATIONS.md b/docs/DEPRECATIONS.md
@@ -16,8 +16,6 @@ For example:
 
 ## To be migrated
 
-- file_metric_tag v0.36.0 The `internal_metrics.include_file_tag` should default to `false`.
-
 ## To be removed
 
 - datadog_v1_metrics v0.35.0 Support for `v1` series endpoint in the `datadog_metrics` sink should be removed.

diff --git a/lib/vector-api-client/Cargo.toml b/lib/vector-api-client/Cargo.toml
@@ -30,7 +30,7 @@ tokio-tungstenite = { version = "0.20.1", default-features = false, features = [
 
 # External libs
 chrono = { version = "0.4.31", default-features = false, features = ["serde"] }
-clap = { version = "4.4.7", default-features = false, features = ["derive"] }
+clap = { version = "4.4.8", default-features = false, features = ["derive"] }
 url = { version = "2.4.1", default-features = false }
 uuid = { version = "1", default-features = false, features = ["serde", "v4"] }
 indoc = { version = "2.0.4", default-features = false }
diff --git a/lib/vector-buffers/Cargo.toml b/lib/vector-buffers/Cargo.toml
@@ -34,14 +34,14 @@ vector-config-macros = { path = "../vector-config-macros", default-features = fa
 vector-common = { path = "../vector-common", default-features = false, features = ["byte_size_of", "serde"] }
 
 [dev-dependencies]
-clap = "4.4.7"
+clap = "4.4.8"
 criterion = { version = "0.5", features = ["html_reports", "async_tokio"] }
 crossbeam-queue = "0.3.8"
-hdrhistogram = "7.5.2"
+hdrhistogram = "7.5.3"
 metrics-tracing-context = { version = "0.14.0", default-features = false }
 metrics-util = { version = "0.15.1", default-features = false, features = ["debugging"] }
 once_cell = "1.18"
-proptest = "1.3"
+proptest = "1.4"
 quickcheck = "1.0"
 rand = "0.8.5"
 serde_yaml = { version = "0.9", default-features = false }

diff --git a/lib/vector-config/src/external/datetime.rs b/lib/vector-config/src/external/datetime.rs
@@ -20,8 +20,8 @@ impl Configurable for TimeZone {
 
     fn metadata() -> Metadata {
         let mut metadata = Metadata::default();
-        metadata.set_title("Timezone reference.");
-        metadata.set_description(r#"This can refer to any valid timezone as defined in the [TZ database][tzdb], or "local" which refers to the system local timezone.
+        metadata.set_title("Timezone to use for any date specifiers in template strings.");
+        metadata.set_description(r#"This can refer to any valid timezone as defined in the [TZ database][tzdb], or "local" which refers to the system local timezone. It will default to the [globally configured timezone](https://vector.dev/docs/reference/configuration/global-options/#timezone).
 
 [tzdb]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones"#);
         metadata.add_custom_attribute(CustomAttribute::kv(

diff --git a/lib/vector-core/Cargo.toml b/lib/vector-core/Cargo.toml
@@ -34,7 +34,7 @@ ordered-float = { version = "4.1.1", default-features = false }
 openssl = { version = "0.10.59", default-features = false, features = ["vendored"] }
 parking_lot = { version = "0.12.1", default-features = false }
 pin-project.workspace = true
-proptest = { version = "1.3", optional = true }
+proptest = { version = "1.4", optional = true }
 prost-types = { version = "0.12", default-features = false }
 prost = { version = "0.12", default-features = false, features = ["std"] }
 quanta = { version = "0.12.1", default-features = false }
@@ -80,7 +80,7 @@ criterion = { version = "0.5.1", features = ["html_reports"] }
 env-test-util = "1.0.1"
 quickcheck = "1"
 quickcheck_macros = "1"
-proptest = "1.3"
+proptest = "1.4"
 similar-asserts = "1.5.0"
 tokio-test = "0.4.3"
 toml = { version = "0.8.8", default-features = false, features = ["parse"] }

diff --git a/lib/vector-stream/Cargo.toml b/lib/vector-stream/Cargo.toml
@@ -19,6 +19,6 @@ vector-common = { path = "../vector-common" }
 vector-core = { path = "../vector-core" }
 
 [dev-dependencies]
-proptest = "1.3"
+proptest = "1.4"
 rand = "0.8.5"
 rand_distr = "0.4.3"
diff --git a/lib/vector-vrl/cli/Cargo.toml b/lib/vector-vrl/cli/Cargo.toml
@@ -7,6 +7,6 @@ publish = false
 license = "MPL-2.0"
 
 [dependencies]
-clap = { version = "4.4.7", features = ["derive"] }
+clap = { version = "4.4.8", features = ["derive"] }
 vector-vrl-functions = { path = "../functions" }
 vrl.workspace = true
diff --git a/lib/vector-vrl/tests/Cargo.toml b/lib/vector-vrl/tests/Cargo.toml
@@ -13,7 +13,7 @@ vector-vrl-functions = { path = "../../vector-vrl/functions" }
 ansi_term = "0.12"
 chrono = "0.4"
 chrono-tz = "0.8"
-clap = { version = "4.4.7", features = ["derive"] }
+clap = { version = "4.4.8", features = ["derive"] }
 glob = "0.3"
 prettydiff = "0.6"
 regex = "1"