feat(codecs): add support for protobuf encoding #18598

goakley · 2023-09-18T18:32:33Z

As part of #1536, Vector needs the ability to serialize events to the protobuf format. This PR adds a serializer to the codecs crate.

This also acts as a (spiritual) followup to the recent #18019 protobuf deserializer change - this PR includes logic that treats that PR's deserializer as the mirror of this PR's serializer.

netlify · 2023-09-18T18:32:38Z

✅ Deploy Preview for vrl-playground ready!

Name	Link
🔨 Latest commit	`62b1f38`
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/65146c20ddea790008b14433
😎 Deploy Preview	https://deploy-preview-18598--vrl-playground.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify · 2023-09-18T18:32:39Z

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	`62b1f38`
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/65146c20a998d00008b38203

aws-amplify-us-east-1 · 2023-09-18T18:33:49Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18598.d1hoyoksbulg25.amplifyapp.com

aws-amplify-us-east-1 · 2023-09-18T18:33:49Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18598.d2lr4eds605rpz.amplifyapp.com

pront

Hi @goakley, thank you for this PR!

Overall this is already in a good state. I left a few comments below and also, there are some minor spelling errors that need fixing.

Another very useful test to have is a roundtrip test similar to lib/codecs/tests/native.rs. Not a blocker for this PR though.

lib/codecs/src/encoding/format/protobuf.rs

pront · 2023-09-19T22:34:32Z

lib/codecs/tests/data/encoding/protobuf/test.proto

+    optional uint32 index = 2;
+  }
+  repeated EmbeddedMessage messages = 1;
+}


Let's also add an enum Foo, a default and a map similar to lib/codecs/tests/data/decoding/protobuf/test_protobuf3.proto.

Hahaha, guess who forgot to finish implementing all the protobuf constructs? Added enum, map, and google.protobuf.Timestamp.

default is interesting because it's removed in proto3 (only proto2 allows custom default values). For encoding, this shouldn't matter anyway, since prost will load the default when it parses the .desc file and use that default if we don't explicitly set a field during the encoding process. Thoughts on if we should even test such a situation?

Thank you for strengthening the tests.

Hmm, we stumbled upon the no presence (proto3) vs explicit presence (proto2) semantics here. We might have a situation where we cannot differentiate between a non-set value and a default value.

However, we have optional in proto3 as a workaround. The type would be Option<T> for such fields as described in the docs. Repeated fields are an interesting case here since they are just Vec<T> and not Option<Vec<T>>.

I'm not 100% sure how this maps to the encoding implementation. The implementation here only attempts to set fields that are part of the vector Value::Object, meaning that prost itself is left to decide what happens when the value is unset - it will either encode it as missing or encode it with a default value. If the field is optional, that could change how prost encodes the data, but shouldn't change how we set the fields, no?

lib/codecs/src/encoding/format/protobuf.rs

pront · 2023-09-19T22:42:07Z

lib/codecs/src/encoding/format/protobuf.rs

+    }
+
+    #[test]
+    fn test_encode_decoding_protobuf_test_data() {


👍 good test

lib/codecs/src/encoding/format/protobuf.rs

aws-amplify-us-east-1 · 2023-09-21T17:47:52Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18598.d1a7j77663uxsc.amplifyapp.com

goakley · 2023-09-21T17:50:27Z

The spirit of lib/codecs/tests/native.rs seems to be testing the round trip encode+decode, is that correct?

maycmlee

👍 for docs

pront

Thank you @goakley, there are some clippy errors and left a couple of comments. But overall, this looks good!

pront · 2023-09-21T19:31:42Z

lib/codecs/src/encoding/format/protobuf.rs

+            ])),
+        )
+        .unwrap();
+        assert!(mfield!(message, "i32").as_i32() == Some(-1234));


🥜 We generally prefer assert_eq for better readability and diagnostics.

pront · 2023-09-21T19:45:45Z

lib/codecs/src/encoding/format/protobuf.rs

+
+    #[test]
+    fn test_encode_decoding_protobuf_test_data() {
+        // test_protobuf (proto2)


🥜 We can remove this comment here and below.

pront · 2023-09-21T19:51:54Z

lib/codecs/src/encoding/format/protobuf.rs

+        (Value::Bytes(b), Kind::Enum(descriptor)) => {
+            let string = String::from_utf8_lossy(&b).into_owned();
+            // check for an exact enum name match
+            if let Some(d) = descriptor.values().filter(|v| v.name() == &string).next() {


🥜 Can we merge this and the following filter?

Oops, yep, I was under the impression that enum values were case sensitive, but they are not. Merged.

pront · 2023-09-21T19:52:28Z

The spirit of lib/codecs/tests/native.rs seems to be testing the round trip encode+decode, is that correct?

Yes, correct.

pront · 2023-09-27T15:43:25Z

lib/codecs/tests/protobuf.rs

+    let protobuf_message = read_protobuf_bin_message(&test_data_dir().join("person_someone.pb"));
+    let desc_file = test_data_dir().join("test_protobuf.desc");
+    let message_type: String = "test_protobuf.Person".into();
+    let (mut serializer, deserializer) = build_rializers(desc_file, message_type);


🥜 fix typo

pront · 2023-09-27T15:45:00Z

lib/codecs/src/encoding/format/protobuf.rs

@@ -0,0 +1,469 @@
+use crate::common::protobuf::get_message_descriptor;


🥜 Fix clippy errors

pront · 2023-09-27T17:57:17Z

lib/codecs/tests/protobuf.rs

@@ -0,0 +1,67 @@
+//! Tests for the behaviour of Protobuf serializer and deserializer (together).


Awesome, thank you for adding this.

pront · 2023-09-28T14:23:41Z

@goakley this looks good now, do you plan to make any more changes before we go ahead and merge?

goakley · 2023-09-28T17:07:32Z

@pront nope, I'm good if you're good, you can merge. I will have a followup PR to use this codec in a sink.

jszwedko

Approving the UX bits here (i.e. the documentation).

github-actions · 2023-09-29T20:30:16Z

Regression Detector Results

Run ID: 219fec4a-2029-4fd4-ae10-f8ddf5462b32
Baseline: 2439b62
Comparison: d3b84e9
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
datadog_agent_remap_blackhole_acks	ingress throughput	+4.03	[+3.89, +4.17]	100.00%
syslog_loki	ingress throughput	+3.98	[+3.88, +4.08]	100.00%
file_to_blackhole	egress throughput	+3.26	[+2.04, +4.48]	100.00%
datadog_agent_remap_datadog_logs_acks	ingress throughput	+0.81	[+0.68, +0.95]	100.00%
splunk_hec_route_s3	ingress throughput	+0.60	[+0.08, +1.13]	94.21%
fluent_elasticsearch	ingress throughput	+0.57	[+0.27, +0.88]	99.78%
http_text_to_http_json	ingress throughput	+0.54	[+0.44, +0.64]	100.00%
datadog_agent_remap_datadog_logs	ingress throughput	+0.39	[+0.23, +0.55]	99.99%
socket_to_socket_blackhole	ingress throughput	+0.32	[+0.29, +0.36]	100.00%
http_to_http_noack	ingress throughput	+0.30	[+0.10, +0.49]	98.86%
syslog_log2metric_humio_metrics	ingress throughput	+0.15	[+0.00, +0.29]	90.71%
http_to_http_json	ingress throughput	+0.00	[-0.05, +0.05]	6.59%
splunk_hec_indexer_ack_blackhole	ingress throughput	-0.00	[-0.17, +0.16]	3.88%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	-0.01	[-0.17, +0.16]	4.55%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.01	[-0.15, +0.13]	6.02%
http_to_s3	ingress throughput	-0.11	[-0.40, +0.17]	48.23%
otlp_grpc_to_blackhole	ingress throughput	-0.26	[-0.37, -0.14]	99.96%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-0.33	[-0.49, -0.18]	99.95%
http_to_http_acks	ingress throughput	-0.56	[-0.72, -0.41]	100.00%
otlp_http_to_blackhole	ingress throughput	-0.82	[-1.00, -0.65]	100.00%
syslog_humio_logs	ingress throughput	-1.06	[-1.20, -0.92]	100.00%
enterprise_http_to_http	ingress throughput	-1.10	[-1.28, -0.92]	100.00%
syslog_splunk_hec_logs	ingress throughput	-1.55	[-1.66, -1.44]	100.00%
syslog_regex_logs2metric_ddmetrics	ingress throughput	-1.95	[-2.33, -1.57]	100.00%
datadog_agent_remap_blackhole	ingress throughput	-2.31	[-2.41, -2.21]	100.00%

github-actions · 2023-09-29T20:59:38Z

Regression Detector Results

Run ID: b60f0908-3d76-4763-a660-eae9c7d64a30
Baseline: decaaeb
Comparison: 737f5c3
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
enterprise_http_to_http	ingress throughput	+2.76	[+2.58, +2.95]	100.00%
splunk_hec_route_s3	ingress throughput	+1.59	[+1.06, +2.12]	100.00%
datadog_agent_remap_datadog_logs_acks	ingress throughput	+1.19	[+1.05, +1.33]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	+0.77	[+0.57, +0.98]	100.00%
http_to_http_acks	ingress throughput	+0.41	[+0.26, +0.57]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+0.40	[+0.29, +0.50]	100.00%
socket_to_socket_blackhole	ingress throughput	+0.08	[-0.03, +0.19]	78.41%
otlp_http_to_blackhole	ingress throughput	+0.02	[-0.15, +0.19]	13.35%
splunk_hec_indexer_ack_blackhole	ingress throughput	+0.01	[-0.16, +0.18]	5.49%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	+0.00	[-0.16, +0.17]	1.28%
http_to_http_json	ingress throughput	-0.00	[-0.05, +0.05]	6.70%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.01	[-0.15, +0.14]	4.72%
otlp_grpc_to_blackhole	ingress throughput	-0.05	[-0.17, +0.08]	46.34%
http_to_s3	ingress throughput	-0.17	[-0.46, +0.12]	67.34%
http_text_to_http_json	ingress throughput	-0.23	[-0.36, -0.09]	99.42%
http_to_http_noack	ingress throughput	-0.70	[-0.93, -0.47]	100.00%
file_to_blackhole	egress throughput	-0.86	[-2.08, +0.36]	75.20%
syslog_splunk_hec_logs	ingress throughput	-0.88	[-0.95, -0.81]	100.00%
syslog_humio_logs	ingress throughput	-1.20	[-1.33, -1.08]	100.00%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-1.36	[-1.51, -1.20]	100.00%
datadog_agent_remap_blackhole	ingress throughput	-1.64	[-1.75, -1.52]	100.00%
syslog_regex_logs2metric_ddmetrics	ingress throughput	-1.78	[-2.20, -1.35]	100.00%
datadog_agent_remap_datadog_logs	ingress throughput	-1.84	[-1.99, -1.69]	100.00%
syslog_loki	ingress throughput	-2.00	[-2.05, -1.94]	100.00%
fluent_elasticsearch	ingress throughput	-2.11	[-2.43, -1.80]	100.00%

pront · 2023-10-03T19:39:21Z

Thank you @goakley 🎉

@spencergilbert

# [3.0.0](answerbook/vector@v2.1.2...v3.0.0) (2024-01-17) ### Bug Fixes * **amqp sink**: remove duplicate events (vectordotdev#18932) [a916605](answerbook/vector@a916605) - GitHub * **amqp sink**: remove unnecessary unwrap & emit event dropped errors (vectordotdev#18923) [26f430c](answerbook/vector@26f430c) - GitHub * **amqp sink**: remove unused feature flag (vectordotdev#18948) [cb53588](answerbook/vector@cb53588) - GitHub * **buffers**: apply stricter file permissions to buffer data files when possible (vectordotdev#18895) [cf7298f](answerbook/vector@cf7298f) - GitHub * **clickhouse sink**: fix healthcheck uri (vectordotdev#19067) [7a7b53b](answerbook/vector@7a7b53b) - GitHub * **codecs**: fix 'ProtobufSerializerConfig' input type (vectordotdev#19264) [d2fea65](answerbook/vector@d2fea65) - Jesse Szwedko * **codecs**: native JSON serialization/deserialization for special f64 values (vectordotdev#18650) [39b9298](answerbook/vector@39b9298) - GitHub * **config**: Only try default paths if the path is not specified (vectordotdev#18681) [37fc9db](answerbook/vector@37fc9db) - GitHub * **datadog_agent source, datadog_metrics sink**: handle interval for non-rate series metrics (vectordotdev#18889) [dc9d966](answerbook/vector@dc9d966) - GitHub * **datadog_agent source**: return 200 on empty object payload (vectordotdev#19093) [ab7983a](answerbook/vector@ab7983a) - Jesse Szwedko * **datadog_metrics sink**: evaluate series v1 env var at runtime (vectordotdev#19148) [7b292ce](answerbook/vector@7b292ce) - Jesse Szwedko * **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [6a40169](answerbook/vector@6a40169) - GitHub * **datadog_metrics sink**: Revert to using v1 endpoint by default (vectordotdev#19138) [3158f46](answerbook/vector@3158f46) - Jesse Szwedko * **datadog_metrics sink**: the integration tests weren't actually validating anything (vectordotdev#18754) [afc166f](answerbook/vector@afc166f) - GitHub * **datadog_traces sink**: improve request size limiting (vectordotdev#18903) [a477d72](answerbook/vector@a477d72) - GitHub * **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [e445721](answerbook/vector@e445721) - GitHub * **dev**: Gate config conversion tests (vectordotdev#18698) [c35ae64](answerbook/vector@c35ae64) - GitHub * **dev**: update environment for website development (vectordotdev#18657) [4327776](answerbook/vector@4327776) - GitHub * **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [ed97f0d](answerbook/vector@ed97f0d) - GitHub * **docker source**: do not emit component error for out of order logs (vectordotdev#18649) [93d7af4](answerbook/vector@93d7af4) - GitHub * **http_server source**: panic when http server receives metric events (vectordotdev#18781) [b107ff7](answerbook/vector@b107ff7) - GitHub * **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [a1863e6](answerbook/vector@a1863e6) - GitHub * **kafka source**: fix acknowledgement handling during shutdown and rebalance events (vectordotdev#17497) [f2efb1a](answerbook/vector@f2efb1a) - GitHub * **loki sink**: update to use the global list of compression algorithms (vectordotdev#19099) [218963a](answerbook/vector@218963a) - Jesse Szwedko * **releasing**: Update cargo-deb (vectordotdev#19009) [88194e7](answerbook/vector@88194e7) - GitHub * **releasing**: Update example YAML config data_dir (vectordotdev#18896) [0b27019](answerbook/vector@0b27019) - GitHub * remove gh token call (vectordotdev#19047) [1c864aa](answerbook/vector@1c864aa) - GitHub * **sources**: emit `ComponentEventsDropped` when source send is cancelled (vectordotdev#18859) [a0e2769](answerbook/vector@a0e2769) - GitHub * **tls**: for incoming connection alpn negotiation should be done using set_alpn_select_callback (vectordotdev#18843) [8a5b67e](answerbook/vector@8a5b67e) - GitHub ### Chores * Add SHA256 checksums file to GH releases (vectordotdev#18701) [18f07a0](answerbook/vector@18f07a0) - GitHub * **ci**: Add a summary if the regression workflow is skipped (vectordotdev#18724) [17bd2b1](answerbook/vector@17bd2b1) - GitHub * **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [3ade682](answerbook/vector@3ade682) - GitHub * **ci**: add dependabot group for futures (vectordotdev#18954) [c23efce](answerbook/vector@c23efce) - GitHub * **ci**: Bump aws-actions/amazon-ecr-login from 1 to 2 (vectordotdev#18752) [8e2032c](answerbook/vector@8e2032c) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 4.0.0 to 4.0.1 (vectordotdev#18771) [c9804f0](answerbook/vector@c9804f0) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (vectordotdev#18783) [a784018](answerbook/vector@a784018) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (vectordotdev#18866) [811b7f7](answerbook/vector@811b7f7) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (vectordotdev#18981) [72560b1](answerbook/vector@72560b1) - GitHub * **ci**: Bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18723) [f98cd5d](answerbook/vector@f98cd5d) - GitHub * **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [5bf18df](answerbook/vector@5bf18df) - GitHub * **ci**: Bump tspascoal/get-user-teams-membership from 2 to 3 (vectordotdev#18808) [4701bb9](answerbook/vector@4701bb9) - GitHub * **ci**: filter team members from gardener issue comment workflow (vectordotdev#18915) [bf56ac5](answerbook/vector@bf56ac5) - GitHub * **ci**: Fix cookie banner style issues (vectordotdev#18745) [7a55e54](answerbook/vector@7a55e54) - GitHub * **ci**: Remove unusued Dockerfile (vectordotdev#18824) [9d1a676](answerbook/vector@9d1a676) - GitHub * **ci**: Revet bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18742) [7dce292](answerbook/vector@7dce292) - GitHub * **ci**: Run deny check nightly instead of on every PR (vectordotdev#18799) [ae117dc](answerbook/vector@ae117dc) - GitHub * **ci**: temporarily peg greptimedb to `v0.4.0` to unblock CI (vectordotdev#18838) [0776cc0](answerbook/vector@0776cc0) - GitHub * convert test config to yaml (vectordotdev#18856) [8b00214](answerbook/vector@8b00214) - GitHub * **core**: Add a CLI flag to allow for empty configs (vectordotdev#19021) [df4921b](answerbook/vector@df4921b) - GitHub * **core**: add more event metadata to proto (vectordotdev#18816) [2deeba1](answerbook/vector@2deeba1) - GitHub * **core**: Refactor `vector-core::stream` into its own package (vectordotdev#18900) [96f4d73](answerbook/vector@96f4d73) - GitHub * **core**: Set up internal topology API (vectordotdev#18919) [c9c184e](answerbook/vector@c9c184e) - GitHub * **datadog_metrics sink**: Set partial Origin Metrics in edge cases (vectordotdev#18677) [3dab239](answerbook/vector@3dab239) - GitHub * **datadog_metrics sink**: support and migrate to the `v2` series API endpoint (vectordotdev#18761) [3485f2c](answerbook/vector@3485f2c) - GitHub * **datadog**: remove deprecated config options (vectordotdev#18940) [f42751d](answerbook/vector@f42751d) - GitHub * **deps**: Add more `dependabot` groups (vectordotdev#18719) [d0e605e](answerbook/vector@d0e605e) - GitHub * **deps**: Bump @babel/traverse from 7.17.0 to 7.23.2 in /website (vectordotdev#18852) [85d2f17](answerbook/vector@85d2f17) - GitHub * **deps**: Bump apache-avro from 0.15.0 to 0.16.0 (vectordotdev#18685) [b37ce3c](answerbook/vector@b37ce3c) - GitHub * **deps**: Bump async-compression from 0.4.3 to 0.4.4 (vectordotdev#18848) [1a8a8cc](answerbook/vector@1a8a8cc) - GitHub * **deps**: Bump async-graphql from 5.0.10 to 6.0.9 (vectordotdev#18988) [4a4eb61](answerbook/vector@4a4eb61) - GitHub * **deps**: Bump async-nats from 0.32.0 to 0.32.1 (vectordotdev#18735) [047c772](answerbook/vector@047c772) - GitHub * **deps**: Bump async-trait from 0.1.73 to 0.1.74 (vectordotdev#18849) [8d82257](answerbook/vector@8d82257) - GitHub * **deps**: Bump base64 from 0.21.4 to 0.21.5 (vectordotdev#18907) [e754dee](answerbook/vector@e754dee) - GitHub * **deps**: Bump bitmask-enum from 2.2.2 to 2.2.3 (vectordotdev#19057) [7a16ee2](answerbook/vector@7a16ee2) - GitHub * **deps**: Bump bstr from 1.6.2 to 1.7.0 (vectordotdev#18810) [91221c6](answerbook/vector@91221c6) - GitHub * **deps**: Bump cached from 0.45.1 to 0.46.0 (vectordotdev#18660) [331c5a0](answerbook/vector@331c5a0) - GitHub * **deps**: Bump cached from 0.46.0 to 0.46.1 (vectordotdev#19058) [51c6b57](answerbook/vector@51c6b57) - GitHub * **deps**: Bump cargo_toml from 0.16.3 to 0.17.0 (vectordotdev#18978) [b55e436](answerbook/vector@b55e436) - GitHub * **deps**: Bump chrono from 0.4.30 to 0.4.31 (vectordotdev#18583) [052ed98](answerbook/vector@052ed98) - GitHub * **deps**: Bump chrono-tz from 0.8.3 to 0.8.4 (vectordotdev#18979) [7f44b4c](answerbook/vector@7f44b4c) - GitHub * **deps**: Bump clap from 4.4.5 to 4.4.6 (vectordotdev#18715) [4d98fdf](answerbook/vector@4d98fdf) - GitHub * **deps**: Bump clap_complete from 4.4.2 to 4.4.3 (vectordotdev#18716) [27b2c93](answerbook/vector@27b2c93) - GitHub * **deps**: Bump console-subscriber from 0.1.10 to 0.2.0 (vectordotdev#18732) [eda0378](answerbook/vector@eda0378) - GitHub * **deps**: Bump csv from 1.2.2 to 1.3.0 (vectordotdev#18768) [7cb8b52](answerbook/vector@7cb8b52) - GitHub * **deps**: Bump dd-rust-license-tool to 1.0.2 (vectordotdev#18711) [570bd52](answerbook/vector@570bd52) - GitHub * **deps**: Bump dyn-clone from 1.0.14 to 1.0.16 (vectordotdev#19040) [8ba28e0](answerbook/vector@8ba28e0) - GitHub * **deps**: Bump fakedata_generator from 0.2.4 to 0.4.0 (vectordotdev#18910) [7debc60](answerbook/vector@7debc60) - GitHub * **deps**: Bump flate2 from 1.0.27 to 1.0.28 (vectordotdev#18850) [b3889bc](answerbook/vector@b3889bc) - GitHub * **deps**: Bump futures-util from 0.3.28 to 0.3.29 (vectordotdev#18951) [741aec3](answerbook/vector@741aec3) - GitHub * **deps**: Bump goauth from 0.13.1 to 0.14.0 (vectordotdev#18872) [1913ee5](answerbook/vector@1913ee5) - GitHub * **deps**: Bump hashbrown from 0.14.0 to 0.14.1 (vectordotdev#18731) [31d92c2](answerbook/vector@31d92c2) - GitHub * **deps**: Bump hashbrown from 0.14.1 to 0.14.2 (vectordotdev#18893) [aebe8db](answerbook/vector@aebe8db) - GitHub * **deps**: Bump indexmap from 2.0.0 to 2.0.1 (vectordotdev#18705) [65643da](answerbook/vector@65643da) - GitHub * **deps**: Bump indexmap from 2.0.1 to 2.0.2 (vectordotdev#18737) [f47df40](answerbook/vector@f47df40) - GitHub * **deps**: Bump inventory from 0.3.12 to 0.3.13 (vectordotdev#19024) [72eacf5](answerbook/vector@72eacf5) - GitHub * **deps**: Bump lading to 0.19.1 (vectordotdev#18869) [2d7c1bb](answerbook/vector@2d7c1bb) - GitHub * **deps**: Bump libc from 0.2.148 to 0.2.149 (vectordotdev#18800) [4002ef0](answerbook/vector@4002ef0) - GitHub * **deps**: Bump libc from 0.2.149 to 0.2.150 (vectordotdev#19059) [611a652](answerbook/vector@611a652) - GitHub * **deps**: Bump lru from 0.11.1 to 0.12.0 (vectordotdev#18767) [c531a3b](answerbook/vector@c531a3b) - GitHub * **deps**: Bump memchr from 2.6.3 to 2.6.4 (vectordotdev#18736) [c7482d0](answerbook/vector@c7482d0) - GitHub * **deps**: Bump memmap2 from 0.7.1 to 0.8.0 (vectordotdev#18659) [ca9e5b4](answerbook/vector@ca9e5b4) - GitHub * **deps**: Bump memmap2 from 0.8.0 to 0.9.0 (vectordotdev#18765) [dcbbb9b](answerbook/vector@dcbbb9b) - GitHub * **deps**: Bump mongodb from 2.6.1 to 2.7.0 (vectordotdev#18703) [539a40f](answerbook/vector@539a40f) - GitHub * **deps**: Bump mongodb from 2.7.0 to 2.7.1 (vectordotdev#19023) [602f630](answerbook/vector@602f630) - GitHub * **deps**: Bump num_enum from 0.7.0 to 0.7.1 (vectordotdev#18975) [c9b6d45](answerbook/vector@c9b6d45) - GitHub * **deps**: Bump num-traits from 0.2.16 to 0.2.17 (vectordotdev#18802) [c4fbc25](answerbook/vector@c4fbc25) - GitHub * **deps**: Bump OpenDAL to v0.41 (vectordotdev#19039) [5655f76](answerbook/vector@5655f76) - GitHub * **deps**: Bump openssl from 0.10.57 to 0.10.58 (vectordotdev#19025) [c4f2d0e](answerbook/vector@c4f2d0e) - GitHub * **deps**: Bump openssl from 0.10.58 to 0.10.59 (vectordotdev#19054) [ee232f8](answerbook/vector@ee232f8) - GitHub * **deps**: Bump openssl-src from 300.1.5+3.1.3 to 300.1.6+3.1.4 (vectordotdev#18936) [a75a043](answerbook/vector@a75a043) - GitHub * **deps**: Bump ordered-float from 4.1.0 to 4.1.1 (vectordotdev#18818) [99643ca](answerbook/vector@99643ca) - GitHub * **deps**: Bump postcss from 8.4.6 to 8.4.31 in /website (vectordotdev#18750) [92d2be9](answerbook/vector@92d2be9) - GitHub * **deps**: Bump proc-macro2 from 1.0.67 to 1.0.69 (vectordotdev#18803) [bc3b3a2](answerbook/vector@bc3b3a2) - GitHub * **deps**: Bump proptest from 1.2.0 to 1.3.1 (vectordotdev#18738) [2b15c63](answerbook/vector@2b15c63) - GitHub * **deps**: Bump pulsar from 6.0.1 to 6.1.0 (vectordotdev#19004) [43f5913](answerbook/vector@43f5913) - GitHub * **deps**: Bump quanta from 0.11.1 to 0.12.0 (vectordotdev#18774) [e9d2dae](answerbook/vector@e9d2dae) - GitHub * **deps**: Bump quanta from 0.12.0 to 0.12.1 (vectordotdev#19005) [051de5a](answerbook/vector@051de5a) - GitHub * **deps**: Bump ratatui from 0.23.0 to 0.24.0 (vectordotdev#18908) [8b56a93](answerbook/vector@8b56a93) - GitHub * **deps**: Bump regex from 1.10.0 to 1.10.2 (vectordotdev#18858) [434849d](answerbook/vector@434849d) - GitHub * **deps**: Bump regex from 1.9.5 to 1.9.6 (vectordotdev#18739) [8e98321](answerbook/vector@8e98321) - GitHub * **deps**: Bump regex from 1.9.6 to 1.10.0 (vectordotdev#18812) [4d02abf](answerbook/vector@4d02abf) - GitHub * **deps**: Bump reqwest from 0.11.20 to 0.11.22 (vectordotdev#18760) [bac60ad](answerbook/vector@bac60ad) - GitHub * **deps**: Bump rustix from 0.37.19 to 0.37.25 (vectordotdev#18879) [3ca32b8](answerbook/vector@3ca32b8) - GitHub * **deps**: Bump semver from 1.0.18 to 1.0.19 (vectordotdev#18662) [aca7753](answerbook/vector@aca7753) - GitHub * **deps**: Bump semver from 1.0.19 to 1.0.20 (vectordotdev#18811) [d9aca80](answerbook/vector@d9aca80) - GitHub * **deps**: Bump serde from 1.0.188 to 1.0.189 (vectordotdev#18834) [0d09898](answerbook/vector@0d09898) - GitHub * **deps**: Bump serde from 1.0.189 to 1.0.190 (vectordotdev#18945) [73afddb](answerbook/vector@73afddb) - GitHub * **deps**: Bump serde_with from 3.3.0 to 3.4.0 (vectordotdev#18874) [33243ac](answerbook/vector@33243ac) - GitHub * **deps**: Bump serde_yaml from 0.9.25 to 0.9.27 (vectordotdev#18956) [2ee96b1](answerbook/vector@2ee96b1) - GitHub * **deps**: Bump serde-toml-merge from 0.3.2 to 0.3.3 (vectordotdev#18804) [abb9101](answerbook/vector@abb9101) - GitHub * **deps**: Bump serde-wasm-bindgen from 0.6.0 to 0.6.1 (vectordotdev#18935) [ffed6f7](answerbook/vector@ffed6f7) - GitHub * **deps**: Bump sha2 from 0.10.7 to 0.10.8 (vectordotdev#18684) [87af0bd](answerbook/vector@87af0bd) - GitHub * **deps**: Bump socket2 from 0.5.4 to 0.5.5 (vectordotdev#18902) [691fdca](answerbook/vector@691fdca) - GitHub * **deps**: Bump syn from 2.0.37 to 2.0.38 (vectordotdev#18789) [ec5238e](answerbook/vector@ec5238e) - GitHub * **deps**: Bump syn from 2.0.38 to 2.0.39 (vectordotdev#19056) [0f0a0b4](answerbook/vector@0f0a0b4) - GitHub * **deps**: Bump tempfile from 3.6.0 to 3.8.0 (vectordotdev#18686) [c0d24b9](answerbook/vector@c0d24b9) - GitHub * **deps**: Bump tempfile from 3.8.0 to 3.8.1 (vectordotdev#18957) [40961ed](answerbook/vector@40961ed) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#18773) [0e61acc](answerbook/vector@0e61acc) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#19052) [53ffbc3](answerbook/vector@53ffbc3) - GitHub * **deps**: Bump the clap group with 1 update (vectordotdev#18906) [cddb835](answerbook/vector@cddb835) - GitHub * **deps**: Bump the clap group with 2 updates (vectordotdev#18925) [78934c2](answerbook/vector@78934c2) - GitHub * **deps**: Bump the futures group with 1 update (vectordotdev#18961) [0a5e3db](answerbook/vector@0a5e3db) - GitHub * **deps**: Bump the tonic group with 2 updates (vectordotdev#18714) [c95df7c](answerbook/vector@c95df7c) - GitHub * **deps**: Bump the zstd group with 1 update (vectordotdev#18826) [96ef9ee](answerbook/vector@96ef9ee) - GitHub * **deps**: Bump thiserror from 1.0.48 to 1.0.49 (vectordotdev#18683) [6b92a83](answerbook/vector@6b92a83) - GitHub * **deps**: Bump thiserror from 1.0.49 to 1.0.50 (vectordotdev#18892) [16df7ea](answerbook/vector@16df7ea) - GitHub * **deps**: Bump tokio from 1.32.0 to 1.33.0 (vectordotdev#18809) [76971bd](answerbook/vector@76971bd) - GitHub * **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [8abed12](answerbook/vector@8abed12) - GitHub * **deps**: Bump toml from 0.8.0 to 0.8.1 (vectordotdev#18687) [efbe673](answerbook/vector@efbe673) - GitHub * **deps**: Bump toml from 0.8.1 to 0.8.2 (vectordotdev#18747) [f9e51e1](answerbook/vector@f9e51e1) - GitHub * **deps**: Bump toml from 0.8.2 to 0.8.3 (vectordotdev#18909) [22402ca](answerbook/vector@22402ca) - GitHub * **deps**: Bump toml from 0.8.3 to 0.8.4 (vectordotdev#18913) [249330a](answerbook/vector@249330a) - GitHub * **deps**: Bump toml from 0.8.4 to 0.8.5 (vectordotdev#18950) [4a525a8](answerbook/vector@4a525a8) - GitHub * **deps**: Bump toml from 0.8.5 to 0.8.6 (vectordotdev#18962) [5e7ae83](answerbook/vector@5e7ae83) - GitHub * **deps**: Bump tracing-log from 0.1.3 to 0.1.4 (vectordotdev#18914) [e4fd78c](answerbook/vector@e4fd78c) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump tracing-log from 0.1.4 to 0.2.0 (vectordotdev#18941) [30a1e26](answerbook/vector@30a1e26) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump trust-dns-proto from 0.23.0 to 0.23.1 (vectordotdev#18846) [0568d7a](answerbook/vector@0568d7a) - GitHub * **deps**: Bump trust-dns-proto from 0.23.1 to 0.23.2 (vectordotdev#18911) [1eaf8b1](answerbook/vector@1eaf8b1) - GitHub * **deps**: Bump uuid from 1.4.1 to 1.5.0 (vectordotdev#18880) [a025caa](answerbook/vector@a025caa) - GitHub * **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [decaaeb](answerbook/vector@decaaeb) - GitHub * **deps**: Bump wasm-bindgen from 0.2.87 to 0.2.88 (vectordotdev#19026) [63bb9e4](answerbook/vector@63bb9e4) - GitHub * **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [4295985](answerbook/vector@4295985) - GitHub * **deps**: Bump wiremock from 0.5.19 to 0.5.21 (vectordotdev#19055) [4622ef6](answerbook/vector@4622ef6) - GitHub * **deps**: Bump zerocopy from 0.7.21 to 0.7.31 (vectordotdev#19394) [74d6cb1](answerbook/vector@74d6cb1) - Jesse Szwedko * **deps**: clean up VRL crate features (vectordotdev#18740) [1452d54](answerbook/vector@1452d54) - GitHub * **deps**: Group csv crate updates (vectordotdev#18797) [98aa157](answerbook/vector@98aa157) - GitHub * **deps**: Remove usages of atty (vectordotdev#18985) [371580c](answerbook/vector@371580c) - GitHub * **deps**: Update dependencies (vectordotdev#18971) [3ead10f](answerbook/vector@3ead10f) - GitHub * **deps**: Update lading to 0.19.0 (vectordotdev#18861) [dc729f5](answerbook/vector@dc729f5) - GitHub * **deps**: Update license-tool.toml webpki version (vectordotdev#18986) [0051ec0](answerbook/vector@0051ec0) - GitHub * **deps**: Update VRL to 0.8.1 (vectordotdev#19011) [f5ea285](answerbook/vector@f5ea285) - GitHub * **dev**: Add `vector-lib` wrapper for three more libs (vectordotdev#18992) [1eb418b](answerbook/vector@1eb418b) - GitHub * **dev**: Add wrapper for `codecs` to `vector-lib` (vectordotdev#18959) [5f30f74](answerbook/vector@5f30f74) - GitHub * **dev**: Add wrapper for `enrichment` to `vector-lib` (vectordotdev#18977) [e61f308](answerbook/vector@e61f308) - GitHub * **dev**: Add wrapper for `file-source` in `vector-lib` (vectordotdev#18984) [f44da16](answerbook/vector@f44da16) - GitHub * **dev**: Add wrapper for `lookup` in `vector-lib` (vectordotdev#18995) [164f1e9](answerbook/vector@164f1e9) - GitHub * **dev**: Add wrapper for `vector-buffers` to `vector-lib` (vectordotdev#18964) [2cef62c](answerbook/vector@2cef62c) - GitHub * **dev**: Detail the format of DEPRECATIONS.md file (vectordotdev#19016) [223dd7b](answerbook/vector@223dd7b) - GitHub * **dev**: Move some macros into `vector-core` (vectordotdev#19002) [4f613ce](answerbook/vector@4f613ce) - GitHub * **dev**: Remove deprecation action item for armv7 RPMs (vectordotdev#19018) [ff7b95f](answerbook/vector@ff7b95f) - GitHub * **dev**: Set up `vector-lib` wrapper crate with `vector-common` (vectordotdev#18927) [00c40d7](answerbook/vector@00c40d7) - GitHub * **dev**: Wrap `vector-config` in `vector-lib` as `configurable` (vectordotdev#18944) [42beb3f](answerbook/vector@42beb3f) - GitHub * **dev**: Wrap `vector-core` in `vector-lib` (vectordotdev#18934) [270fdfd](answerbook/vector@270fdfd) - GitHub * **dev**: Wrap `vector-stream` in `vector-lib` (vectordotdev#18953) [8a02b16](answerbook/vector@8a02b16) - GitHub * **docs**: Add alpha to traces and beta to metrics in descriptions (vectordotdev#19139) [1b9fb9b](answerbook/vector@1b9fb9b) - Jesse Szwedko * **docs**: add highlight post for secrets in disk buffers (vectordotdev#18994) [0cc9389](answerbook/vector@0cc9389) - GitHub * **docs**: Add spec for `listen` option (vectordotdev#18080) [29e5e22](answerbook/vector@29e5e22) - GitHub * **docs**: Replace setup.vector.dev references (vectordotdev#19080) [def235e](answerbook/vector@def235e) - Jesse Szwedko * **docs**: update a few more examples to YAML (vectordotdev#19103) [a59329a](answerbook/vector@a59329a) - Jesse Szwedko * **docs**: update quickstart.md to use YAML (vectordotdev#18796) [0e76fe0](answerbook/vector@0e76fe0) - GitHub * **external docs**: First batch of editorial edits for the Functions doc (vectordotdev#18780) [1da9005](answerbook/vector@1da9005) - GitHub * **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [7ad4112](answerbook/vector@7ad4112) - GitHub * Follow redirects for `sh.vector.dev` (vectordotdev#19000) [9893b86](answerbook/vector@9893b86) - GitHub * **gcp_stackdriver_metrics sink**: rewrite to stream based sink (vectordotdev#18749) [92268e4](answerbook/vector@92268e4) - GitHub * **kubernetes**: Regenerate manifests from 0.27.0 chart (vectordotdev#19001) [5e9dd1d](answerbook/vector@5e9dd1d) - GitHub * **metrics**: improve creation of Origin metadata structures (vectordotdev#18788) [f0adce7](answerbook/vector@f0adce7) - GitHub * Note the version to remove the v1 metrics support from the Datadog Metrics sink (vectordotdev#19017) [be9f229](answerbook/vector@be9f229) - GitHub * **observability, blackhole sink**: Don't report by default (vectordotdev#18963) [3b85b48](answerbook/vector@3b85b48) - GitHub * **observability**: deprecate obsolete http metrics (vectordotdev#18972) [f33dce2](answerbook/vector@f33dce2) - GitHub * **observability**: fix tokio unstable (vectordotdev#18776) [67c4beb](answerbook/vector@67c4beb) - GitHub * **observability**: remove `peer_addr` internal metric tag (vectordotdev#18982) [b9447f6](answerbook/vector@b9447f6) - GitHub * **observability**: remove deprecated `component_name` metric tag (vectordotdev#18942) [c6f5d2b](answerbook/vector@c6f5d2b) - GitHub * **observability**: remove metrics replaced by component_errors_total (vectordotdev#18965) [17f4ed2](answerbook/vector@17f4ed2) - GitHub * **prometheus_remote_write sink**: remote write sink rewrite (vectordotdev#18676) [5c1707f](answerbook/vector@5c1707f) - GitHub * **regression**: Unmark regression tests as erratic now (vectordotdev#19020) [2cdf654](answerbook/vector@2cdf654) - GitHub * **releasing, kubernetes**: Update manifests to v0.26.0 of the chart (vectordotdev#18694) [5f4c3ba](answerbook/vector@5f4c3ba) - GitHub * **releasing**: Add deprecation note about respositories.timber.io deprecation (vectordotdev#19078) [09df599](answerbook/vector@09df599) - Jesse Szwedko * **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [ff745ab](answerbook/vector@ff745ab) - GitHub * **releasing**: Add known issue for Datadog Metrics sink in v0.34.0 (vectordotdev#19122) [cee9d07](answerbook/vector@cee9d07) - Jesse Szwedko * **releasing**: Add known issue for protobuf encoder in v0.34.0 (vectordotdev#19244) [f7c3824](answerbook/vector@f7c3824) - Jesse Szwedko * **releasing**: Add upgrade note about TOML breaking change to v0.34.0 (vectordotdev#19120) [dba0ba1](answerbook/vector@dba0ba1) - Jesse Szwedko * **releasing**: Bump Vector version to v0.34.0 (vectordotdev#18693) [1c2f970](answerbook/vector@1c2f970) - GitHub * **releasing**: Fix changelog not for kafka fix in 0.33.1 (vectordotdev#19032) [2501049](answerbook/vector@2501049) - GitHub * **releasing**: Fix formatting for v0.34.0 release note (vectordotdev#19085) [3569271](answerbook/vector@3569271) - Jesse Szwedko * **releasing**: Prepare v0.33.0 release [682f0e0](answerbook/vector@682f0e0) - Jesse Szwedko * **releasing**: Prepare v0.33.1 release [409b69d](answerbook/vector@409b69d) - Jesse Szwedko * **releasing**: Prepare v0.34.0 release [c909b66](answerbook/vector@c909b66) - Jesse Szwedko * **releasing**: Prepare v0.34.1 release [86f1c22](answerbook/vector@86f1c22) - Jesse Szwedko * **releasing**: Prepare v0.34.2 release [d685a16](answerbook/vector@d685a16) - Jesse Szwedko * **releasing**: Typo in v0.33.1 release docs (vectordotdev#18987) [36974a0](answerbook/vector@36974a0) - GitHub * Remove @spencergilbert from CODEOWNERS (vectordotdev#18778) [f300c85](answerbook/vector@f300c85) - GitHub * Remove armv7 RPM package (vectordotdev#18837) [eca8c76](answerbook/vector@eca8c76) - GitHub * remove config/vector.toml (vectordotdev#18833) [efb0d1a](answerbook/vector@efb0d1a) - GitHub * **security**: Ignore RUSTSEC-2023-0071 for now (vectordotdev#19263) [e27b7bd](answerbook/vector@e27b7bd) - Jesse Szwedko * **security**: Remove legacy OpenSSL provider flags (vectordotdev#19015) [2bba40a](answerbook/vector@2bba40a) - GitHub * **sinks**: Update `PartitionBatcher` to use `BatchConfig` (vectordotdev#18792) [4a7d0c3](answerbook/vector@4a7d0c3) - GitHub * Update *-release.md issue templates for vector.dev package release (vectordotdev#18814) [ab8f8d2](answerbook/vector@ab8f8d2) - GitHub * **vrl**: Revive old remap tests (vectordotdev#18678) [53cad38](answerbook/vector@53cad38) - GitHub * **website**: Set download page dropdown to latest version (vectordotdev#18758) [23745f2](answerbook/vector@23745f2) - GitHub * **websites**: Setup preview site workflows (vectordotdev#18924) [9d006c7](answerbook/vector@9d006c7) - GitHub * **websites**: Workflow updates (vectordotdev#19036) [b4ca866](answerbook/vector@b4ca866) - GitHub * **website**: WEB-4247 | Update references from s3 to setup.vector.dev (vectordotdev#19149) [9e1ad37](answerbook/vector@9e1ad37) - Jesse Szwedko * **website**: WEB-4275 | Update Navigation (vectordotdev#19186) [73a668c](answerbook/vector@73a668c) - Doug Smith * **website**: Workflow fixes (vectordotdev#19046) [fb63f8e](answerbook/vector@fb63f8e) - GitHub ### Continuous Integration * mark otlp_http_to_blackhole experiment erratic (vectordotdev#18786) [54b54a5](answerbook/vector@54b54a5) - GitHub ### Features * add PR comment trigger for the workload checks workflow (vectordotdev#18839) [11bc5d9](answerbook/vector@11bc5d9) - GitHub * **amqp**: added integration test for TLS (vectordotdev#18813) [3c4ae86](answerbook/vector@3c4ae86) - GitHub [LOG-16435](https://logdna.atlassian.net/browse/LOG-16435) * **ci**: Add Vector workload checks (vectordotdev#18569) [e2b7de0](answerbook/vector@e2b7de0) - GitHub * **codecs**: add support for protobuf encoding (vectordotdev#18598) [737f5c3](answerbook/vector@737f5c3) - GitHub * **docs**: add fallibility examples (vectordotdev#18931) [08b45a5](answerbook/vector@08b45a5) - GitHub * **journald source**: Add emit_cursor option (vectordotdev#18882) [74051dc](answerbook/vector@74051dc) - GitHub * **regression**: convert all regression cases configs to YAML (vectordotdev#18825) [1fb0f0d](answerbook/vector@1fb0f0d) - GitHub * **timestamp encoding**: add unixtime formats (vectordotdev#18817) [53039e7](answerbook/vector@53039e7) - GitHub ### Miscellaneous * Merge pull request vectordotdev#396 from answerbook/feature/LOG-18978 [2b046bf](answerbook/vector@2b046bf) - GitHub [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Merge tag 'v0.34.2' into feature/LOG-18978 [a88d5ed](answerbook/vector@a88d5ed) - Darin Spivey [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Update RUM domain (vectordotdev#19367) [4acb9de](answerbook/vector@4acb9de) - Jesse Szwedko * chore(docs):Add Obs Pipelines to docs (vectordotdev#19201) [8f3f160](answerbook/vector@8f3f160) - Jesse Szwedko * updating the doc, 2 urls were 404 (vectordotdev#18949) [bf58b06](answerbook/vector@bf58b06) - GitHub * chore!(config, docs): delete deprecated vector.toml code (vectordotdev#18795) [c8557d0](answerbook/vector@c8557d0) - GitHub * [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [43428d3](answerbook/vector@43428d3) - GitHub * Add announcement for new repository URLs (vectordotdev#18798) [239cf94](answerbook/vector@239cf94) - GitHub * **examples**: Convert config/examples from TOML to YAML (vectordotdev#18832) [6ffb072](answerbook/vector@6ffb072) - GitHub * **external docs**: Fix metrics test example (vectordotdev#18725) [96def01](answerbook/vector@96def01) - GitHub * fix truncate arguments (vectordotdev#19068) [9a5cb51](answerbook/vector@9a5cb51) - Jesse Szwedko * **gcp_pubsub source**: Add required fields to documentation examples (vectordotdev#18998) [21f741d](answerbook/vector@21f741d) - GitHub * **nats source**: add subscriber_capacity option (vectordotdev#18899) [e7b563d](answerbook/vector@e7b563d) - GitHub * **sources, sinks**: add telemetry to http and grpc servers (vectordotdev#18887) [e779019](answerbook/vector@e779019) - GitHub * **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [625e4bd](answerbook/vector@625e4bd) - GitHub * **vrl**: add an example of parsing upstreaminfo with parse_nginx_log (vectordotdev#18815) [774094e](answerbook/vector@774094e) - GitHub ### **BREAKING CHANGES** * **security:** Remove legacy OpenSSL provider flags (vectordotdev#19015) * **observability:** remove metrics replaced by component_errors_total (vectordotdev#18965) * **observability:** remove `peer_addr` internal metric tag (vectordotdev#18982) * **observability, blackhole sink:** Don't report by default (vectordotdev#18963) * **observability:** remove deprecated `component_name` metric tag (vectordotdev#18942) * **datadog:** remove deprecated config options (vectordotdev#18940)

feat(codecs): add support for protobuf encoding

2a0754e

goakley requested review from a team and neuronull as code owners September 18, 2023 18:32

goakley requested a review from a team September 18, 2023 18:32

github-actions bot added domain: sinks Anything related to the Vector's sinks domain: codecs Anything related to Vector's codecs (encoding/decoding) domain: external docs Anything related to Vector's external, public documentation labels Sep 18, 2023

goakley mentioned this pull request Sep 18, 2023

New gcp_big_query sink #1536

Open

6 tasks

jszwedko requested a review from pront September 18, 2023 18:46

pront requested changes Sep 19, 2023

View reviewed changes

goakley added 9 commits September 20, 2023 18:46

rename descriptor_set_path to desc_file

01171df

fix spelling errors

9241230

describe encode_message

6a1c01e

factor out common test code

5a42f89

move protobuf test data to one directory

d17a683

create common protobuf module

e8758d4

add map encoding and test

7be964b

add enum encoding and test

af5a4e9

add timestamp encoding and test

1d3a498

goakley requested a review from a team as a code owner September 21, 2023 17:43

fix spelling again

82aac3c

goakley requested a review from pront September 21, 2023 17:52

maycmlee approved these changes Sep 21, 2023

View reviewed changes

pront approved these changes Sep 21, 2023

View reviewed changes

goakley added 4 commits September 26, 2023 22:39

assert -> assert_eq

473177c

remove simple comments

d16507e

enums are not case-sensitive

3759e34

add round trip test

331542f

goakley requested a review from pront September 27, 2023 00:18

pront self-assigned this Sep 27, 2023

pront reviewed Sep 27, 2023

View reviewed changes

goakley added 2 commits September 27, 2023 17:48

spelling (maybe?)

0abcc4c

fix clippy errors

62b1f38

goakley requested a review from pront September 27, 2023 23:18

pront approved these changes Sep 28, 2023

View reviewed changes

jszwedko approved these changes Sep 28, 2023

View reviewed changes

pront added this pull request to the merge queue Sep 29, 2023

dsmith3197 removed this pull request from the merge queue due to a manual request Sep 29, 2023

dsmith3197 added this pull request to the merge queue Sep 29, 2023

Merged via the queue into vectordotdev:master with commit 737f5c3 Sep 29, 2023

goakley deleted the codecs/protobuf-serializer branch September 29, 2023 21:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(codecs): add support for protobuf encoding #18598

feat(codecs): add support for protobuf encoding #18598

goakley commented Sep 18, 2023

netlify bot commented Sep 18, 2023 •

edited

Loading

netlify bot commented Sep 18, 2023 •

edited

Loading

aws-amplify-us-east-1 bot commented Sep 18, 2023

aws-amplify-us-east-1 bot commented Sep 18, 2023

pront left a comment •

edited

Loading

pront Sep 19, 2023

goakley Sep 21, 2023

pront Sep 21, 2023

goakley Sep 27, 2023

pront Sep 19, 2023

aws-amplify-us-east-1 bot commented Sep 21, 2023

goakley commented Sep 21, 2023

maycmlee left a comment

pront left a comment

pront Sep 21, 2023

pront Sep 21, 2023

pront Sep 21, 2023

goakley Sep 27, 2023

pront commented Sep 21, 2023

pront Sep 27, 2023

pront Sep 27, 2023

pront Sep 27, 2023

pront commented Sep 28, 2023

goakley commented Sep 28, 2023

jszwedko left a comment

github-actions bot commented Sep 29, 2023

github-actions bot commented Sep 29, 2023

pront commented Oct 3, 2023

		@@ -0,0 +1,469 @@
		use crate::common::protobuf::get_message_descriptor;

		@@ -0,0 +1,67 @@
		//! Tests for the behaviour of Protobuf serializer and deserializer (together).

feat(codecs): add support for protobuf encoding #18598

feat(codecs): add support for protobuf encoding #18598

Conversation

goakley commented Sep 18, 2023

netlify bot commented Sep 18, 2023 • edited Loading

✅ Deploy Preview for vrl-playground ready!

netlify bot commented Sep 18, 2023 • edited Loading

✅ Deploy Preview for vector-project canceled.

aws-amplify-us-east-1 bot commented Sep 18, 2023

aws-amplify-us-east-1 bot commented Sep 18, 2023

pront left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-amplify-us-east-1 bot commented Sep 21, 2023

goakley commented Sep 21, 2023

maycmlee left a comment

Choose a reason for hiding this comment

pront left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pront commented Sep 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pront commented Sep 28, 2023

goakley commented Sep 28, 2023

jszwedko left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 29, 2023

Regression Detector Results

github-actions bot commented Sep 29, 2023

Regression Detector Results

pront commented Oct 3, 2023

netlify bot commented Sep 18, 2023 •

edited

Loading

netlify bot commented Sep 18, 2023 •

edited

Loading

pront left a comment •

edited

Loading