Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement topology-aware validations #235

Closed
lukesteensen opened this issue Apr 2, 2019 · 0 comments · Fixed by #324
Closed

Implement topology-aware validations #235

lukesteensen opened this issue Apr 2, 2019 · 0 comments · Fixed by #324
Assignees
Labels
type: enhancement A value-adding code change that enhances its existing functionality.

Comments

@lukesteensen
Copy link
Member

lukesteensen commented Apr 2, 2019

In contrast with validations that check the configuration of a single source/transform/sink, we should implement validations that look at the topology as a whole.

Implementation-wise, this is going to look a lot like a type checker. Sources are equivalent to fn() -> T, transforms to fn(T) -> U, and sinks to fn(U). The specific types will be different based on how each is configured (i.e. parsers, encoders, etc). Ideally, such a system would allow us to easily catch cases like the following:

  • Sending log data into a sink that expects metrics
  • Sending structured data to a raw sink with no encoder configured
  • Placing a sampler configured to read a specific field after a raw source that can't possibly set that field
  • Placing a parser configured to read a record's original raw data after a source that only provides structured fields

There is potentially a lot of complexity here, so we should probably dedicate some time for additional research, a spike implementation, and a more detailed proposal. There's some temptation to tie these "types" to the variants of our Record/event type (which is reasonable), but it could also be interesting to explore a more field-oriented direction (i.e. structural instead of nominative typing).

@lukesteensen lukesteensen added the type: enhancement A value-adding code change that enhances its existing functionality. label Apr 2, 2019
@binarylogic binarylogic added this to the 0.2 milestone Apr 17, 2019
aholmberg pushed a commit to aholmberg/vector that referenced this issue Feb 14, 2024
…35-tls-in-amqp

feat(sources): enable TLS in AMQP source(sink)

- enabled tls in lapin using openssl feature;
- new tls integration test;
- new dev certs for rabbitmq;

Ref: LOG-16435
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A value-adding code change that enhances its existing functionality.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants