You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In contrast with validations that check the configuration of a single source/transform/sink, we should implement validations that look at the topology as a whole.
Implementation-wise, this is going to look a lot like a type checker. Sources are equivalent to fn() -> T, transforms to fn(T) -> U, and sinks to fn(U). The specific types will be different based on how each is configured (i.e. parsers, encoders, etc). Ideally, such a system would allow us to easily catch cases like the following:
Sending log data into a sink that expects metrics
Sending structured data to a raw sink with no encoder configured
Placing a sampler configured to read a specific field after a raw source that can't possibly set that field
Placing a parser configured to read a record's original raw data after a source that only provides structured fields
There is potentially a lot of complexity here, so we should probably dedicate some time for additional research, a spike implementation, and a more detailed proposal. There's some temptation to tie these "types" to the variants of our Record/event type (which is reasonable), but it could also be interesting to explore a more field-oriented direction (i.e. structural instead of nominative typing).
The text was updated successfully, but these errors were encountered:
…35-tls-in-amqp
feat(sources): enable TLS in AMQP source(sink)
- enabled tls in lapin using openssl feature;
- new tls integration test;
- new dev certs for rabbitmq;
Ref: LOG-16435
In contrast with validations that check the configuration of a single source/transform/sink, we should implement validations that look at the topology as a whole.
Implementation-wise, this is going to look a lot like a type checker. Sources are equivalent to
fn() -> T
, transforms tofn(T) -> U
, and sinks tofn(U)
. The specific types will be different based on how each is configured (i.e. parsers, encoders, etc). Ideally, such a system would allow us to easily catch cases like the following:There is potentially a lot of complexity here, so we should probably dedicate some time for additional research, a spike implementation, and a more detailed proposal. There's some temptation to tie these "types" to the variants of our
Record
/event type (which is reasonable), but it could also be interesting to explore a more field-oriented direction (i.e. structural instead of nominative typing).The text was updated successfully, but these errors were encountered: