Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(layers/prometheus): provide builder APIs #5072

Merged
merged 19 commits into from
Sep 3, 2024

Conversation

koushiro
Copy link
Contributor

@koushiro koushiro commented Aug 30, 2024

Which issue does this PR close?

Related issue: #5069 (comment)
Related PR: #5073

Rationale for this change

Provide similar APIs for PrometheusLayer and PrometheusClientLayer

What changes are included in this PR?

  • add PrometheusLayer::builder
  • add PrometheusLayerBuilder
  • impl Default for PrometheusLayer
  • move path_label_value into observe mod
  • add more examples in doc

Are there any user-facing changes?

API breaking change

core/src/layers/mod.rs Outdated Show resolved Hide resolved
core/src/layers/observe/mod.rs Show resolved Hide resolved
core/src/layers/observe/mod.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Show resolved Hide resolved
/// Ok(())
/// # }
/// ```
pub fn register(self, registry: &Registry) -> Self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this API. PrometheusLayer::new() should be enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, PrometheusLayer::new should just be a helper function that uses the default config and registers the metrics with the default config to the registry.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I don't believe we should perform register during new unless the upstream API forbid it. Creating the layer itself should have no side effects. Everything should occur during the layer.

So PrometheusLayer::new() should just take a registry and return Self. We will perform the registry during Layer::layer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how about PrometheusClientLayer in #5073

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, how about this: Let's use your previous proposed builder's API.

let layer: impl Layer = PrometheusClientLayer::builder()
    .operation_bytes_buckets(vec![1.0, 2.0])
    .register(registry);

PrometheusClientLayer::builder() returns a PrometheusClientLayerBuilder. Users can only use this layer after calling register. We can provide default() if the upstream lib has default registry concept.

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do we need to use similar APIs in PrometheusLayer?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do we need to use similar APIs in PrometheusLayer?

I believe this design can adapt to all metrics libraries. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if you agree with the design, I'll apply it to PrometheusLayer and PrometheusClientLayer.
But I'm not sure if this design will fit well with other metric layers, I'm not very optimistic after using the metrics library.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we can call with_recorder direcly: https://docs.rs/metrics/latest/src/metrics/recorder/mod.rs.html#126

core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
@koushiro koushiro marked this pull request as draft September 3, 2024 07:20
@koushiro koushiro changed the title refactor(layers/prometheus): provide consistent APIs refactor(layers/prometheus): provide builder APIs Sep 3, 2024
@koushiro koushiro marked this pull request as ready for review September 3, 2024 10:06
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! This API design is really clean and easy to use. There are some nits on the public API.

core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
let labels = OperationLabels::names(false, path_label_level);
let operation_duration_seconds = register_histogram_vec_with_registry!(
/// Register the metrics into the given registry and return a [`PrometheusLayer`].
pub fn register(self, registry: &Registry) -> PrometheusLayer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about returning an error instead of just unwrapping?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return prometheus::Error or opendal::Error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opendal only returns opendal::Error, we can wrap it inside.

@koushiro
Copy link
Contributor Author

koushiro commented Sep 3, 2024

There are some weird errors in CI

error[E0282]: type annotations needed
   --> src/services/icloud/core.rs:530:25
    |
530 |             Some(it) => Ok(Some(it.drivewsid.clone())),
    |                         ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
    |
help: consider specifying the generic arguments
    |
530 |             Some(it) => Ok::<std::option::Option<std::string::String>, E>(Some(it.drivewsid.clone())),
    |                           +++++++++++++++++++++++++++++++++++++++++++++++

error[E0283]: type annotations needed
   --> src/services/icloud/core.rs:530:25
    |
530 |             Some(it) => Ok(Some(it.drivewsid.clone())),
    |                         ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
531 |             None => Ok(None),
532 |         }?;
    |          - type must be known at this point
    |
note: multiple `impl`s satisfying `types::error::Error: std::convert::From<_>` found
   --> src/types/error.rs:418:1
    |
418 | impl From<prometheus::Error> for Error {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: and another `impl` found in the `core` crate: `impl<T> std::convert::From<T> for T;`
    = note: required for `std::result::Result<std::option::Option<std::string::String>, types::error::Error>` to implement `std::ops::FromResidual<std::result::Result<std::convert::Infallible, _>>`
help: consider specifying the generic arguments
    |
530 |             Some(it) => Ok::<std::option::Option<std::string::String>, E>(Some(it.drivewsid.clone())),
    |                           +++++++++++++++++++++++++++++++++++++++++++++++

But if I change the icloud code, these errors are gone

-        let id = match node.items.iter().find(|it| it.name == name) {
-            Some(it) => Ok(Some(it.drivewsid.clone())),
-            None => Ok(None),
-        }?;
-        Ok(id)
+       Ok(node
+          .items
+          .iter()
+          .find(|it| it.name == name)
+          .map(|it| it.drivewsid.clone()))

@Xuanwo
Copy link
Member

Xuanwo commented Sep 3, 2024

But if I change the icloud code, these errors are gone

I think it's a new Clippy lint introduced in the latest Rust version. Would you like to submit a new PR to address it?

core/src/types/error.rs Outdated Show resolved Hide resolved
core/src/types/error.rs Outdated Show resolved Hide resolved
@Xuanwo
Copy link
Member

Xuanwo commented Sep 3, 2024

9b5c096 (#5072)

I apologize for not providing a clear review. I suggest adding a parse_prometheus_error(err: prometheus::Error) -> Error as a standalone function in the Prometheus layer module itself, rather than within Error.

In this way, we keep our public types clean and easy to maintain.

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much. We are very close to merging. I found a few places I missed in the previous round.

core/src/layers/prometheus.rs Outdated Show resolved Hide resolved
core/src/layers/prometheus.rs Show resolved Hide resolved
core/src/layers/prometheus.rs Show resolved Hide resolved
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thank you for your patience!

@Xuanwo Xuanwo merged commit 70a5d7e into apache:main Sep 3, 2024
229 checks passed
@koushiro koushiro deleted the prometheus-consistent-apis branch September 3, 2024 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants