Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: BB-310 add oplog populator dashboard #2358

Merged

Conversation

Kerkesni
Copy link
Contributor

@Kerkesni Kerkesni commented Dec 23, 2022

Issue: BB-310

Screenshot 2023-01-04 at 14 09 30
Screenshot 2023-01-04 at 14 09 38
Screenshot 2023-01-04 at 14 09 48

@bert-e
Copy link
Contributor

bert-e commented Dec 23, 2022

Hello kerkesni,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch 2 times, most recently from 55680af to c0f3eea Compare December 23, 2022 11:34
@scality scality deleted a comment from bert-e Dec 23, 2022
@bert-e
Copy link
Contributor

bert-e commented Dec 23, 2022

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/8.3

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

@bert-e
Copy link
Contributor

bert-e commented Dec 23, 2022

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch from c0f3eea to bc54b0a Compare December 23, 2022 13:03
@scality scality deleted a comment from bert-e Dec 23, 2022
@bert-e
Copy link
Contributor

bert-e commented Dec 23, 2022

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/8.3

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

@bert-e
Copy link
Contributor

bert-e commented Dec 23, 2022

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

Copy link
Contributor

@francoisferrand francoisferrand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

monitoring/oplog-populator/pycache/dashboard.cpython-310.pyc should not be commited

@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch from bc54b0a to 43d345a Compare January 4, 2023 12:18
@bert-e
Copy link
Contributor

bert-e commented Jan 4, 2023

History mismatch

Merge commit #003e089189abaad374b1c3c00e37e894ea21f1aa on the integration branch
w/8.5/feature/BB-310-add-oplog-populator-dashboard is merging a branch which is neither the current
branch feature/BB-310-add-oplog-populator-dashboard nor the development branch
development/8.5.

It is likely due to a rebase of the branch feature/BB-310-add-oplog-populator-dashboard and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

@Kerkesni
Copy link
Contributor Author

Kerkesni commented Jan 4, 2023

/reset

@bert-e
Copy link
Contributor

bert-e commented Jan 4, 2023

Reset complete

I have successfully deleted this pull request's integration branches.

@bert-e
Copy link
Contributor

bert-e commented Jan 4, 2023

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/8.3

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

@bert-e
Copy link
Contributor

bert-e commented Jan 4, 2023

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch from 43d345a to 1ff65b0 Compare January 4, 2023 13:09
@bert-e
Copy link
Contributor

bert-e commented Jan 4, 2023

History mismatch

Merge commit #d6223aa2ea40b0d8adfd4d5753b601a648c4f881 on the integration branch
w/8.5/feature/BB-310-add-oplog-populator-dashboard is merging a branch which is neither the current
branch feature/BB-310-add-oplog-populator-dashboard nor the development branch
development/8.5.

It is likely due to a rebase of the branch feature/BB-310-add-oplog-populator-dashboard and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch from 1ff65b0 to c002108 Compare January 5, 2023 10:49
@Kerkesni Kerkesni force-pushed the feature/BB-310-add-oplog-populator-dashboard branch from c002108 to ab6302d Compare January 6, 2023 10:06
Comment on lines +182 to +183
expr='sum (rate(' + Metrics.CONNECTOR_CONFIG_APPLIED('success="false"') + '))',
)])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if it would be more clear for the user to see the actual number of errors instead of the rate.

Also, in the screenshot this graph show "No data". Is this the default behavior when there have been no failures ? (if yes, it's probably better to show the graph with 0)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is actually a developer bias : we usually run with small number of objects, and we want to be precise : so the precise number is what we expect... but in production, precise values are much higher and don't matter so much: what matters is how frequent changes happen...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this makes sense.
What I had in mind was that this should ideally stay to 0 and we will want to pay attention to even a low number of failures.

]

CONNECTORS = metrics.CounterMetric(
'oplog_populator_connectors', 'isOld',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is isOld used somewhere? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not anymore, but the metric does have that property so i think maybe it makes sense to keep it in the definition of the metric ?

fillOpacity=10,
spanNulls=True,
targets=[Target(
expr='sum by(connector) (' + Metrics.BUCKETS.raw() + ')',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding: what is an "expected" number (order of magnitude) of connectors ?
(what I am thinking is that this graph may become hard to read if there are 10s of connectors)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't know yet actually, the number will depend on the performance of the connectors, meaning that if a single connector can handle tens of thousands of buckets at a time we'll probably have few connectors, and it's true the other way around.

There's a ticket for doing that performance test which will also help determine how we manage the connectors, for now the number of configured connectors is a set value that can be configured at Zenko CR level. This might change later on to make it dynamic in case of sudden degradation of performance with the addition of buckets to connectors.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation !

Copy link
Contributor

@williamlardier williamlardier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM beyond the nit comment



bucket_distribution = PieChart(
title='Bucket Distribution',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is misleading, the Bucket from prometheus and the Bucket from S3... Maybe we can have something easier to understand for a not-so-technical reader?

fillOpacity=10,
spanNulls=True,
targets=[Target(
expr='sum by(connector) (' + Metrics.BUCKETS.raw() + ')',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation !

@Kerkesni
Copy link
Contributor Author

Kerkesni commented Jan 6, 2023

/approve

@bert-e
Copy link
Contributor

bert-e commented Jan 6, 2023

History mismatch

Merge commit #d6223aa2ea40b0d8adfd4d5753b601a648c4f881 on the integration branch
w/8.5/feature/BB-310-add-oplog-populator-dashboard is merging a branch which is neither the current
branch feature/BB-310-add-oplog-populator-dashboard nor the development branch
development/8.5.

It is likely due to a rebase of the branch feature/BB-310-add-oplog-populator-dashboard and the
merge is not possible until all related w/* branches are deleted or updated.

Please use the reset command to have me reinitialize these branches.

The following options are set: approve

@Kerkesni
Copy link
Contributor Author

Kerkesni commented Jan 6, 2023

/reset

@bert-e
Copy link
Contributor

bert-e commented Jan 6, 2023

Reset complete

I have successfully deleted this pull request's integration branches.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Jan 6, 2023

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/8.3

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Jan 6, 2023

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/8.4

  • ✔️ development/8.5

  • ✔️ development/8.6

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/8.3

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Jan 6, 2023

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/8.4

  • ✔️ development/8.5

  • ✔️ development/8.6

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/8.3

Please check the status of the associated issue BB-310.

Goodbye kerkesni.

@bert-e bert-e merged commit 6f95321 into development/8.4 Jan 6, 2023
@bert-e bert-e deleted the feature/BB-310-add-oplog-populator-dashboard branch January 6, 2023 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants