Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic RPC metrics via mc support top rpc #5060

Merged
merged 4 commits into from
Oct 14, 2024

Conversation

klauspost
Copy link
Contributor

@klauspost klauspost commented Oct 14, 2024

Description

Pretty basic information right now:

Example:

λ mc support top rpc myminio
      SERVER            CONCTD  PING     PONG   OUT.Q   RECONNS STR.IN  STR.OUT MSG.IN  MSG.OUT
 To  127.0.0.1:9002       5     0.7ms   1s ago    0        0     ->0      0->    3269    3212
From 127.0.0.1:9002       5     1.1ms   1s ago    0        0     ->0      0->    3213    3269
 To  127.0.0.1:9003       5     0.6ms   1s ago    0        0     ->0      0->    6001    6076
From 127.0.0.1:9003       5     0.6ms   1s ago    0        0     ->0      0->    6077    6001
 To  127.0.0.1:9004       5     0.6ms   1s ago    0        0     ->0      0->    3243    3160
From 127.0.0.1:9004       5     0.4ms   1s ago    0        0     ->0      0->    3161    3243
 To  127.0.0.1:9005       5     0.6ms   1s ago    0        0     ->0      0->    3150    3094
From 127.0.0.1:9005       5     0.3ms   1s ago    0        0     ->0      0->    3095    3150
 To  127.0.0.1:9006       5     0.3ms   1s ago    0        0     ->0      0->    3185    3221
From 127.0.0.1:9006       5     0.6ms   1s ago    0        0     ->0      0->    3222    3185

To/From will be (close to) symmetric if everything is well.

There is already server support for metrics.

Motivation and Context

Add metrics to mc.

How to test this PR?

mc support top rpc ALIAS on a distributed setup

Types of changes

  • New feature (non-breaking change which adds functionality)

Pretty basic information right now:

Example:
```
λ mc support top rpc myminio
      SERVER            CONCTD  PING    OUT Q   RECONNS STR IN  STR OUT MSG IN  MSG OUT
 To  127.0.0.1:9001       5     0.5ms     0        0    -> 20    0 ->   60007    60089
From 127.0.0.1:9001       5     0.5ms     0        0     -> 0    20 ->  60089    60002
 To  127.0.0.1:9002       5     0.0ms     0        0     -> 0    4 ->   34209    33937
From 127.0.0.1:9002       5     0.5ms     0        0     -> 4    0 ->   33938    34209
 To  127.0.0.1:9003       5     0.0ms     0        0     -> 0    4 ->   99379   101165
From 127.0.0.1:9003       5     0.0ms     0        0     -> 4    0 ->   101166   99379
 To  127.0.0.1:9004       5     0.5ms     0        0     -> 0    4 ->   33784    33029
From 127.0.0.1:9004       5     0.0ms     0        0     -> 4    0 ->   33030    33784
 To  127.0.0.1:9005       5     0.0ms     0        0     -> 0    4 ->   33515    33135
From 127.0.0.1:9005       5     0.5ms     0        0     -> 4    0 ->   33136    33515
 To  127.0.0.1:9006       5     0.0ms     0        0     -> 0    4 ->   33627    33161
From 127.0.0.1:9006       5     0.0ms     0        0     -> 4    0 ->   33162    33627
```

There is already server support for metrics.
@zveinn zveinn self-requested a review October 14, 2024 17:39
@harshavardhana harshavardhana merged commit bad9b2c into minio:master Oct 14, 2024
5 checks passed
@klauspost klauspost deleted the add-rpc-metrics branch October 15, 2024 09:30
flydt added a commit to flydt/mc that referenced this pull request Nov 16, 2024
* remove s390x support

* Add basic RPC metrics via `mc support top rpc` (minio#5060)

Example:
```
λ mc support top rpc myminio
      SERVER            CONCTD  PING    OUT Q   RECONNS STR IN  STR OUT MSG IN  MSG OUT
 To  127.0.0.1:9001       5     0.5ms     0        0    -> 20    0 ->   60007    60089
From 127.0.0.1:9001       5     0.5ms     0        0     -> 0    20 ->  60089    60002
 To  127.0.0.1:9002       5     0.0ms     0        0     -> 0    4 ->   34209    33937
From 127.0.0.1:9002       5     0.5ms     0        0     -> 4    0 ->   33938    34209
 To  127.0.0.1:9003       5     0.0ms     0        0     -> 0    4 ->   99379   101165
From 127.0.0.1:9003       5     0.0ms     0        0     -> 4    0 ->   101166   99379
 To  127.0.0.1:9004       5     0.5ms     0        0     -> 0    4 ->   33784    33029
From 127.0.0.1:9004       5     0.0ms     0        0     -> 4    0 ->   33030    33784
 To  127.0.0.1:9005       5     0.0ms     0        0     -> 0    4 ->   33515    33135
From 127.0.0.1:9005       5     0.5ms     0        0     -> 4    0 ->   33136    33515
 To  127.0.0.1:9006       5     0.0ms     0        0     -> 0    4 ->   33627    33161
From 127.0.0.1:9006       5     0.0ms     0        0     -> 4    0 ->   33162    33627
```

There is already server support for metrics.

* Report errors and better file names for mc support inspect  (minio#5062)

Check the returned stream for errors and show them. Don't upload/keep pointless data

Use the request alias to create a better file name...

Example:

```
λ mc support inspect play/testbucket/testdat/**
mc: <ERROR> Unable to download file data. GetRawData: No files matched the given pattern.
```

Previously the error would just be ignored.

```
λ mc support inspect --airgap play/testbucket/testdata/**
File data successfully downloaded as inspect-play_testbucket_testdata.enc
```

Previously all files would be called `inspect-data.enc`, which would just be difficult to distinguish.

* Add support rpc JSON replay. (minio#5064)

Add realtime replay of captured JSON data.

Example:
```
λ mc support top rpc --json myminio >rpc.json

λ mc support top rpc -in=rpc.json
```

* Add string/json output string mc pipe (minio#5065)

This commit adds a string message and json output to mc pipe.  The
implementation is heavily inspired by the one in mc cp, just adapted to
the different semantics of pipe.

* Mark batch job status properly (minio#5066)

Client side overall status was always shown as `in-progress` even if
job was successful or failed.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>

* Make the profiler easier to use (minio#5068)

* Bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (minio#5074)

Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](golang-jwt/jwt@v4.5.0...v4.5.1)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Only warn if less than 28 days to cert expires (minio#5077)

Fixes minio#5076 as much as reasonable.

* lint: Fix new detected lint complaints (minio#5080)

* heal: Simplify bucket healing status (minio#5079)

Avoid using the drive count concept to pick a color for a bucket healing
result. This was straightforward when we were healing a bucket per erasure 
set, but now it becomes more complex to show a result.

Pick green if everything is okay, yellow if a bucket is missing in a
drive, red for any other drive state (unformatted, offline, ..), and
grey for any weird situation.

---------

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Harshavardhana <harsha@minio.io>
Co-authored-by: Klaus Post <klauspost@gmail.com>
Co-authored-by: Michael Fenn <michaelfenn87@gmail.com>
Co-authored-by: Shubhendu <shubhendu@minio.io>
Co-authored-by: dorman <37854724+dormanze@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anis Eleuch <vadmeste@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants