Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hierarchy for Pulsar #11971

Merged
merged 1 commit into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/en/changes/changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@
* Add Service Hierarchy auto matching layer relationships (upper -> lower) as following:
- CLICKHOUSE -> K8S_SERVICE
- VIRTUAL_DATABASE -> CLICKHOUSE
* Add Service Hierarchy auto matching layer relationships (upper -> lower) as following:
- PULSAR -> K8S_SERVICE
- VIRTUAL_MQ -> PULSAR

#### UI

Expand Down
22 changes: 20 additions & 2 deletions docs/en/concepts-and-designs/service-hierarchy.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,12 @@ If you want to customize it according to your own needs, please refer to [Servic
| KAFKA | K8S_SERVICE | [KAFKA On K8S_SERVICE](#kafka-on-k8s_service) |
| VIRTUAL_MQ | RABBITMQ | [VIRTUAL_MQ On RABBITMQ](#virtual_mq-on-rabbitmq) |
| VIRTUAL_MQ | ROCKETMQ | [VIRTUAL_MQ On K8S_SERVICE](#virtual_mq-on-rocketmq) |
| VIRTUAL_MQ | KAFKA | [VIRTUAL_MQ On KAFKA](#virtual_mq-on-kafka) |
| VIRTUAL_MQ | RABBITMQ | [VIRTUAL_MQ On RABBITMQ](#virtual_mq-on-rabbitmq) |
| VIRTUAL_MQ | KAFKA | [VIRTUAL_MQ On KAFKA](#virtual_mq-on-kafka) |
| VIRTUAL_MQ | RABBITMQ | [VIRTUAL_MQ On RABBITMQ](#virtual_mq-on-rabbitmq) |
| CLICKHOUSE | K8S_SERVICE | [CLICKHOUSE On K8S_SERVICE](#clickhouse-on-k8s_service) |
| VIRTUAL_DATABASE | CLICKHOUSE | [VIRTUAL_DATABASE On CLICKHOUSE](#virtual_database-on-clickhouse) |
| PULSAR | K8S_SERVICE | [PULSAR On K8S_SERVICE](#pulsar-on-k8s_service) |
| VIRTUAL_MQ | PULSAR | [VIRTUAL_MQ On PULSAR](#virtual_mq-on-pulsar) |

- The following sections will describe the **default matching rules** in detail and use the `upper-layer On lower-layer` format.
- The example service name are based on SkyWalking [Showcase](https://github.com/apache/skywalking-showcase) default deployment.
Expand Down Expand Up @@ -199,6 +201,22 @@ If you want to customize it according to your own needs, please refer to [Servic
- VIRTUAL_DATABASE.service.name: `clickhouse.skywalking-showcase.svc.cluster.local:8123`
- CLICKHOUSE.service.name: `clickhouse::clickhouse.skywalking-showcase`

#### PULSAR On K8S_SERVICE
- Rule name: `short-name`
- Groovy script: `{ (u, l) -> u.shortName == l.shortName }`
- Description: PULSAR.service.shortName == K8S_SERVICE.service.shortName
- Matched Example:
- PULSAR.service.name: `pulsar::pulsar.skywalking-showcase`
- K8S_SERVICE.service.name: `skywalking-showcase::pulsar.skywalking-showcase`

#### VIRTUAL_MQ On PULSAR
- Rule name: `lower-short-name-with-fqdn`
- Groovy script: `{ (u, l) -> u.shortName.substring(0, u.shortName.lastIndexOf(':')) == l.shortName.concat('.svc.cluster.local') }`
- Description: VIRTUAL_MQ.service.shortName remove port == PULSAR.service.shortName with fqdn suffix
- Matched Example:
- VIRTUAL_MQ.service.name: `pulsar.skywalking-showcase.svc.cluster.local:6650`
- PULSAR.service.name: `pulsar::pulsar.skywalking-showcase`

### Build Through Specific Agents
Use agent tech involved(such as eBPF) and deployment tools(such as operator and agent injector) detect the service hierarchy relations.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ hierarchy:

CLICKHOUSE:
K8S_SERVICE: short-name

PULSAR:
K8S_SERVICE: short-name

VIRTUAL_DATABASE:
MYSQL: lower-short-name-with-fqdn
Expand All @@ -66,6 +69,7 @@ hierarchy:
ROCKETMQ: lower-short-name-with-fqdn
RABBITMQ: lower-short-name-with-fqdn
KAFKA: lower-short-name-with-fqdn
PULSAR: lower-short-name-with-fqdn

# Use Groovy script to define the matching rules, the input parameters are the upper service(u) and the lower service(l) and the return value is a boolean,
# which are used to match the relation between the upper service(u) and the lower service(l) on the different layers.
Expand Down Expand Up @@ -98,6 +102,7 @@ layer-levels:
CLICKHOUSE: 2
RABBITMQ: 2
KAFKA: 2
PULSAR: 2

MESH_DP: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -429,7 +429,32 @@
"entity": "Service",
"name": "Pulsar-Cluster",
"id": "Pulsar-Cluster",
"isRoot": false
"isRoot": false,
"isDefault": true,
"expressions": [
"avg(meter_pulsar_total_topics)",
"avg(meter_pulsar_total_subscriptions)",
"avg(meter_pulsar_message_rate_in)",
"avg(meter_pulsar_message_rate_out)",
"avg(meter_pulsar_storage_size)"
],
"expressionsConfig": [
{
"label": "total topics"
},
{
"label": "total subscriptions"
},
{
"label": "message rate in"
},
{
"label": "message rate out"
},
{
"label": "storage size"
}
]
}
}
]
Loading