Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] support span serializing and data routing #1878

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

KayzzzZ
Copy link
Collaborator

@KayzzzZ KayzzzZ commented Nov 12, 2024

No description provided.

Signed-off-by: qianlu.kk <qianlu.lk@alibaba-inc.com>
Signed-off-by: qianlu.kk <qianlu.lk@alibaba-inc.com>
@KayzzzZ
Copy link
Collaborator Author

KayzzzZ commented Nov 13, 2024

当前 PR 包含两个功能:

  1. span 序列化:根据服务端提供的 spec
  2. 支持 subpath:复用 match tags 和 router 的逻辑
    这两个功能涉及的改动参考下图
    image
    联调和评审通过后,我会把这两个功能,分开提两个 PR

该分支目前处于联调阶段,整体的实现思路基本不会出现太大的修改了。希望可以先评审一下目前的思路和实现方式,是否需要调整,包括:

  1. span tag 的用法是否标准,目前定义在了 tag constants 中;
  2. 序列化的整体流程是否合理;
  3. flusher sls 的配置解析是否合理。尤其需要注意下, subpath 场景没有 logstore,我提供了一个假的,不确定是否会有潜在的影响;
  4. 引入 subpath 会影响 DiskBufferWriter 的逻辑,增加了一个字段来做判断。

后续联调过程中可能还会出现的改动大概包括以下几种:

  1. 增加一些默认的 tag,包括 group tags 和 event tags
  2. 增加一些请求 header 或 request parameters:可能会增加,但是尽量不会,因为可能影响到 diskbuffer 那里的实现
  3. eBPF 相关 event 时间戳的改动

Comment on lines 400 to 441
if (telemetryType == "arms") {
// Parse Match segment
const char* key = "Match";
const Json::Value* itr = config.find(key, key + strlen(key));
if (!itr) {
// Error
LOG_WARNING(sLogger, ("invalid config!", "telemetry arms need add match tags!"));
return false;
}
// Type
string type;
// Key
std::string tagKey;
// Value
std::string tagValue;
const std::set<std::string> supportDataTypes = {
"trace",
"metric",
"agent_info",
};
if (!itr->isObject() || !GetMandatoryStringParam(*itr, "Type", type, errorMsg) || type != "tag" ||
!GetMandatoryStringParam(*itr, "Key", tagKey, errorMsg) ||
!GetMandatoryStringParam(*itr, "Value", tagValue, errorMsg) ||
tagKey != "data_type" || !supportDataTypes.count(tagValue)) {
// error
LOG_WARNING(sLogger, ("invalid config!", "telemetry arms need add match tags!")("type",type)("key", tagKey)("value", tagValue));
return false;
}
if (tagValue == "trace") {
mSubpath = "/apm/trace/arms/v1/trace_log";
mLogstore = "__arms_default_trace__";
LOG_WARNING(sLogger, ("successfully set subpath", mSubpath) ("logstore", mLogstore));
} else if (tagValue == "metric") {
mSubpath = "/apm/metric/arms/v1/metric_log";
mLogstore = "__arms_default_metric__";
LOG_WARNING(sLogger, ("successfully set subpath", mSubpath) ("logstore", mLogstore));
} else if (tagValue == "agent_info") {
mSubpath = "/apm/meta/arms/v1/meta_log/AgentInfo";
mLogstore = "__arms_default_agentinfo__";
LOG_WARNING(sLogger, ("successfully set subpath", mSubpath) ("logstore", mLogstore));
}
} else {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这边telemetrytype要不干脆写成三种,arms_xxx这样子的,这样就不用这段逻辑了

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@@ -84,14 +83,29 @@ Json::Value SpanEvent::SpanLink::ToJson() const {
root["traceState"] = mTraceState.to_string();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

字段的标准要统一

mContext->GetLogstoreName(),
mContext->GetRegion());
}
if (telemetryType == "metrics") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

telemetryType if逻辑的层次是并列的,if else 顺序 log、metirc、span

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的我修改一下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants