Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zipkin exporter #471

Merged
merged 48 commits into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
8fd81b0
zipkin changes
lalitb Dec 18, 2020
a532b72
reformat
lalitb Dec 18, 2020
861da22
fix cmake format
lalitb Dec 18, 2020
586d5c9
fix build
lalitb Dec 18, 2020
babbc5d
remove formats
lalitb Dec 18, 2020
5570d18
remove console debugs
lalitb Dec 18, 2020
1039f4c
remove debug
lalitb Dec 18, 2020
fd5a5b6
Update exporters/zipkin/CMakeLists.txt
lalitb Dec 21, 2020
d4f22e3
fix endpoint
lalitb Dec 21, 2020
0fbe3af
add noop response
lalitb Dec 22, 2020
6d3e445
Merge branch 'master' into zipkin
lalitb Dec 22, 2020
cdf5dc4
fix nullptr for http errors
lalitb Dec 22, 2020
19d789f
remove otlp stuff
lalitb Dec 22, 2020
84b0458
fix sync mode
lalitb Dec 28, 2020
93beff6
merge conflict
lalitb Jan 18, 2021
81049c7
merge conflict
lalitb Jan 26, 2021
791e2a1
fixes
lalitb Jan 27, 2021
aff76e2
fix
lalitb Jan 27, 2021
dcc758b
fix
lalitb Jan 27, 2021
b3e96c7
fix spell
lalitb Jan 27, 2021
4c97dc7
fix
lalitb Jan 27, 2021
3806ee2
fix
lalitb Jan 27, 2021
0125752
fix
lalitb Jan 27, 2021
ae0a1a0
fix
lalitb Jan 27, 2021
8c351c1
review fix
lalitb Jan 29, 2021
b2cfa8e
Merge branch 'main' into zipkin
lalitb Jan 29, 2021
803eaa3
Merge branch 'main' into zipkin
lalitb Feb 1, 2021
7283a85
fix
lalitb Feb 12, 2021
67d6793
Merge branch 'main' into zipkin
lalitb Feb 12, 2021
5764899
fix
lalitb Feb 12, 2021
18390cf
fix
lalitb Feb 12, 2021
590bbcf
Merge branch 'main' into zipkin
lalitb Feb 15, 2021
b54304d
fix
lalitb Feb 15, 2021
ad1506e
Merge branch 'zipkin' of github.com:lalitb/opentelemetry-cpp into zipkin
lalitb Feb 15, 2021
7450085
Merge branch 'main' into zipkin
lalitb Feb 15, 2021
11ffb1e
Merge branch 'main' into zipkin
lalitb Feb 18, 2021
e6e333f
Merge branch 'main' into zipkin
lalitb Feb 23, 2021
d4c7217
Merge branch 'main' into zipkin
lalitb Feb 24, 2021
653d084
fix annotations as per specs
lalitb Feb 25, 2021
e0057f3
Merge branch 'zipkin' of github.com:lalitb/opentelemetry-cpp into zipkin
lalitb Feb 25, 2021
6c89e4f
Merge branch 'main' into zipkin
lalitb Mar 2, 2021
0eb9ce4
Merge branch 'main' into zipkin
lalitb Mar 2, 2021
4633712
Update exporters/zipkin/include/opentelemetry/exporters/zipkin/zipkin…
lalitb Mar 3, 2021
359a385
Update exporters/zipkin/include/opentelemetry/exporters/zipkin/zipkin…
lalitb Mar 3, 2021
051fccf
fix comments
lalitb Mar 3, 2021
acb1040
add changelog
lalitb Mar 3, 2021
f97e6ca
fix md
lalitb Mar 3, 2021
fa6826a
fix
lalitb Mar 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ if(WITH_STL)
endif()

option(WITH_OTLP "Whether to include the OpenTelemetry Protocol in the SDK" OFF)
option(WITH_ZIPKIN "Whether to include the Zipkin exporter in the SDK" OFF)

option(WITH_PROMETHEUS "Whether to include the Prometheus Client in the SDK"
OFF)
Expand Down Expand Up @@ -162,6 +163,7 @@ include_directories(sdk/include)
include_directories(sdk)
add_subdirectory(sdk)
include_directories(.)
include_directories(ext/include)
add_subdirectory(exporters)
if(WITH_EXAMPLES)
add_subdirectory(examples)
Expand Down
4 changes: 4 additions & 0 deletions exporters/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ if(WITH_PROMETHEUS)
add_subdirectory(prometheus)
endif()

if(WITH_ZIPKIN)
add_subdirectory(zipkin)
endif()

if(WITH_ELASTICSEARCH)
add_subdirectory(elasticsearch)
endif()
31 changes: 31 additions & 0 deletions exporters/zipkin/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Copyright 2020, OpenTelemetry Authors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need update this to Copyright 2021?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

include_directories(include)
find_package(CURL REQUIRED)
if(CURL_FOUND)
add_compile_definitions(WITH_CURL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems add_compile_definitions was added to CMake since 3.12 but we support CMake 3.1 above. We may add -DWITH_CURL as compiler flag manually for older CMake.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank. using add_definitions now.

endif()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this REQUIRED argument is given, we can remove the if condition, as configuration will abort anyway if CURL is not found.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Have removed the REQUIRED argument, as it should be possible to run zipkin with other http client libraries.

add_library(zipkin_trace_exporter src/zipkin_exporter.cc src/recordable.cc)
if(BUILD_TESTING)
add_executable(zipkin_recordable_test test/zipkin_recordable_test.cc)

target_link_libraries(zipkin_recordable_test ${GTEST_BOTH_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT} zipkin_trace_exporter)

gtest_add_tests(
TARGET zipkin_recordable_test
TEST_PREFIX exporter.
TEST_LIST zipkin_recordable_test)
endif() # BUILD_TESTING
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#pragma once

#include "nlohmann/json.hpp"
#include "opentelemetry/sdk/trace/recordable.h"
#include "opentelemetry/version.h"

OPENTELEMETRY_BEGIN_NAMESPACE
namespace exporter
{
namespace zipkin
{
using ZipkinSpan = nlohmann::json;

enum class TransportFormat
{
JSON,
PROTOBUF
};

class Recordable final : public sdk::trace::Recordable
{
public:
const ZipkinSpan &span() const noexcept { return span_; }

void SetIds(trace::TraceId trace_id,
trace::SpanId span_id,
trace::SpanId parent_span_id) noexcept override;

void SetAttribute(nostd::string_view key,
const opentelemetry::common::AttributeValue &value) noexcept override;

void AddEvent(nostd::string_view name,
core::SystemTimestamp timestamp,
const common::KeyValueIterable &attributes) noexcept override;

void AddLink(const opentelemetry::trace::SpanContext &span_context,
const common::KeyValueIterable &attributes) noexcept override;

void SetStatus(trace::CanonicalCode code, nostd::string_view description) noexcept override;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update CanonicalCode to StatusCode?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, plan to do it based on which PR is merged first :)


void SetName(nostd::string_view name) noexcept override;

void SetStartTime(opentelemetry::core::SystemTimestamp start_time) noexcept override;

virtual void SetSpanKind(opentelemetry::trace::SpanKind span_kind) noexcept override;

void SetDuration(std::chrono::nanoseconds duration) noexcept override;

private:
ZipkinSpan span_;
};
} // namespace zipkin
} // namespace exporter
OPENTELEMETRY_END_NAMESPACE
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#pragma once

#include "opentelemetry/exporters/zipkin/recordable.h"
#include "opentelemetry/ext/http/client/http_client_factory.h"
#include "opentelemetry/ext/http/common/url_parser.h"
#include "opentelemetry/sdk/trace/exporter.h"
#include "opentelemetry/sdk/trace/span_data.h"

#include "nlohmann/json.hpp"

OPENTELEMETRY_BEGIN_NAMESPACE
namespace exporter
{
namespace zipkin
{

std::string ZIPKIN_ENDPOINT_DEFAULT = "http://localhost:9411/api/v2/spans";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this constant, and also change the naming to kZipkinEndpointDefault?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.


/**
* Struct to hold Zipkin exporter options.
*/
struct ZipkinExporterOptions
{
// The endpoint to export to. By default the OpenTelemetry Collector's default endpoint.
std::string endpoint = ZIPKIN_ENDPOINT_DEFAULT;
TransportFormat format = TransportFormat::JSON;
std::string service_name = "default-service";
std::string ipv4;
std::string ipv6;
};

namespace trace_sdk = opentelemetry::sdk::trace;
namespace http_client = opentelemetry::ext::http::client;

/**
* The Zipkin exporter exports span data in JSON format as expected by Zipkin
*/
class ZipkinExporter final : public trace_sdk::SpanExporter, public http_client::EventHandler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you derive from EventHandler here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. It is not needed, and so removed.

{
public:
/**
* Create an ZipkinExporter using all default options.
lalitb marked this conversation as resolved.
Show resolved Hide resolved
*/
ZipkinExporter();

/**
* Create an ZipkinExporter using the given options.
lalitb marked this conversation as resolved.
Show resolved Hide resolved
*/
ZipkinExporter(const ZipkinExporterOptions &options);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use explicit for all 1 arguments constructor. I think it is a bad practice to rely on converting constructors (https://en.cppreference.com/w/cpp/language/converting_constructor) in general unless explicitly wanted to have this behavior like std::string_view.

/cc @maxgolov @lalitb @g-easy @jsuereth I see that in other places you do not use explicit, what is the overall rule?

Copy link
Member Author

@lalitb lalitb Jan 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. Will fix it. We follow google c++ style guide:
(https://google.github.io/styleguide/cppguide.html#Implicit_Conversions):
Do not define implicit conversions. Use the explicit keyword for conversion operators and single-argument constructors.

Also, would be good to add cpplint CI check to catch these error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fixed for zipkin. Will have separate PR for adding explicit keyword at other places.


/**
* Create a span recordable.
* @return a newly initialized Recordable object
*/
std::unique_ptr<trace_sdk::Recordable> MakeRecordable() noexcept override;

/**
* Export a batch of span recordables in JSON format.
* @param spans a span of unique pointers to span recordables
*/
trace_sdk::ExportResult Export(
const nostd::span<std::unique_ptr<trace_sdk::Recordable>> &spans) noexcept override;

/**
* Shut down the exporter.
* @param timeout an optional timeout, default to max.
*/
bool Shutdown(
std::chrono::microseconds timeout = std::chrono::microseconds::max()) noexcept override
{
return true;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extra ; looks unnecessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed.


void OnResponse(http_client::Response &response) noexcept override
{ /*Not required */
}

virtual void OnEvent(http_client::SessionState state,
nostd::string_view msg) noexcept override{/* Not required */};

virtual void OnConnecting(const http_client::SSLCertificate &) noexcept override{
/* Not required */};

private:
void InitializeLocalEndpoint();

private:
// The configuration options associated with this exporter.
bool isShutdown_ = false;
std::shared_ptr<http_client::SessionManager> http_session_manager_;
nlohmann::json local_end_point_;
ZipkinExporterOptions options_;
ext::http::common::UrlParser url_parser_;
};
} // namespace zipkin
} // namespace exporter
OPENTELEMETRY_END_NAMESPACE
184 changes: 184 additions & 0 deletions exporters/zipkin/src/recordable.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
#include "opentelemetry/exporters/zipkin/recordable.h"

#include <iostream>

OPENTELEMETRY_BEGIN_NAMESPACE
namespace exporter
{
namespace zipkin
{

const int kAttributeValueSize = 14;

void Recordable::SetIds(trace::TraceId trace_id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function reusable by other exporters, would they use the same field names? id , parentId, traceId - should this be moved into common utility function within the SDK?

@mishal23 - you need to borrow this same code for ETW exporter, to populate the actual IDs in your PR.

Copy link
Member Author

@lalitb lalitb Dec 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxgolov - Not sure if we can move it to common. As different exporters expect these fields in different formats. E.g, Zipkin (json) as hex chars of 16/32 length, Jaeger (binary thrift) as uint64_t.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation of Recordable is exporter specific. As exporter could choose different underlying storage, agree that it is unnecessary to provide a common implementation.

trace::SpanId span_id,
trace::SpanId parent_span_id) noexcept
{
char traceid[trace::TraceId::kSize * 2] = {0};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent naming for traceid, probably trace_id_lower_base16?

Copy link
Member Author

@lalitb lalitb Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Have changed for all three id's.

trace_id.ToLowerBase16(traceid);
char spanid[trace::SpanId::kSize * 2] = {0};
span_id.ToLowerBase16(spanid);
char parentid[trace::SpanId::kSize * 2] = {0};
parent_span_id.ToLowerBase16(parentid);
span_["id"] = std::string(spanid, 16);
span_["parentId"] = std::string(parentid, 16);
span_["traceId"] = std::string(traceid, 32);
}

void PopulateAttribute(nlohmann::json &attribute,
nostd::string_view key,
const opentelemetry::common::AttributeValue &value)
{
// Assert size of variant to ensure that this method gets updated if the variant
// definition changes
static_assert(
nostd::variant_size<opentelemetry::common::AttributeValue>::value == kAttributeValueSize,
"AttributeValue contains unknown type");

// attribute->set_key(key.data(), key.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed. Removed it.


if (nostd::holds_alternative<bool>(value))
{
attribute[key.data()] = nostd::get<bool>(value);
}
else if (nostd::holds_alternative<int>(value))
{
attribute[key.data()] = nostd::get<int>(value);
}
else if (nostd::holds_alternative<int64_t>(value))
{

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this blank line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done :)

attribute[key.data()] = nostd::get<int64_t>(value);
}
else if (nostd::holds_alternative<unsigned int>(value))
{
attribute[key.data()] = nostd::get<unsigned int>(value);
}
else if (nostd::holds_alternative<uint64_t>(value))
{
attribute[key.data()] = nostd::get<uint64_t>(value);
}
else if (nostd::holds_alternative<double>(value))
{
attribute[key.data()] = nostd::get<double>(value);
}
else if (nostd::holds_alternative<nostd::string_view>(value))
{
attribute[key.data()] = nostd::string_view(nostd::get<nostd::string_view>(value).data(),
nostd::get<nostd::string_view>(value).size());
}
else if (nostd::holds_alternative<nostd::span<const bool>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const bool>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const int>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const int>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const int64_t>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const int64_t>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const unsigned int>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const unsigned int>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const uint64_t>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const uint64_t>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const double>>(value))
{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const double>>(value))
{
attribute[key.data()].push_back(val);
}
}
else if (nostd::holds_alternative<nostd::span<const nostd::string_view>>(value))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious about the perf implication of these if-else checks, not sure if a small jump table would make it faster. Not a blocker for this PR though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we are doing these variant checks at multiple places, would be better to use dispatch table throughout. Will add a ticket for same.

{
attribute[key.data()] = {};
for (const auto &val : nostd::get<nostd::span<const nostd::string_view>>(value))
{
attribute[key.data()].push_back(std::string(val.data(), val.size()));
}
}
}

void Recordable::SetAttribute(nostd::string_view key,
const opentelemetry::common::AttributeValue &value) noexcept
{
if (!span_.contains("tags"))
{
span_["tags"] = nlohmann::json::object();
}
PopulateAttribute(span_["tags"], key, value);
}

void Recordable::AddEvent(nostd::string_view name,
core::SystemTimestamp timestamp,
const common::KeyValueIterable &attributes) noexcept
{
nlohmann::json annotations = nlohmann::json::object(); // empty object
attributes.ForEachKeyValue([&](nostd::string_view key, common::AttributeValue value) noexcept {
PopulateAttribute(annotations, key, value);
return true;
});
span_["annotations"][name.data()]["value"] = annotations;
span_["annotations"]["timestamp"] =
std::chrono::duration_cast<std::chrono::milliseconds>(timestamp.time_since_epoch()).count();
}

void Recordable::AddLink(const opentelemetry::trace::SpanContext &span_context,
const common::KeyValueIterable &attributes) noexcept
{
// TODO: Currently not supported by specs:
// https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/sdk_exporters/zipkin.md
}

void Recordable::SetStatus(trace::CanonicalCode code, nostd::string_view description) noexcept
{
span_["tags"]["otel.status_code"] = code;
if (description.size())
span_["tags"]["otel.status_description"] = description;
}

void Recordable::SetName(nostd::string_view name) noexcept
{
span_["name"] = name.data();
}

void Recordable::SetStartTime(opentelemetry::core::SystemTimestamp start_time) noexcept
{
span_["timestamp"] = start_time.time_since_epoch().count();
}

void Recordable::SetDuration(std::chrono::nanoseconds duration) noexcept
{
span_["duration"] = duration.count();
}

void Recordable::SetSpanKind(opentelemetry::trace::SpanKind span_kind) noexcept {}
} // namespace zipkin
} // namespace exporter
OPENTELEMETRY_END_NAMESPACE
Loading