OpenTelemetry integration

[Yiling-J]

This pull request introduces observability enhancements by integrating the OpenTelemetry SDK for tracing, logging, and metrics collection. It also includes an example Grafana LGTM Docker Compose setup for easy local development and monitoring. Additionally, improvements to API context management are made through a refactor and the addition of a custom Gin context linter. Idea from docker-otel-lgtm.

This PR does the following:

• OpenTelemetry SDK for Traces, Logs, and Metrics: Integration of the OpenTelemetry SDK to capture and export traces, logs, and metrics from the application for improved observability.
• Grafana LGTM Example Setup: A ready-to-use Grafana setup using a docker-compose file to quickly launch a Grafana instance for visualizing telemetry data.
• Example Grafana Dashboards: Added example dashboards to display traces, logs, and metrics, enabling easy monitoring and visualization of the telemetry data.
• Gin Context Refactor and Linter: A custom linter that ensures gin.Context is not mistakenly used as context.Context and refactor all api/handler context.
• Integration README: A comprehensive README with step-by-step instructions on configuring OpenTelemetry, setting up Grafana, and monitoring telemetry data.

How to Set Up:

1. Set up Grafana LGTM:

   • Clone the repository and navigate to the builder/instrumentation/compose-example directory.

   • Start the Grafana LGTM instance using Docker Compose:

     docker-compose up -d

   • Access the Grafana dashboard at http://localhost:3000.

2. Import Example Dashboards:

   • Three example dashboards for logs, traces, and metrics are provided under builder/instrumentation/grafana-dashboards-example. Import them into Grafana for quick monitoring.

3. Enable OpenTelemetry SDK Reporting:

   • To enable reporting to the OpenTelemetry Collector, update your configuration file as follows:

     [instrumentation]
     otlp_endpoint = "http://localhost:4317"
     otlp_logging = true

   • Alternatively, you can use environment variables:

     OPENCSG_TRACING_OTLP_ENDPOINT="http://localhost:4317"
     OPENCSG_TRACING_OTLP_LOGGING=true

4. Restart the Server:

   • Restart the server and monitor the data through the Grafana dashboards.

[starship-github]

Linter Issue Report

During the code review, a list issues were found. These issues could affect the code quality, maintainability, and consistency. Below is the detailed Linter issue report:

linters/gincontext/main.go

Lint Issue: "go/types" imported and not used

• Location: Line 7
• Code Snippet:

  import (
  	"flag"
  	"fmt"
  	"go/ast"
  	"go/types" // Issue here
  	"log"

• Actionable Suggestion:
  Please remove the "go/types" import statement from your import block as it is not being used in the code. This will clean up the code and eliminate the lint issue. If you plan to use it later, consider adding it back when necessary.

---

Please make the suggested changes to improve the code quality.

[starship-github]

Possible Issues And Suggestions:

• api/handler/repo_test.go

  • Comments:
    • The context cancellation function 'cancel' is declared but not used, which could lead to context leaks.
  • Suggestions:

    cc, _ := context.WithCancel(tester.gctx.Request.Context())
    

• Line 55 in api/handler/mirror_source.go

  • Comments:
    • The change from ctx to ctx.Request.Context() might not be necessary if ctx is already a *gin.Context which provides request-scoped context.

• Line 56 in api/handler/runtime_architecture.go

  • Comments:
    • Changing the context from ctx to ctx.Request.Context() is redundant if ctx is of type *gin.Context, which already provides access to the request context.

• Line 54 in api/handler/space_resource.go

  • Comments:
    • The modification to use ctx.Request.Context() instead of ctx might be unnecessary if ctx is a *gin.Context that already encapsulates the request context.

• builder/instrumentation/otel.go

  • Comments:
    • Ensure error handling is consistent and comprehensive, especially in the setup of OpenTelemetry components where multiple points of failure exist.

• Ensure thorough testing of the OpenTelemetry integration to catch any potential issues with tracing, logging, or metrics collection.

• Verify the new error handling and context propagation changes in the model handler functions do not introduce any regressions.

MR Evaluation:

This feature is still under test, evaluation are given by AI and might be inaccurate.

After evaluation, the code changes in the Merge Request get score: 100.

Tips

CodeReview Commands (invoked as MR or PR comments)

• @codegpt /review to trigger an code review.
• @codegpt /evaluate to trigger code evaluation process.
• @codegpt /describe to regenerate the summary of the MR.
• @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
• @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

• Review comments: Directly reply to a review comment made by StarShip.
  Example:
  • @codegpt How to fix this bug?
• Files and specific lines of code (under the "Files changed" tab):
  Tag @codegpt in a new review comment at the desired location with your query.
  Examples:
  • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

• Visit our Documentation
  for detailed information on how to use Starship CodeReview.

About Us:

Visit the OpenCSG StarShip website for the Dashboard and detailed information on CodeReview, CodeGen, and other StarShip modules.

[ganisback]

LGTM

[starship-github]

The StarShip CodeReviewer was triggered but terminated because it encountered an issue: The MR state is not opened.

Tips

CodeReview Commands (invoked as MR or PR comments)

• @codegpt /review to trigger an code review.
• @codegpt /evaluate to trigger code evaluation process.
• @codegpt /describe to regenerate the summary of the MR.
• @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
• @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

• Review comments: Directly reply to a review comment made by StarShip.
  Example:
  • @codegpt How to fix this bug?
• Files and specific lines of code (under the "Files changed" tab):
  Tag @codegpt in a new review comment at the desired location with your query.
  Examples:
  • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

• Visit our Documentation
  for detailed information on how to use Starship CodeReview.