Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HTTP health check handler for server health monitoring #952

Merged
merged 7 commits into from
Aug 7, 2024
Merged

Add HTTP health check handler for server health monitoring #952

merged 7 commits into from
Aug 7, 2024

Conversation

taeng0204
Copy link
Member

@taeng0204 taeng0204 commented Aug 4, 2024

Added a handler to allow health checks to be performed with plain HTTP GET requests, rather than health checks using rpc.

What this PR does / why we need it:
In issue #832 , we needed a health check via HTTP GET or HEAD method because of NCP's load balancer.
The existing grpchealth could not satisfy this, so we implemented a separate handler.
This allows us to check the health of a specific service via the path /healthz/ and the health of a specific service via the query service=<serviceName>.
Which issue(s) this PR fixes:

Fixes #832

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


Additional documentation:


Checklist:

  • Added relevant tests or not required
  • Didn't break anything

Summary by CodeRabbit

  • New Features

    • Introduced health check functionality, allowing clients to verify the server's status via HTTP GET requests.
    • Added an HTTP handler for health checks, enhancing service reliability and monitoring.
  • Bug Fixes

    • Improved error handling for unsupported request methods.
  • Tests

    • Renamed existing health check test for clarity.
    • Added multiple new tests for both gRPC and HTTP health checks to ensure comprehensive coverage.

@CLAassistant
Copy link

CLAassistant commented Aug 4, 2024

CLA assistant check
All committers have signed the CLA.

Copy link

coderabbitai bot commented Aug 4, 2024

Walkthrough

The recent changes introduce a straightforward HTTP health check endpoint to the server, enhancing service observability. A new handler responds to GET requests, providing an accessible way for uptime checkers to monitor service health. This feature complements existing gRPC health checks, ensuring a versatile and user-friendly monitoring solution.

Changes

File(s) Change Summary
server/rpc/health/health.go, server/rpc/httphealth/httphealth.go Introduced HTTP health check handlers and response structs for health status.
server/rpc/server.go Refactored health check handling to improve code clarity and modularity.
test/integration/health_test.go Renamed gRPC health check test and added new HTTP health check tests for comprehensive coverage.

Assessment against linked issues

Objective Addressed Explanation
Add Plain HTTP Health Check Endpoint (#832)

🐰 In the meadow, we hop with glee,
A health check now waits, as easy as can be!
With GET requests, our status is clear,
Uptime checkers smile, their worries disappear!
So bounce along, in code we trust,
For healthy servers, we must adjust! 🌱


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@krapie krapie self-requested a review August 4, 2024 12:57
@krapie krapie added the enhancement 🌟 New feature or request label Aug 4, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Outside diff range, codebase verification and nitpick comments (1)
server/rpc/health/health.go (1)

1-15: Add a package-level comment.

The file lacks a package-level comment describing the purpose and usage of the health package. Adding this would improve code readability and maintainability.

// Package health provides an HTTP handler for health checks.
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 54910c1 and 89f0189.

Files selected for processing (3)
  • server/rpc/health/health.go (1 hunks)
  • server/rpc/server.go (2 hunks)
  • test/integration/health_test.go (2 hunks)
Additional context used
golangci-lint
server/rpc/health/health.go

55-55: Error return value of w.Write is not checked

(errcheck)

GitHub Check: build
server/rpc/health/health.go

[failure] 55-55:
Error return value of w.Write is not checked (errcheck)


[failure] 27-27:
exported: exported type HealthCheckResponse should have comment or be unexported (revive)


[failure] 31-31:
exported: exported function NewHTTPHealthCheckHandler should have comment or be unexported (revive)

Additional comments not posted (6)
server/rpc/health/health.go (1)

35-36: Use http.StatusMethodNotAllowed for unsupported methods.

The handler should return http.StatusMethodNotAllowed for unsupported methods like POST, PUT, etc. This ensures that only GET requests are allowed.

test/integration/health_test.go (2)

Line range hint 35-50:
LGTM! The gRPC health check test is correctly implemented.

The TestRPCHealthCheck function is correctly implemented and verifies the gRPC health check functionality.


52-65: LGTM! The HTTP health check test is correctly implemented.

The TestHTTPHealthCheck function is correctly implemented and verifies the HTTP health check functionality.

server/rpc/server.go (3)

39-39: LGTM! The new health check import is correctly added.

The import statement for the health package is correctly added.


66-70: LGTM! The health checker initialization is correctly implemented.

The healthChecker variable is correctly initialized with the necessary services.


77-77: LGTM! The HTTP health check handler is correctly integrated.

The NewHTTPHealthCheckHandler is correctly integrated into the server's HTTP multiplexer.

server/rpc/health/health.go Outdated Show resolved Hide resolved
server/rpc/health/health.go Outdated Show resolved Hide resolved
server/rpc/health/health.go Outdated Show resolved Hide resolved
@krapie
Copy link
Member

krapie commented Aug 4, 2024

@taeng0204 Could you please sign the CLA?
Also you need to lint the code to pass the CI.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
server/rpc/health/health.go (1)

32-33: Add a comment for the exported function NewHTTPHealthCheckHandler.

The exported function NewHTTPHealthCheckHandler should have a comment or be unexported, as per Go's best practices.

// NewHTTPHealthCheckHandler creates a new HTTP handler for health checks.
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 89f0189 and 84852b2.

Files selected for processing (3)
  • server/rpc/health/health.go (1 hunks)
  • server/rpc/server.go (2 hunks)
  • test/integration/health_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • server/rpc/server.go
Additional context used
GitHub Check: build
server/rpc/health/health.go

[failure] 28-28:
exported: type name will be used as health.HealthCheckResponse by other packages, and that stutters; consider calling this CheckResponse (revive)

Additional comments not posted (3)
test/integration/health_test.go (2)

Line range hint 35-50:
LGTM! The renaming enhances clarity.

The renaming of the function from TestHealthCheck to TestRPCHealthCheck clarifies its focus on gRPC health check functionality. The implementation is correct.


52-65: LGTM! The new test function is well implemented.

The new function TestHTTPHealthCheck correctly implements the HTTP health check test. It verifies the response status code and decodes the response body into health.HealthCheckResponse.

server/rpc/health/health.go (1)

55-59: Check the error return value of w.Write.

The error return value of w.Write should be checked to ensure that the response is successfully written to the client.

- w.Write(resp)
+ if _, err := w.Write(resp); err != nil {
+   http.Error(w, err.Error(), http.StatusInternalServerError)
+ }

Likely invalid or redundant comment.

server/rpc/health/health.go Outdated Show resolved Hide resolved
@taeng0204
Copy link
Member Author

taeng0204 commented Aug 4, 2024

@krapie
There are a lot of lint errors that are not related to the file I'm dealing with, so I didn't find it, sorry.
For clean commits, I modified the pushed commits with amend and force since it's just me working on the branch, is that okay?
I didn't realize you were using squash merge...!

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 84852b2 and 9c88ea1.

Files selected for processing (2)
  • server/rpc/health/health.go (1 hunks)
  • test/integration/health_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (2)
  • server/rpc/health/health.go
  • test/integration/health_test.go

Added a handler to allow health checks to be performed with plain HTTP GET
requests, rather than health checks using rpc.
Rename existing names used by other packages for lint
@taeng0204
Copy link
Member Author

taeng0204 commented Aug 5, 2024

My real email address and the email info in the commit didn't match, which is why the CLA wasn't working...
I'm so sorry😭

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9c88ea1 and e615224.

Files selected for processing (3)
  • server/rpc/health/health.go (1 hunks)
  • server/rpc/server.go (2 hunks)
  • test/integration/health_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (2)
  • server/rpc/health/health.go
  • test/integration/health_test.go
Additional comments not posted (3)
server/rpc/server.go (3)

39-39: Import statement for health package added.

The import statement for the health package has been correctly added to support the new HTTP health check functionality.


66-70: Initialization of healthChecker variable.

The healthChecker variable has been correctly initialized with the services' names, enhancing code clarity and maintainability.


77-77: Addition of HTTP health check handler.

The HTTP health check handler has been correctly added to the server multiplexer, improving service reliability and monitoring capabilities.

Copy link
Member

@krapie krapie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution!

The code is seamlessly integrated to existing RPC health check and rest of the codes. This is way better than I first thought about adding endpoint for health check.

I have left some small suggestions below 😄

server/rpc/server.go Outdated Show resolved Hide resolved
test/integration/health_test.go Outdated Show resolved Hide resolved
The package name includes health, so we remove health from the handler function
@taeng0204 taeng0204 changed the title Add HTTP health check handler for server health monitoring (#832) Add HTTP health check handler for server health monitoring Aug 6, 2024
Write a health check for each service and a check failure test for the unknown service.
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between e615224 and 0bfba1f.

Files selected for processing (3)
  • server/rpc/health/health.go (1 hunks)
  • server/rpc/server.go (2 hunks)
  • test/integration/health_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • server/rpc/health/health.go
Additional comments not posted (11)
server/rpc/server.go (2)

39-39: Import statement for health package looks good.

The import statement for the health package is necessary for the new HTTP health check handler.


66-77: Initialization of healthChecker and addition of HTTP health check handler looks good.

The initialization of healthChecker and the addition of the HTTP health check handler to the multiplexer are implemented correctly.

However, ensure that the new HTTP health check handler is properly integrated and tested.

test/integration/health_test.go (9)

23-30: Import statements for health checks and testing look good.

The import statements for the health package, json, http, and testing are necessary for the new test functions.


53-70: New test function for gRPC health check of Yorkie service looks good.

The test function performs a gRPC health check for the Yorkie service and asserts the expected status.


72-89: New test function for gRPC health check of Admin service looks good.

The test function performs a gRPC health check for the Admin service and asserts the expected status.


91-108: New test function for gRPC health check of Health service looks good.

The test function performs a gRPC health check for the Health service and asserts the expected status.


110-126: New test function for gRPC health check of unknown service looks good.

The test function performs a gRPC health check for an unknown service and asserts the expected error.


143-156: New test function for HTTP health check of Yorkie service looks good.

The test function performs an HTTP health check for the Yorkie service and asserts the expected status and response.


158-171: New test function for HTTP health check of Admin service looks good.

The test function performs an HTTP health check for the Admin service and asserts the expected status and response.


173-186: New test function for HTTP health check of Health service looks good.

The test function performs an HTTP health check for the Health service and asserts the expected status and response.


188-196: New test function for HTTP health check of unknown service looks good.

The test function performs an HTTP health check for an unknown service and asserts the expected status and response.

@krapie
Copy link
Member

krapie commented Aug 6, 2024

@taeng0204 If you finish your work, please re-request review.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
test/integration/health_test.go (1)

Line range hint 36-57:
Consider using subtests for better structure.

Using subtests can help organize the checks for known and unknown services, making the test output more readable.

func TestRPCHealthCheck(t *testing.T) {
	// use gRPC health check
	t.Run("KnownService", func(t *testing.T) {
		conn, err := grpc.Dial(
			defaultServer.RPCAddr(),
			grpc.WithTransportCredentials(insecure.NewCredentials()),
		)
		assert.NoError(t, err)
		defer func() {
			assert.NoError(t, conn.Close())
		}()

		cli := healthpb.NewHealthClient(conn)
		resp, err := cli.Check(context.Background(), &healthpb.HealthCheckRequest{})
		assert.NoError(t, err)
		assert.Equal(t, resp.Status, healthpb.HealthCheckResponse_SERVING)
	})

	// use gRPC health check for unknown service
	t.Run("UnknownService", func(t *testing.T) {
		conn, err := grpc.Dial(
			defaultServer.RPCAddr(),
			grpc.WithTransportCredentials(insecure.NewCredentials()),
		)
		assert.NoError(t, err)
		defer func() {
			assert.NoError(t, conn.Close())
		}()

		cli := healthpb.NewHealthClient(conn)
		_, err = cli.Check(context.Background(), &healthpb.HealthCheckRequest{
			Service: "unknown",
		})
		assert.Error(t, err)
	})
}
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 0bfba1f and 1f596d7.

Files selected for processing (1)
  • test/integration/health_test.go (2 hunks)
Additional comments not posted (7)
test/integration/health_test.go (7)

59-76: LGTM!

The function correctly performs a gRPC health check for the Yorkie service.


78-95: LGTM!

The function correctly performs a gRPC health check for the Admin service.


97-114: LGTM!

The function correctly performs a gRPC health check for the health service.


139-152: LGTM!

The function correctly performs an HTTP health check for the Yorkie service.


154-167: LGTM!

The function correctly performs an HTTP health check for the Admin service.


169-181: LGTM!

The function correctly performs an HTTP health check for the health service.


23-30: LGTM!

The imports are necessary and correctly added to support the new tests.

test/integration/health_test.go Outdated Show resolved Hide resolved
@taeng0204 taeng0204 requested a review from krapie August 7, 2024 05:12
Modified test functions for each service to simplify them
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1f596d7 and 932cf62.

Files selected for processing (1)
  • test/integration/health_test.go (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • test/integration/health_test.go

@taeng0204
Copy link
Member Author

@krapie I've completed it to requirements.
Thanks for your help. ☺️
Does Yorkie have any documentation or guidelines for writing test code?

Copy link
Member

@krapie krapie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Test code looks good to me.
One last suggestion before merging the PR.

Does Yorkie have any documentation or guidelines for writing test code?

We only have overall guideline in CONTRIBUTING.MD file.

server/rpc/server.go Outdated Show resolved Hide resolved
@taeng0204 taeng0204 requested a review from krapie August 7, 2024 12:47
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 932cf62 and 18e682c.

Files selected for processing (3)
  • server/rpc/httphealth/httphealth.go (1 hunks)
  • server/rpc/server.go (2 hunks)
  • test/integration/health_test.go (2 hunks)
Additional comments not posted (10)
server/rpc/httphealth/httphealth.go (3)

1-18: File header and package declaration look good.

The file header and package declaration are standard and correctly formatted.


20-30: Imports and type definition look good.

The imported packages are relevant, and the CheckResponse type is correctly defined.


32-62: NewHandler function looks good.

The function correctly handles HTTP GET requests, checks service health, and returns a JSON response. Error handling is appropriately implemented.

test/integration/health_test.go (4)

Line range hint 1-22:
File header and package declaration look good.

The file header and package declaration are standard and correctly formatted.


23-40: Imports and global variables look good.

The imported packages are relevant, and the services variable is correctly defined.


Line range hint 42-79:
TestRPCHealthCheck function looks good.

The function correctly tests the gRPC health checks for default, known, and unknown services, including proper error handling.


81-124: TestHTTPHealthCheck function looks good.

The function correctly tests the HTTP health checks for default, known, and unknown services, including proper error handling.

server/rpc/server.go (3)

Line range hint 1-18:
File header and package declaration look good.

The file header and package declaration are standard and correctly formatted.


Line range hint 19-39:
Imports look good.

The imported packages are relevant and correctly organized.


Line range hint 41-79:
NewServer function looks good.

The function correctly initializes the server, sets up health check handlers, and includes proper error handling.

Copy link
Member

@krapie krapie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@krapie krapie merged commit 495a54f into yorkie-team:main Aug 7, 2024
4 checks passed
hackerwins pushed a commit that referenced this pull request Aug 8, 2024
Added the handler to allow health checks to be performed with plain HTTP GET
requests needed for traditional uptime checker or load balancer, along with
existing gRPC health check.
@krapie
Copy link
Member

krapie commented Aug 9, 2024

@taeng0204 There are two things that I want to share:

  1. I think we need to also support HTTP HEAD method to use UptimeRobot. We are currently using the free version, and it only supports HTTP HEAD method (I thought GET method was also supported on free tier)
  2. We need to think about the path /healthz/ again in cluster condition. In yorkie-cluster, envoy proxy uses the same path for their own health check. So we have to explicitly route to our own Yorkie service, or change the path to something like /yorkie.v1/healthz.

@taeng0204
Copy link
Member Author

@krapie I've processed it in this branch and created a new PR as we talked about today.
Please check it out 😄

raararaara pushed a commit that referenced this pull request Oct 7, 2024
Added the handler to allow health checks to be performed with plain HTTP GET
requests needed for traditional uptime checker or load balancer, along with
existing gRPC health check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement 🌟 New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add Plain HTTP Health Check Endpoint
3 participants