Skip to content

Commit

Permalink
feat: open source grace
Browse files Browse the repository at this point in the history
This moves the internally-reviewed set of changes for open source to
GitHub. The project itself should be fully functional and feature
complete.

Signed-off-by: Rob Liebowitz <rliebowitz@morningconsult.com>
  • Loading branch information
rliebz committed Aug 21, 2023
1 parent d6dac64 commit 8693b80
Show file tree
Hide file tree
Showing 12 changed files with 1,799 additions and 4 deletions.
9 changes: 9 additions & 0 deletions .commitlintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
module.exports = {
extends: ["@commitlint/config-conventional"],
rules: {
"body-max-line-length": [2, "always", 200],
"subject-case": [2, "never", ["start-case", "pascal-case", "upper-case"]],
"subject-empty": [1, "never"],
"type-empty": [1, "never"],
},
};
109 changes: 109 additions & 0 deletions .golangci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
issues:
max-same-issues: 0
exclude-use-default: false
exclude-rules:
- path: '_test\.go'
linters:
- bodyclose
- gocognit
- goconst
- gocyclo
- gosec
- lll
- prealloc

# Overly picky
- linters: [revive]
text: 'package-comments'
- linters: [revive]
text: 'if-return'

# Duplicates of errcheck
- linters: [gosec]
text: 'G104: Errors unhandled'
- linters: [gosec]
text: 'G307: Deferring unsafe method'
# Not a good rule since it ignores defaults
- linters: [gosec]
text: 'G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server'

# Contexts are best assigned defensively
- linters: [ineffassign]
text: 'ineffectual assignment to `ctx`'
- linters: [staticcheck]
text: 'SA4006: this value of `ctx` is never used'

# Irrelevant for test examples
- linters: [gocritic]
path: example_test\.go
text: 'exitAfterDefer'

run:
timeout: 5m

linters:
enable:
- bodyclose
- errcheck
- errchkjson
- exportloopref
- goconst
- gocognit
- gocritic
- gocyclo
- godot
- gofumpt
- goimports
- gosec
- lll
- misspell
- nakedret
- nolintlint
- prealloc
- revive
- unconvert
- unparam

linters-settings:
errcheck:
exclude-functions:
# Errors we wouldn't act on after checking
- (*database/sql.DB).Close
- (*database/sql.Rows).Close
- (io.Closer).Close
- (*os.File).Close
- (net/http.ResponseWriter).Write

# Handled by errchkjson
- encoding/json.Marshal
- encoding/json.MarshalIndent
- (*encoding/json.Encoder).Encode

gocognit:
min-complexity: 10

goconst:
min-len: 0
min-occurrences: 3

gocritic:
disabled-checks:
- appendAssign

gocyclo:
min-complexity: 10

goimports:
local-prefixes: github.com/morningconsult/grace

golint:
min-confidence: 0

govet:
check-shadowing: true

nakedret:
max-func-lines: 0

revive:
confidence: 0
210 changes: 206 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,209 @@
# .github
Meta repository for all Morning Consult projects
# grace

[![Go Reference](https://pkg.go.dev/badge/github.com/morningconsult/grace.svg)](https://pkg.go.dev/github.com/morningconsult/grace)

# What is this?
A Go library for starting and stopping applications gracefully.

It is supposed to apply default metadata files to all projects. I learned about it from [terraform-aws-modules](https://github.com/terraform-aws-modules/.github)
Grace facilitates gracefully starting and stopping a Go web application.
It helps with waiting for dependencies - such as sidecar upstreams - to be available
and handling operating system signals to shut down.

Requires Go >= 1.21.

## Usage

In your project directory:

```shell
go get github.com/morningconsult/grace
```

## Features

* Graceful handling of upstream dependencies that might not be available when
your application starts
* Graceful shutdown of multiple HTTP servers when operating system signals are
received, allowing in-flight requests to finish.
* Automatic startup and control of a dedicated health check HTTP server.
* Passing of signal context to other non-HTTP components with a generic
function signature.

### Gracefully shutting down an application

Many HTTP applications need to handle graceful shutdowns so that in-flight requests
are not terminated, leaving an unsatisfactory experience for the requester. Grace
helps with this by catching operating system signals and allowing your HTTP servers
to finish processing requests before being forcefully stopped.

To use this, add something similar to the following example to the end of your
application's entrypoint. `grace.Run` should be returned in your entrypoint/main
function.

An absolute minimal configuration to get a graceful server would be the following:

```go
ctx := context.Background()
httpHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Write([]byte("hello there"))
})

// This is the absolute minimum configuration necessary to have a gracefully
// shutdown server.
g := grace.New(ctx, grace.WithServer("localhost:9090", httpHandler))
err := g.Run(ctx)
```

Additionally, it will also handle setting up a health check server with any check functions
necessary. The health server will be shut down as soon as a signal is caught. This
helps to ensure that the orchestration system running your application marks it as unhealthy
and stops sending it any new requests, while the in-flight requests to your actual
application are still allowed to finish gracefully.

An minimal example with a health check server and your application server would be similar
to the following:

```go
httpHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Write([]byte("hello there"))
})

dbPinger := grace.HealthCheckerFunc(func(ctx context.Context) error {
// ping database
return nil
})

g := grace.New(
ctx,
grace.WithHealthCheckServer("localhost:9092", grace.WithCheckers(dbPinger)),
grace.WithServer("localhost:9090", httpHandler, grace.WithServerName("api")),
)
```

A full example with multiple servers, background jobs, and health checks:

```go
// Set up database pools, other application things, server handlers,
// etc.
httpHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Write([]byte("hello there"))
})

metricsHandler := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Write([]byte("here are the metrics"))
})

dbPinger := grace.HealthCheckerFunc(func(ctx context.Context) error {
// ping database
return nil
})

redisPinger := grace.HealthCheckerFunc(func(ctx context.Context) error {
// ping redis.
return nil
})

bgWorker := func(ctx context.Context) error {
// Start some background work
return nil
}

// Create the new grace instance with your addresses/handlers.
// Here, we create:
//
// 1. A health check server listening on 0.0.0.0:9092 that will
// respond to requests at /-/live and /-/ready, running the dbPinger
// and redisPinger functions for each request to /-/ready.
// This overrides the default endpoints of /livez and /readyz.
// 2. Our application server on localhost:9090 with the httpHandler.
// It specifies the default read and write timeouts, and a graceful
// stop timeout of 10 seconds.
// 3. Our metrics server on localhost:9091, with a shorter stop timeout
// of 5 seconds.
// 4. A function to start a background worker process that will be called
// with the context to be notified from OS signals, allowing for background
// processes to also get stopped when a signal is received.
// 5. A custom list of operating system signals to intercept that override the
// defaults.
g := grace.New(
ctx,
grace.WithHealthCheckServer(
"0.0.0.0:9092",
grace.WithCheckers(dbPinger, redisPinger),
grace.WithLivenessEndpoint("/-/live"),
grace.WithReadinessEndpoint("/-/ready"),
),
grace.WithServer(
"localhost:9090",
httpHandler,
grace.WithServerName("api"),
grace.WithServerReadTimeout(grace.DefaultReadTimeout),
grace.WithServerStopTimeout(10*time.Second),
grace.WithServerWriteTimeout(grace.DefaultWriteTimeout),
),
grace.WithServer(
"localhost:9091",
metricsHandler,
grace.WithServerName("metrics"),
grace.WithServerStopTimeout(5*time.Second),
),
grace.WithBackgroundJobs(bgWorker),
grace.WithStopSignals(
os.Interrupt,
syscall.SIGHUP,
syscall.SIGTERM,
),
)

if err = g.Run(ctx); err != nil {
log.Fatal(err)
}
```

### Waiting for dependencies

If your application has upstream dependencies, such as a sidecar that exposes a
remote database, you can use grace to wait for them to be available before
attempting a connection.

At the top of your application's entrypoint (before setting up database connections!)
use the `Wait` method to wait for specific addresses to respond to TCP/HTTP pings before
continuing with your application setup:

```go
err := grace.Wait(
ctx,
10*time.Second,
grace.WithWaitForTCP("localhost:6379"), // redis
grace.WithWaitForTCP("localhost:5432"), // postgres
grace.WithWaitForHTTP("http://localhost:9200"), // elasticsearch
grace.WithWaitForHTTP("http://localhost:19000/ready"), // envoy sidecar
)
if err != nil {
log.Fatal(err)
}
```

## Local Development

### Testing

#### Linting

The project uses [`golangci-lint`](https://golangci-lint.run) for linting. Run
with

```sh
golangci-lint run
```

Configuration is found in:

- `./.golangci.yaml` - Linter configuration.

#### Unit Tests

Run unit tests with

```sh
go test ./...
```
Loading

0 comments on commit 8693b80

Please sign in to comment.