This is a backend service skeleton or boilerplate to speed up development process. Over time, this package has become opinionated and behaves more like a framework with a set of predefined features.
This package was developed using GoLang v1.21.
Thank you for using Codoworks Go Boilerplate.
___ _ _
/ __\___ __| | _____ _____ _ __| | _____
/ / / _ \ / _ |/ _ \ \ /\ / / _ \| '__| |/ / __|
/ /__| (_) | (_| | (_) \ V V / (_) | | | <\__ \
\____/\___/ \__,_|\___/ \_/\_/ \___/|_| |_|\_\___/ cli.
Mindset
This service is designed for developers to build backend API as quickly as possible - almost as simply as copy and pasting components. The goal is to be able to clone this repository, rename it and get started with your first RESTful CRUD API within minutes.
There are many HTTP libraries on GitHub that enable quick and easy API development, but as your project scales it can quickly become messy without proper structure and workflows, and even more when preparing for production or managing security.
Codoworks Go Boilerplate has all your production needs taken care of in advance, so you can focus your efforts on creating business and application logics.
It's worth noting that this package builds upon Echo but you can easily adapt it to a different framework.
It's designed with 3 separate routers (public, protected and hidden). Each router has its individual configuration that you can customise to your needs. This enables the following structure:
https://your-domain.com/api/<your public api> # Publicly consumable API
https://api.your-domain.com/<your protected api> # Your application's API
https://api.your-domain.com/.admin/<your hidden api> # Administrative API, not supposed to be available via public internet
With this structure, the default router is assumed to be the protected one as most use cases tend to contain a user interface element with a login functionality.
Speaking of login, this service is designed to be used with self-hosted ory kratos for authentication. Since this boilerplate is designed to be modular, you may choose another service provider instead of Kratos. All you have to do is change the authentication.go middleware to your desired service.
Similarly for authorisation, ory keto is the default client for this service as it is well-designed to manage large volumes of transactions.
Often, you may need to run something in the background like a clean up job or perhaps an email watcher. This service is designed with that in mind too. It even provides a watcher that you can run with an http server or independently. Using the same structure you can create your own too.
A well-designed service should also enable the user to perform certain administrative tasks out of the box, like providing a specific user with a given email address or system admin permission. It's likely that a user interface for such feature is not a priority on your product roadmap, hence an API is never developed and as a result, you cannot make use of Postman. To prevent this type of scenario, this boilerplate is shipped with tasks that you can easily extend and execute via command line.
Needless to say, almost every service requires a database, this one is no different too. Codowworks Go Boilerplate is designed with MySQL, Postgres and SQLite integration. By default, it uses SQLite to allow a quick start, switching platforms is just a matter of changing an environment variable.
Within this boilerplate are also database migrations, logging, routing, hot-reloading, CORS, timeout and even graceful shutdown, which are some of the features you need to get to production as soon as possible.
We hope you enjoy using Codoworks Go Boilerplate. If you do, please support us by giving this repository a star.
Full Feature List
- CLI commands (via Cobra)
- HTTP server (via Echo)
- Public router
- Protected router
- Hidden router
- Daemon processes or workers
- Tasks for custom one-off operations
- Middlewares
- HTTP header checks and setters
- Auto error handling and response
- Authentication via Ory Kratos
- Authorisation via Ory Keto
- CORS handling
- Auto trim trailing slashes
- Request timeout
- Gzip responses
- XSS check
- Databases
- DB connection (PostgreSQL, MySQL)
- DB models (ORM using Gorm)
- DB migrations (using Go Migrate)
- DB seeds (using Go Migrate)
- JSON forms and model mapping
- Data validation
- Clients
- Forward HTTP client to forward authorization headers
- Ory Kratos Client - authentication
- Ory Keto Client - authorization
- Custom logger
- Graceful shutdown
- Feature toggle: [ory_kratos, ory_keto, db, redis]
- Clone this repository
git clone git@github.com:codoworks/go-boilerplate.git - Run
cd go-boilerplate - Run
go get - Run
go run . db migrate - Run
go run . db seed - Run
go run . startto start the server, you should see the following:
⇨ http server started on [::]:8081
⇨ http server started on [::]:8080
⇨ http server started on [::]:8079
- List available routes using
go run . info protected-api-routesand use your favourite API client to test. or use the following to get started and make sure you're up and running.
curl -H "Accept: application/json" http://127.0.0.1:8081/health/alive
curl -H "Accept: application/json" http://127.0.0.1:8081/health/readyRecommended: run
go run .and explore all available options, it should be straightforward.
For more details on running and using the service, scroll down to "Operations" section.
To learn about developing and extending this service, scroll down to "make it your own" section.
Docker
The service is shipped with a few Docker compose files to get you started, all of which are automated with a Makefile to make things consistent.
From the boilerplate root folder, run the quick-start target from the Makefile.
make quick-startTo run an example using MySQL database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile.
make quick-start-mysqlTo run an example using Postgres database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile.
make quick-start-postgresUsage
Environment variables are evaluated in the following order to allow flexibility when running in production:
.envfile- environment variables
- cmd flags (if available)
During development, it is recommended to use a .env file. You can find a reference under /.env.sample` to get started.
To ease your development process, we've included a command to print the environment to better understand your app behaviour. Simply run go run . info env. Together with go run . info features, you should be able to get to the bottom of an issue.
List of all available env vars
| Var Name | Required | Description |
|---|---|---|
HOST |
optional | service host address. default: 0.0.0.0 |
PROTECTED_API_PORT |
optional | Service port. Default: 8080 |
PUBLIC_API_PORT |
optional | Service port. Default: 8081 |
HIDDEN_API_PORT |
optional | Service port. Default: 8079 |
DB_HOST |
optional | Database host |
DB_PORT |
optional | Database port |
DB_USER |
optional | Database username |
DB_PASSWORD |
optional | Database password |
DB_NAME |
optional | Database name |
DB_TIMEZONE |
optional | Database timezone. Required with Postgres platform |
DB_PLATFORM |
optional | Enum: ["postgres", "mysql", "sqlite"]. default: "sqlite" |
KRATOS_PUBLIC_SERVICE |
optional | Ory Kratos public API URL |
KRATOS_ADMIN_SERVICE |
optional | Ory Kratos admin API URL |
KETO_READ_SERVICE |
optional | Ory Keto read API URL |
KETO_WRITE_SERVICE |
optional | Ory Keto write API URL |
REDIS_HOST |
optional | Redis host URL. Required if Redis is enabled |
REDIS_PORT |
optional | Redis port. Required if Redis is enabled |
REDIS_PASSWORD |
optional | Redis password. Required if Redis is enabled |
LOG_LEVEL |
optional | Enum: ["info", "warn", "debug", "error"]. default: "info" |
CORS_ALLOW_ORIGINS |
optional | Allowed origins. Default: "*" |
REQUEST_TIMEOUT_DURATION |
optional | Number in seconds. Default: "60" |
DISABLE_FEATURES |
optional | List of features to disable in runtime, make sure its comma separated without spaces |
The service can run in one of two modes: production mode or development mode.
Development mode is activated using the -d or --dev flag. Running in this mode will lock the service host to 127.0.0.1 to avoid firewall issues when developing using MacOS. You can override this setting using -H 0.0.0.0 if needed.
Development mode will also activate useful middlewares that help print incoming request body, input data validation errors for debugging, and set the logger level to debug for ease of development. Everything else is identical to running in production mode.
You can change the behaviour of the service using flags, see the list of flags below for more.
List of all flags
| Flag Name | Shorthand | type | Description |
|---|---|---|---|
--dev |
-d |
bool | Run in development mode |
--env |
-e |
bool | Print environment variables |
--host |
-H |
string | (optional) Service host. Overrides env vars |
--port |
-P |
string | (optional) Service port. Overrides env vars |
--watcher |
(N/A) | bool | (optional) Start watcher in the backgoround |
--log |
-l |
string | (optional) Log level |
It is convenient to automatically restart the service every time you save your changes. For that, you can use air, which is a separate Go package you can install using the following command:
go install github.com/cosmtrek/air@latestOnce air is installed, you simply need to run air to start the service. Configurations for this can be found under ./.air.toml.
Live reloading will also work in Docker. The Dockerfile.dev is configured to install and run the service via air.
This service is shipped with a cmd client, which means you can use ./go-boilerplate to view all available commands and help menu.
- You need to build the service first before you can use
go-boilerplate- both
./go-boilerplateandgo run .can be followed by any flags, commands and sub-commands
The service requires Accept: application/json header by default for all requests.
It also requires Content-Type: application/json with POST, PUT and DELETE requests.
If you're writing a small project with a few endpoints then running Go in your terminal shouldn't be much of a problem. You can use live-reload while you're editing your code in your favourite editor.
To run the service without building, run go run . which will achieve the same result as running ./go-boilerplate after building the binary.
The name
go-boilerplatewill change if you change the package name as mentioned here.
However, when you are running a large project with multiple micro-services (multiple instances of this boilerplate), it can be handy to live edit your code while in Docker. For this, we have designed the Dockerfile.dev to get you started.
Simply run make quick-start to get up and running. To stop it, use Ctrl+C.
To build, run go build . which will generate a binary with the default name of the package. In this case, it will be ./go-boilerplate unless you change it (which is recommended).
If you have executed the above, you may notice that the version ./go-boilerplate version is set to 2.x.x-default during run time. That's because it is the second iteration of this boilerplate. It is recommended that you burn the version into the binary in build time to create versioned builds. To do that, use the following command to build:
go build -ldflags="-w -s -extldflags '-static' -X main.VERSION=<YOUR.VERION.HERE>"
# Example
go build -ldflags="-w -s -extldflags '-static' -X main.VERSION=1.0.0"
./go-boilerplate version
# v1.0.0Once built, a single binary file is generated. It is an executable file that you can rename and place in any folder as long as your profile PATH can find it. A good place to place it on your local machine would be in /usr/bin which is where most binaries are.
If you wish to deploy this service locally, all you need to do is build as per the section above then ship the outputted binary into a location where your terminal's PATH can find it. You should be able to use it just by calling its name in your terminal.
The "Usage" section should get you familiarised with all the parameters that are configurable. Deploying it should not be a problem in any dockerised environment.
From a containerisation perspective, I'd encourage you to place this binary in an empty container i.e. FROM scratch in your Dockerfile. This helps keep the container size to a minimum. When tested on an M1 Mac Machine, we got an 18MB container.
Extending the service (make it your own)
This section is all about extending the service to create your own application and APIs.
The first thing you should do is to change the package name, find
github.com/codoworks/go-boilerplatein all the files and replace it with your own package name. You can choose to use the generalgithub.com/(org-name)/(project-name)naming pattern for consistency.
Migrations
Migrations help create your database and track how it evolves overtime. Here, we use GoMigrate to achieve this. Some added complexity is added to enable easy extendability and generate better logs throughout your development process.
Migrations go under pkg/db/migrations/<myNewMigration>.go. Its implemention uses Go's init() function, which means they're added to the list in alphabetical order. They migrate in that order (top to bottom) and rollback in the reverse order (bottom up). For this, it is best to maintain the naming convention of YYYYMMDD[00-99]_migration_description.
Here's a sample migration to get you started:
func init() {
m := &gormigrate.Migration{}
m.ID = "2022081801_create_cats_table"
m.Migrate = func(db *gorm.DB) error {
type Cat struct {
models.ModelBase
Name string `gorm:"size:255"`
Type string `gorm:"size:255"`
}
return AutoMigrateAndLog(db, &Cat{}, m.ID)
}
m.Rollback = func(db *gorm.DB) error {
if err := db.Migrator().DropTable("cats"); err != nil {
logFail(m.ID, err, true)
}
logSuccess(m.ID, true)
return nil
}
AddMigration(m)
}The variable m holds the migration details and is added to the list of migrations at the end. m.ID is the identifier used by gomigrate to keep track of the migrations that already ran. So, make sure to change that for every migration.
Every migration has 2 methods to be implemented, the Migrate() and Rollback() method as described above. Make sure you use the logSuccess, logFail and AutoMigrateAndLog() functions to print the migrations that ran. This will come in very handy for remote deployments.
It's recommended to declare your models within each migration (separately from the models package) to keep track of the database schema change through time. You can add or delete columns, rename columns, and execute raw SQL in migrations.
A general good practice would be to flatten your migrations once your application achieves version 1, leaving only neat table creation in each migration.
Seeds
Seeds are very similar to migrations, but seeds do not implement the Rollback function.
Just like migrations, seeds are applied once and tracked using their unique identifier ID by GoMigrate.
Seeds are part of the whole package which allows you to access models, clients and other components directly to configure the application, and perhaps provide dummy data to help with development.
Here's a seed skeleton to get you started. Copy the following structure into a new file under seeds and change the s.ID property.
func init() {
var s = &gormigrate.Migration{}
s.ID = "2022081801_new_seed"
s.Migrate = func(db *gorm.DB) error {
logSuccess(s.ID)
return nil
}
AddSeed(s)
}And here's a sample seed to give an idea of how you can utilise seeds.
func init() {
var s = &gormigrate.Migration{}
s.ID = "2022081801_seed_cats_data"
s.Migrate = func(db *gorm.DB) error {
var err error
var cats []*models.Cat
cats = append(cats, &models.Cat{
Name: "Kitty",
Type: "Persian",
})
cats = append(cats, &models.Cat{
Name: "Tom",
Type: "Siamese",
})
for _, cat := range cats {
err = cat.Save()
if err != nil {
logFail(s.ID, err)
return err
}
}
logSuccess(s.ID)
return nil
}
AddSeed(s)
}Models
Models can sometimes be a complex aspect of any application. In this section, you'll find a rundown on how you can compose your models or database entities.
The first thing is to create a struct that matches your database schema. Almost all models should embed the ModelBase struct that provides the ID, CreatedAt and UpdatedAt properties. Exceptions may include a many-to-many table where you only need to store 2 identifiers. To learn more about model declarations, you can refer to Gorm's official comprehensive documentation.
Here's a Cat model that should correspond to a Cats table that contains 5 properties i.e. ID, CreatedAt, UpdatedAt, Name and Type in a database.
type Cat struct {
ModelBase
Name string `gorm:"size:255"`
Type string `gorm:"size:255"`
}Notice how every property contains a gorm decoration to specify things like field size, uniqueness or foreign keys etc. For more details, please refer to Gorm's documentation.
Your model may sometimes contain properties that do not correspond to a database column. To do that, you simply need to use the gorm:"-" decoration.
Note: Given that this package is designed to work with multiple database servers like MySQL or Postgres, some data types may be available in some servers and not others. It's worth testing your application with different servers from time to time to accomodate easy switching of database server, unless your use case relies on a specific data type - in which case you're making a calculated decision to lock your application to that server.
Now that you have a structure that corresponds to a table in your database, some common functionality is in order. Generally, one would at least expect the basic CRUD functionality. Here's a basic CRUD implementation that is required for any model:
FindAll(), for retrieving all records in the table
func (model *Cat) FindAll() (models []*Cat, err error) {
result := db.Model(model).Find(&models)
return models, result.Error
}FindMany(), for retrieving many items given an array of IDs
func (model *Cat) FindMany(ids []string) (models []*Cat, err error) {
result := db.Model(model).Find(&models, ids)
return models, result.Error
}Find(), for retrieving a single item with a given ID
func (model *Cat) Find(id string) (m *Cat, err error) {
result := db.Model(model).Where("ID=?", id).First(&m)
return m, result.Error
}Save(), for creating a new record in the database and assigning a new ID to it
func (model *Cat) Save() error {
return db.Model(model).Create(&model).Error
}'Update(), for updating a record in the database given an existing ID
func (model *Cat) Update() error {
return db.Model(model).Updates(&model).Error
}Delete(), for deleting a record in the database given an existing ID
func (model *Cat) Delete(id string) error {
return db.Model(model).Where("ID=?", id).Delete(&model).Error
}All of the above functions will return an error if they cannot perform what they're supposed to. That's useful to inform users if the data they're looking for exists or is stored. For detailed utilisation of these functions, check out the handlers folder.
These functions are not abstracted to allow granular control over each model, as each individual model can quickly morph into something very large with child elements, preload functions and pagination.
Given the basic functionality defined in the previous section, we've created the ability to do something like the following:
...
catModel := &Cat{}
myCat, err := catModel.Find(catID)
if err != nil {
fmt.Println("couldn't find cat with ID", catID)
}
...The problem with the code above is that you will need to instantiate a new struct catModel from &Cat{} in order to have a pointer receiver that can call the Find() function. You can avoid that by using the following common getter structure for all models, right at the top of the model before its declaration to maintain consistency.
var cat *Cat = &Cat{}
func CatModel() *Cat {
return cat
}The above will now create a singleton pattern that you can access from any component within the package like models.CatModel().Find().
Note: the
CatModel()method should only be used to fetch data from the database. Saving, updating and deleting data should be applied to an actual instance that has been returned through aFind(),FindAll()orFindMany()function.
Once you have retrieved the records needed from the database, you may want to send those records as a response. To do that, you can use forms. Every model is expected to have at least one method named MapToForm() that returns a JSON representation of that model.
Forms are basic structures that may or may not exactly match all the properties that a model has. The reason it has been done this way is to enable multiple forms where one can contain all model properties e.g. intended for an admin user to view, while another may contain a sanitised version of that model e.g. intended only for a read-only user.
For more details on creating a form, scroll down to the forms section below. Here you'll find a sample implementation of MapToForm() function.
func (model *Cat) MapToForm() *CatForm {
form := &CatForm{
Name: model.Name,
Type: model.Type,
}
form.ID = model.ID
form.CreatedAt = model.CreatedAt
form.UpdatedAt = model.UpdatedAt
return form
}Here's a complete code as a model sample that you can copy as a base model.
var cat *Cat = &Cat{}
func CatModel() *Cat {
return cat
}
type Cat struct {
ModelBase
Name string `gorm:"size:255"`
Type string `gorm:"size:255"`
}
func (model *Cat) MapToForm() *CatForm {
form := &CatForm{
Name: model.Name,
Type: model.Type,
}
form.ID = model.ID
form.CreatedAt = model.CreatedAt
form.UpdatedAt = model.UpdatedAt
return form
}
func (model *Cat) FindAll() (models []*Cat, err error) {
result := db.Model(model).Find(&models)
return models, result.Error
}
func (model *Cat) FindMany(ids []string) (models []*Cat, err error) {
result := db.Model(model).Find(&models, ids)
return models, result.Error
}
func (model *Cat) Find(id string) (m *Cat, err error) {
result := db.Model(model).Where("ID=?", id).First(&m)
return m, result.Error
}
func (model *Cat) Save() error {
return db.Model(model).Create(&model).Error
}
func (model *Cat) Update() error {
return db.Model(model).Updates(&model).Error
}
func (model *Cat) Delete(id string) error {
return db.Model(model).Where("ID=?", id).Delete(&model).Error
}Copy the code above and replace the name Cat to get started.
Forms
Forms are data contracts that are used to send responses to clients and receive/ bind user input.
Each model can have many forms to enable sending specific values with different endpoints. An example scenario would be having an admin with full access to all data in a record whereas a customer has access only to a subset of that data.
Data validation is applied to fields in forms. Here's a sample form to get you started.
type CatForm struct {
FormBase
Name string `json:"name" validate:"required,min=2,max=50"`
Type string `json:"type" validate:"required,min=2,max=80"`
}
func (form *CatForm) MapToModel() *Cat {
return &Cat{
Name: form.Name,
Type: form.Type,
}
}The FormBase struct provides the ID, CreatedAt and UpdatedAt fields.
Each field should specify the name mapping in JSON format along with validation rules. For more on validations check out the Playground Validator documentation. To skip validations all together, use validate:"-".
Finally, each form should have a MapToModel() function that returns a model, so it can be stored after it has been validated. Note that forms do not set a model's ID property as that is the job of the model. Instead, it must be set manually prior to a database operation. Think of this like an actual form you fill up that has a section "for office use only".
Handlers
Note: This go boilerplate uses Echo. If you're ever in doubt, you can refer to Echo's documentation for more details on what's possible with routers.
A handler is any function with the func (c echo.Context) error signature. All handlers should be stored under pkg/api/handlers and categorized in directories following their entity name in plural form. For readability and maintainability, we encourage maintaining a single handler in a single file as we all know that Go files can quickly grow.
Handlers should also be nested, which means a Cats handlers directory can contain a sub directory, such as cats/tags, that helps avoid long file names and improve readability.
How handlers look will largely depend on your project's business logic and requirements. For reference, here's a quick sample to give an idea on how you should construct your handler.
func Get(c echo.Context) error {
id := c.Param("id")
if id == "" {
return helpers.Error(c, constants.ERROR_ID_NOT_FOUND, nil)
}
m, err := models.CatModel().Find(id)
if err != nil {
return helpers.Error(c, err, nil)
}
return c.JSON(http.StatusOK, handlers.Success(m.MapToForm()))
}And perhaps another example to demonstrate how to receive user input and store a model.
func Post(c echo.Context) error {
f := &models.CatForm{}
if err := c.Bind(f); err != nil {
return helpers.Error(c, constants.ERROR_BINDING_BODY, err)
}
if err := helpers.Validate(f); err != nil {
return c.JSON(http.StatusBadRequest, handlers.ValidationErrors(err))
}
m := f.MapToModel()
if err := m.Save(); err != nil {
return helpers.Error(c, err, nil)
}
return c.JSON(http.StatusOK, handlers.Success(m.MapToForm()))
}Routers
Note: This go boilerplate uses Echo. If you're ever in doubt, you can refer to Echo's documentation for more details on what's possible with routers.
This boilerplate is shipped with 3 routers, public, privdate and hidden routers - all of which ollow the same structure and procedure with slight differences in what is registered within each.
Why have 3 routers? Well, some projects may have public and protected routes, and such use case is straightforward. The latter implements an authentication middleware while the first does not. Attempting to achieve such behaviour within a single router can be tricky, so isolated routers running on different ports are used instead. The third "hidden" router is provided to enable a pattern commonly used to allow one microservice to communicate with another without exposing those routes to the public internet. With that said, wiring those 3 routers can easily be achieved through a different service like Kubernetes or NGINX.
All routers should go through the following process:
- Initialisation
- Checking for DevMode and enabling related middlewares
- Register common middleware
- Register health routes
- Register security middleware
- Register user-defined routes
- Register error handler
Have a look at pkg/api/routers/protectedApi.go to familiarise yourself with router initialisation process. If you've already created your handlers from the previous section, all you need is to add your new route to this file as such:
func registerProtectedAPIRoutes() {
cats := protectedApiRouter.Echo.Group("/cats") // Your new REST resource
cats.GET("", catsHandlers.Index) // GET "/cats/" route and handler
cats.GET("/:id", catsHandlers.Get) // GET "/cats/:id" route and handler
cats.POST("", catsHandlers.Post) // POST "/cats/" route and handler
cats.PUT("/:id", catsHandlers.Put) // PUT "/cats/:id" route and handler
cats.DELETE("/:id", catsHandlers.Delete) // DELETE "/cats/:id" route and handler
// add more routes here ...
}Tasks
Tasks is a way to extend the command line CLI without having to go through the trouble of understanding the initialisation process.
To create a new task, simply add the following sample into a new file ./pkg/tasks/<myTask>.go:
func init() {
var t = &Task{
Name: "myTask",
Description: "This is my task",
RequiredArgs: []string{"key1", "key2"}, // add args here
Run: execMyTask,
}
Tasks.AddTask(t)
}
func execMyTask(env *TaskEnv, args map[string]string) error {
// task implementation goes here...
fmt.Println("My task is executed!")
return nil
}Tasks are automatically injected with an env object that contains the environment. They are also injected with an args map containing any values added to the exec command, as long as they're separated by '=' e.g. key1=value1 key2=value2 key3=value3 .
You can also set the required arguments in myTask.RequiredArgs = []string{"key1", "key2"} to prevent execution until all arguments are provided.
To execute the task above, simply run go run . task exec myTask and you should get the "My first task is executed!" message.
Error Handling
This boilerplate automates error handling and error responses.
First, let's talk about logging errors. When using proper logging mechanics and log levels, you can then leave all your logs in the code and have them printed depending on their severity. The package is shipped with the function helpers.Error(), a wrapper that's intended to log an error and return it. These logs will only be visible if the LOG_LEVEL env var permits. Avoid using fmt.Println() at all times, instead, use logger.Debug() or if you're within a handler, you can use the c.Logger().Debug() helper.
Given that each handler can return an error, the router is configured to handle the error using pkg/api/handlers/errors/automatedHttpError.go which will unwrap the error and match it with a list of registered errors under pkg/api/handlers/errors/errors.go. Finally, it will construct an error response and respond to that request.
Validation errors are no different, except they're unwrapped further and sent to the user as individual form inputs so they can be displayed.
You're encouraged to register and maintain as many errors as you can in the same way. It's useful to have a specific error code mapped to each error, that way we can determine exactly what went wrong in each user flow.
Adding Env Vars & Features
All environment variables reside in pkg/config/features. They're categorised within their respective features such as database.go or service.go. Each env var must have a mapstructure: decoration that spells it in caps when parsing an ENV. You can add your own, it's as simple as adding a new line in any of these files, or create your own.
Below is a sample of pkg/config/features/service.go:
type ServiceConfig struct {
Host string `mapstructure:"HOST"`
ProtectedApiPort string `mapstructure:"PROTECTED_API_PORT"`
PublicApiPort string `mapstructure:"PUBLIC_API_PORT"`
HiddenApiPort string `mapstructure:"HIDDEN_API_PORT"`
LogLevel string `mapstructure:"LOG_LEVEL"`
RequestTimeoutDuration string `mapstructure:"REQUEST_TIMEOUT_DURATION"`
WatcherSleepInterval string `mapstructure:"WATCHER_SLEEP_INTERVAL"`
}
var service = &Feature{
Name: constants.FEATURE_SERVICE,
Config: &ServiceConfig{},
enabled: true,
configured: false,
ready: false,
requirements: []string{
"Host",
"ProtectedApiPort",
"PublicApiPort",
"HiddenApiPort",
"LogLevel",
"RequestTimeoutDuration",
"WatcherSleepInterval",
},
}
func init() {
Features.Add(service)
}From the example above, you can find a type ServiceConfig that states what env vars are to be expected. These are automatically read from the environment. Env vars must belong to a feature which can be toggled on or off. A feature can also define which env vars are required for it to start.
If you wish to disable a feature, you can mention it in the list of DISABLE_FEATURES var in run-time.
Reading the env vars is the job of pkg/config/envVars.go. Each config struct must be registered in envVars.go. The config struct is then automatically injected to its respective feature after initialisation.
It is possible to set a default value for each variable, this can be done in pkg/config/envVars.go under setDefaults().
By the time the CMD calls the Proc, all env vars should have already been read and injected into their features, making them available for the rest of the package.
Folder Structure
The package is split into 3 directories
| Directory | Description |
|---|---|
/ci |
Contains all files related to building or deploying the service such as Docker, Docker compose, configuration and K8S files |
/cmd |
Contains all available commands |
/pkg |
Contains all source code files. This is where you'll be spending most of your time |
| Directory | Description |
|---|---|
/pkg/api |
Everything related to Echo, routers and handlers go in here |
/pkg/clients |
These are clients used throughout the service. They can be third-party services or simple config providers for workflows |
/pkg/config |
Service configuration and environment variable management |
/pkg/db |
Everything related to database entities and models, migrations, and seed data |
/pkg/proc |
Entry points for all processes |
/pkg/tasks |
User-defined tasks available via the command line CLI |
/pkg/utils |
General utilities used throughout the package that do not belong to any specific package |
Package tree view
+- pkg
| +- api
| | +- handlers
| | | +- healthz
| | | +- errors
| | | +- cats <--- example handlers
| | | +- <--- additional handlers
| | +- helpers
| | | +- helpers.go
| | | +- <--- additional helpers
| | +- middlewares
| | | +- authentication.go
| | | +- authorization.go
| | | +- <--- additional middleware
| | +- routers
| | | +- router.go
| | | +- hiddenApi.go
| | | +- protectedApi.go
| | | +- publicApi.go
| | | +- <--- additional routers
| | +- api.go
| +- clients
| | +- db
| | +- fhttp
| | +- keto
| | +- kratos
| | +- logger
| | +- redis
| +- config
| | +- autoEnv.go
| | +- config.go
| | +- feature.go
| | +- features.go
| | +- flags.go
| | +- service.go
| +- db
| | +- migrations
| | | +- migrations.go <--- list of migrations to run, be sure to add yours here
| | | +- <--- additional migrations
| | +- models
| | | +- models.go
| | | +- forms.go
| | | +- cat.go <--- example model
| | | +- catForm.go <--- example form
| | | +- <--- additional routers
| | +- seeds
| | | +- seeds.go <--- list of seeds to run, be sure to add yours here
| | | +- <--- additional seeds
| +- proc <--- entry point to all processes
| | +- proc.go
| | +- hiddenApi.go
| | +- protectedApi.go
| | +- publicApi.go
| | +- watcher.go
| +- tasks
| | +- myFirstTask.go
| | +- <--- additional tasks
| +- utils
| | +- constants
| | | +- constants.go <--- all literal values
| | +- init.go
| | +- utils.go <--- reusable functions that don't belong anywhere else
+- go.mod
+- main.go
Dependencies
This package is purely written in Go, which helps with dependency management. All dependencies can be easily installed using the go get command.
There are only 2 optional dependencies that can be installed separately. The first is Air used for live-reload, and the other is Docker.
List of run-time dependencies:
- GoLang v1.20
- Cobra v1.8.0
- Viper v1.18.2
- Echo v4.11.3
- Gorm v1.25.6
- MySQL v1.5.2
- PostgreSQL v1.25.6
- SQLite v1.5.4
- GoMigrate v2.1.1
- Playground Validator v10.17.0
- Ory Kratos v1.0.0
- Ory Keto v0.5.2
- Redis
List of development dependencies:
Known Issues
- Gorm v1.25.6 and v1.25.7 are known to cause issues with PostgreSQL database. If you experience an error
this driver does not support LastInsertID(), try downgrading Gorm to v1.25.5
Roadmap
- Enhanced routers
- Feature toggle for [
hiddenApi,protectedApi,publicApi] - Keto client
- Redis client
- Forward HTTP client
- Enhanced error handling
- Quick start examples
- Example with Kratos for authentication
- Example with Keto for authorisation
- Code cleanup and in-line documentation
- Swagger integration
- Postman collection
- More documentation
- Tests
- Github Actions
- Landing page / Website
Feel free to start a new discussion, submit a new PR, make a feature request or etc.. If you would like to join the team, reach out to us on Discord. We are always looking for contributors!
