go-feature-flag v1 feedback

go-feature-flag is heading towards v1 and it may require a change on how to design your flags (no worries we will keep backward compatibility).
I would love to get your feedback about this change in this discussion #229

Thank you!

🎛️ go-feature-flag

Feature flags with no complex system to maintain!

Installation

go get github.com/thomaspoignant/go-feature-flag

What is go-feature-flag?

A simple and complete feature flag solution, without any complex backend system to install, all you need is a file as your backend.

No server is needed, just add a file to your central system and all your services will react to the changes in this file.

go-feature-flags supports:

• Storing your configuration flags file on various locations (HTTP, S3, GitHub, file, Google Cloud Storage ...).
• Configuring your flags in various format (JSON, TOML and YAML).
• Adding complex rules to target your users.
• Use complex rollout strategy for your flags :
  • Run A/B testing experimentation.
  • Progressively rollout a feature.
  • Schedule your flag updates.
• Exporting your flags usage data (S3, log, file, Google Cloud Storage ...).
• Getting notified when a flag has been changed (webhook and slack).

If you are not familiar with feature flags, also called feature Toggles, you can read this article from Martin Fowler where he explains why this is a great pattern.

I've also written an article explaining why feature flags can fasten your iteration cycle.

go-feature-flag.demo.mp4

The code of this demo is available in thomaspoignant/go-feature-flag-demo repository.

Getting started

First, you need to initialize the ffclient with the location of your backend file.

err := ffclient.Init(ffclient.Config{
    PollingInterval: 3 * time.Second,
    Retriever: &ffclient.HTTPRetriever{
        URL:    "http://example.com/flag-config.yaml",
    },
})
defer ffclient.Close()

This example will load a file from an HTTP endpoint and will refresh the flags every 3 seconds (if you omit the PollingInterval, the default value is 60 seconds).

Now you can evaluate your flags anywhere in your code.

user := ffuser.NewUser("user-unique-key")
hasFlag, _ := ffclient.BoolVariation("test-flag", user, false)
if hasFlag {
    // flag "test-flag" is true for the user
} else {
    // flag "test-flag" is false for the user
}

The full documentation is available on https://thomaspoignant.github.io/go-feature-flag/
You can find more examples in the examples/ directory.

Configuration

go-feature-flag needs to be initialized to be used.
During the initialization you must give a ffclient.Config{} configuration object.

ffclient.Config{} is the only location where you can put the configuration.

Example

ffclient.Init(ffclient.Config{ 
    PollingInterval:   3 * time.Second,
    Logger:         log.New(file, "/tmp/log", 0),
    Context:        context.Background(),
    Retriever:      &ffclient.FileRetriever{Path: "testdata/flag-config.yaml"},
    FileFormat:     "yaml",
    Notifiers: []ffclient.NotifierConfig{
        &ffclient.WebhookConfig{
            EndpointURL: " https://example.com/hook",
            Secret:     "Secret",
            Meta: map[string]string{
                "app.name": "my app",
            },
        },
    },
    DataExporter: ffclient.DataExporter{
        FlushInterval:   10 * time.Second,
        MaxEventInMemory: 1000,
        Exporter: &ffexporter.File{
            OutputDir: "/output-data/",
        },
    },
    StartWithRetrieverError: false,
    Environment: os.Getenv("MYAPP_ENV"),
})

Configuration fields

Field	Description
Retriever	The configuration retriever you want to use to get your flag file. See Store your flag file for the configuration details.
Context	(optional) The context used by the retriever. Default: context.Background()
Environment	(optional) The environment the app is running under, can be checked in feature flag rules. Default: ""
DataExporter	(optional) DataExporter defines how to export data on how your flags are used. see export data section for more details.
FileFormat	(optional) Format of your configuration file. Available formats are yaml, toml and json, if you omit the field it will try to unmarshal the file as a yaml file. Default: YAML
Logger	(optional) Logger used to log what go-feature-flag is doing. If no logger is provided the module will not log anything. Default: No log
Notifiers	(optional) List of notifiers to call when your flag file has been changed. See notifiers section for more details.
PollingInterval	(optional) Duration to wait before refreshing the flags. The minimum polling interval is 1 second. Default: 60 * time.Second
StartWithRetrieverError	(optional) If true, the SDK will start even if we did not get any flags from the retriever. It will serve only default values until the retriever returns the flags. The init method will not return any error if the flag file is unreachable. Default: false
Offline	(optional) If true, the SDK will not try to retrieve the flag file and will not export any data. No notification will be send neither. Default: false

Multiple configuration flag files

go-feature-flag comes ready to use out of the box by calling the Init function and it will be available everywhere.
Since most applications will want to use a single central flag configuration, the package provides this. It is similar to a singleton.

In all the examples above, they demonstrate using go-feature-flag in its singleton style approach.
You can also create many go-feature-flag clients to use in your application.
See the documentation for more details.

Where do I store my flags file?

The module supports different ways of retrieving the flag file.
Available retriever are:

• From GitHub
• From an HTTP endpoint
• From a S3 Bucket
• From a file
• From Google Cloud Storage
• From Kubernetes ConfigMaps

Flags file format

go-feature-flag core feature is to centralize all your feature flags in a source file, and to avoid hosting and maintaining a backend server to manage them.

Your file should be a YAML, JSON or TOML file with a list of flags (examples: YAML, JSON, TOML).

The easiest way to create your configuration file is to used GO Feature Flag Editor available at https://thomaspoignant.github.io/go-feature-flag-editor/.
If you prefer to do it manually please follow instruction bellow.

A flag configuration looks like:

YAML

test-flag:
  percentage: 100
  rule: key eq "random-key"
  true: true
  false: false
  default: false
  disable: false
  trackEvents: true
  version: 1
  rollout:
    experimentation:
      start: 2021-03-20T00:00:00.10-05:00
      end: 2021-03-21T00:00:00.10-05:00

  test-flag2:
    rule: key eq "not-a-key"
    percentage: 100
    true: true
    false: false
    default: false
    version: 12

JSON

{
  "test-flag": {
    "percentage": 100,
    "rule": "key eq \"random-key\"",
    "true": true,
    "false": false,
    "default": false,
    "disable": false,
    "trackEvents": true,
    "version": 1,
    "rollout": {
      "experimentation": {
        "start": "2021-03-20T05:00:00.100Z",
        "end": "2021-03-21T05:00:00.100Z"
      }
    }
  },
  "test-flag2": {
    "rule": "key eq \"not-a-key\"",
    "percentage": 100,
    "true": true,
    "false": false,
    "default": false,
    "version": 12
  }
}

TOML

[test-flag]
percentage = 100.0
rule = "key eq \"random-key\""
true = true
false = false
default = false
disable = false
trackEvents = true
version = 1.0

[test-flag.rollout]

  [test-flag.rollout.experimentation]
  start = 2021-03-20T05:00:00.100Z
  end = 2021-03-21T05:00:00.100Z

[test-flag2]
rule = "key eq \"not-a-key\""
percentage = 100.0
true = true
false = false
default = false
version = 12.0

Field	Description
flag-key	Name of your flag. It must be unique. On the example the flag keys are test-flag and test-flag2.
true	Value returned by the flag if the rule is evaluated to true and the user is in the active percentage.
false	Value returned by the flag if the rule is evaluated to true and the user is not in the active percentage.
default	Value returned by the flag if the rule is evaluated to false.
percentage	(optional) Percentage of users who should be affected by the flag. Default: 0 The percentage is computed by calculating a hash of the user key (100000 variations), it means that you can have 3 numbers after the comma.
rule	(optional) Condition to determine on which user the flag should be applied. Rule format is described in the rule format section. If no rule is set, the flag applies to all users (percentage still apply).
disable	(optional) True if the flag is disabled. Default: false
trackEvents	(optional) False if you don't want to export the data in your data exporter. Default: true
version	(optional) The version is the version of your flag. This number is used to display the information in the notifiers and data collection, you have to update it your self. Default: 0
rollout	(optional) rollout contains a specific rollout strategy you want to use. See rollout section for more details.

Rule format

The rule format is based on the nikunjy/rules library.

All the operations can be written capitalized or lowercase (ex: eq or EQ can be used).
Logical Operations supported are AND OR.

Compare Expression and their definitions (a|b means you can use either one of the two a or b):

eq|==: equals to 
ne|!=: not equals to
lt|<: less than 
gt|>: greater than
le|<=: less than equal to
ge|>=: greater than equal to 
co: contains 
sw: starts with 
ew: ends with
in: in a list
pr: present
not: not of a logical expression

Examples

• Select a specific user: key eq "example@example.com"
• Select all identified users: anonymous ne true
• Select a user with a custom property: userId eq "12345"

Users

Feature flag targeting and rollouts are all determined by the user you pass to your Variation calls. The SDK defines a User struct and a UserBuilder to make this easy.

Here's an example:

// User with only a key
user1 := ffuser.NewUser("user1-key")

// User with a key plus other attributes
user2 = ffuser.NewUserBuilder("user2-key").
 AddCustom("firstname", "John").
 AddCustom("lastname", "Doe").
 AddCustom("email", "john.doe@example.com").
 Build()

The most common attribute is the user's key and this is the only mandatory user attribute. The key should also uniquely identify each user. You can use a primary key, an e-mail address, or a hash, as long as the same user always has the same key.
We recommend using a hash if possible.
All the other attributes are optional.

ℹ️ Custom attributes are one of the most powerful features. They let you have rules on these attributes and target users according to any data that you want.

You can also distinguish logged-in users from anonymous users in the SDK (check documentation about anonymous users).

Variation

The Variation methods determine whether a flag is enabled or not for a specific user. There is a Variation method for each type:
BoolVariation , IntVariation , Float64Variation , StringVariation , JSONArrayVariation , JSONVariation

result, _ := ffclient.BoolVariation("your.feature.key", user, false)

// result is now true or false depending on the setting of
// this boolean feature flag

Variation methods take the feature flag key, a user, and a default value.

The default value is return when an error is encountered (ffclient not initialized, variation with wrong type, flag does not exist ...).

In the example, if the flag your.feature.key does not exists, result will be false.
Not that you will always have a usable value in the result.

Get all flags for a specific user

If you want to send the information about a specific user to a front-end, you will want a snapshot of all the flags for this user at a specific time.

The method ffclient.AllFlagsState returns a snapshot of flag values and metadata.
The function is evaluating all available flags for the user and return a flagstate.AllFlagsState object containing the information you need.

The MarshalJSON() function will return a JSON Object, that can be directly used by your front-end application.
More details in the documentation.

Rollout

A critical part of every new feature release is orchestrating the actual launch schedule between Product, Engineering, and Marketing teams.

Delivering powerful user experiences typically requires software teams to manage complex releases and make manual updates at inconvenient times.

But it doesn’t have to, having a complex rollout strategy allows you to have lifecycle for your flags.

Complex rollout strategy available

• Canary releases - impact randomly a subset of your users.
• Progressive rollout - increase the percentage of your flag over time.
• Scheduled rollout - update your flag over time.
• Experimentation rollout - serve your feature only for a determined time (perfect for A/B testing).

Notifiers

If you want to be informed when a flag has changed, you can configure a notifier.

A notifier will send one notification to the targeted system to inform them that a new flag configuration has been loaded.

ℹ️ go-feature-flag can handle more than one notifier at a time.

Available notifiers are:

• Slack - Get a slack message with the changes.
• Webhook - Call an API with the changes.

Export data

If you want to export data about how your flag are used, you can use the DataExporter.
It collects all the variations events and can save these events on several locations:

• File - create local files with the variation usages.
• Log - use your logger to write the variation usages.
• S3 - export your variation usages to S3.
• Google Cloud Storage - export your variation usages to Google Cloud Storage.
• Webhook - export your variation usages by calling a webhook.

Currently, we are supporting only feature events.
It represents individual flag evaluations and are considered "full fidelity" events.

An example feature event below:

{
    "kind": "feature",
    "contextKind": "anonymousUser",
    "userKey": "ABCD",
    "creationDate": 1618228297,
    "key": "test-flag",
    "variation": "Default",
    "value": false,
    "default": false
}

The format of the data is described in the documentation.

Events are collected and send in bulk to avoid spamming your exporter (see details in how to configure data export).

How to configure data export?

In your ffclient.Config add the DataExporter field and configure your export location.

To avoid spamming your location everytime you have a variation called, go-feature-flag is storing in memory all the events and send them in bulk to the exporter.
You can decide the threshold on when to send the data with the properties FlushInterval and MaxEventInMemory. The first threshold hit will export the data.

If there are some flags you don't want to export, you can use trackEvents fields on these specific flags to disable the data export (see flag file format).

Example

ffclient.Config{ 
    // ...
   DataExporter: ffclient.DataExporter{
        FlushInterval:   10 * time.Second,
        MaxEventInMemory: 1000,
        Exporter: &ffexporter.File{
            OutputDir: "/output-data/",
        },
    },
    // ...
}

The full configuration is described in the documentation.

How can I contribute?

This project is open for contribution, see the contributor's guide for some helpful tips.