Gautocloud provides a simple abstraction that golang based applications can use to discover information about the cloud environment on which they are running, to connect to services automatically with ease of use in mind. It provides out-of-the-box support for discovering common services on Heroku, Cloud Foundry and kubernetes cloud platforms, and it supports custom automatic connectors.
This project can be assimilated to the spring-cloud-connector project but for golang (and with its own concepts).
- Usage by example
- Default connectors
- Community connectors
- Cloud Environments
- Concept
- Create your own connector
- Create your own Cloud Environment
- Use it without the facade
- Use mocked facade for your test
- Run tests
- Contributing
- FAQ
Let's define a context: We are in a Cloud Foundry environment where we connect a MySql service on our application.
We now wants to use this service without parsing a json or anything else to have a MySql client to use our service. Gautocloud is here to help for this kind of use case.
This software will retrieve all services found in your environment and will pass informations from service to what we call a connector. A connector is responsible to create, in our context, a MySql client which make it available in your program.
You only needs to import a connector to make it usable by gautocloud. This system of import let you have only what you need to run your app (=do not create a huge binary) and let the possibility to create custom connector.
Example (provide a *net/sql.DB struct):
package main
import (
"fmt"
"github.com/cloudfoundry-community/gautocloud"
"os"
log "github.com/sirupsen/logrus"
_ "github.com/cloudfoundry-community/gautocloud/connectors/databases/client/mysql" // this register the connector mysql to gautocloud
"github.com/cloudfoundry-community/gautocloud/connectors/databases/dbtype"
)
func init(){
// Gautocloud use logrus as logger, see: https://github.com/sirupsen/logrus
// When using facade, first log messages are emitted with default level from logrus (INFO) and debug message cannot be seen.
// To be able to see log message on load simply add the env var `GAUTOCLOUD_DEBUG` to `true`, you will be able to
// see debug message from gautocloud (you can also set `json` instead of `true` to see logs as json).
log.SetLevel(log.DebugLevel)
}
func main() {
appInfo := gautocloud.GetAppInfo() // retrieve all informations about your application instance
fmt.Println(appInfo.Name) // give the app name
// by injection
var c *dbtype.MysqlDB // this is just a wrapper of *net/sql.DB you can use as normal sql.DB client
err := gautocloud.Inject(&c) // you can also use gautocloud.InjectFromId("mysql", &c) where "mysql" is the id of the connector to use
if err != nil {
panic(err)
}
defer c.Close()
// c is now useable as a *sql.DB
// e.g.: err = c.Ping()
// or you can also do by return
// data, err := gautocloud.GetFirst("mysql")
// if err != nil {
// panic(err)
// }
// c = data.(*dbtype.MysqlDB)
}Imagine now that we have multiple MySql services connected to your Cloud Foundry app, you can also have multiple client:
package main
import (
"github.com/cloudfoundry-community/gautocloud"
_ "github.com/cloudfoundry-community/gautocloud/connectors/databases/client/mysql" // this register the connector mysql to gautocloud
"github.com/cloudfoundry-community/gautocloud/connectors/databases/dbtype"
)
func main() {
// by injection
var cs []*dbtype.MysqlDB // this is just a wrapper of *net/sql.DB you can use as normal sql.DB client
err := gautocloud.Inject(&cs) // you can also use gautocloud.InjectFromId("mysql", &cs) where "mysql" is the id of the connector to use
if err != nil {
panic(err)
}
// you have now a slice containing all mysql client you can have
// or you can also do by return
// data, err := gautocloud.GetAll("mysql") // a connector may give you different types that's why GetAll return a slice of interface{}
// cs = make([]*dbtype.MysqlDB,0)
// for _, elt := range data {
// svcSlice = append(cs, elt.(*dbtype.MysqlDB))
// }
}Tip: You can either do the same thing without the gautocloud facade, see: use gautocloud without facade
Doc for default connectors can be found here: /docs/connectors.md.
You can see connectors made by the community on the dedicated wiki page: https://github.com/cloudfoundry-community/gautocloud/wiki/Connectors
- Cloud Detection: if the
VCAP_APPLICATIONenv var exists and not empty - Service detection by name: Look if a service in
VCAP_SERVICESmatch the name required by a connector. - Service detection by tags: Look if a service in
VCAP_SERVICESmatch one of tag required by a connector. - App information id: id of the app given by Cloud Foundry
- App information name: name of the app given during
cf push - App information properties:
uris: (type: []string) list of routes associated to the apps.host: (type: string) host of the app.home: (type: string) root folder for the deployed app.index: (type: int) index of the app.memory_limit: (type: string) maximum amount of memory that each instance of the application can consume.space_id: (type: string) id of the space.space_name_id: (type: string) name of the space.temp_dir: (type: string) directory location where temporary and staging files are stored.user: (type: string) user account under which the container runs.version: (type: string) version of the app.working_dir: (type: string) present working directory, where the buildpack that processed the application ran.
Tip: you can also use in local but settings the env var DYNO and create env var corresponding
to a service you want to connect. (see: /test-integration/test_integration_test.go as an example)
- Cloud Detection: if the
DYNOenv var exists - Service detection by name: Look all env var which contains the name required by a connector. Env var key are after parsed to create credentials. Example:
you have env var:
- `MY_SVC_NAME=myname`
- `MY_SVC_HOST=localhost`
Connector required name `SVC`.
CloudEnv decode `MY_SVC_NAME` to [MY, SVC, NAME] and [MY, SVC, VALUE]
and detect that there is SVC in those two env var.
It returns a service with credentials:
{
"name": "myname",
"host": "localhost"
}
Note:
- if a env var key doesn't contain
_(e.g.:SVC=localhost) it will give those credentials:{"svc": "localhost", "uri": "localhost"}. - If the value of an env var is in json it will be decode the json and pass content as credentials
Retrieves:
- Service detection by tags: each tag work like by name.
- App information id: id of the app given by the env var
DYNO - App information name: Set the env var
GAUTOCLOUD_APP_NAMEto give a name to your app instead it will be<unknown> - App information properties:
host: (type: string) host of the app.
- Cloud Detection: if the
KUBERNETES_PORTenv var exists - Service detection by name: Look all env var which contains the name required by a connector. Env var key are after parsed to create credentials. Example:
you have env var:
- `MY_SVC_SERVICE_NAME=myname`
- `MY_SVC_SERVICE_HOST=localhost`
Connector required name `SVC`.
CloudEnv remove `_SERVICE` from the key and decode `MY_SVC_NAME` to [MY, SVC, NAME] and [MY, SVC, VALUE]
and detect that there is SVC in those two env var.
It returns a service with credentials:
{
"name": "myname",
"host": "localhost"
}
Note:
- if a env var key doesn't contains
_(e.g.:SVC=localhost) it will give those credentials:{"svc": "localhost", "uri": "localhost"}. - If the value of an env var is in json it will be decode the json and pass content as credentials
Retrieves:
- Service detection by tags: each tag work like by name.
- App information id: id of the app given by the env var
HOSTNAME - App information name: Name of the app given by the env var
HOSTNAME - App information properties:
host: (type: string) host of the app.- All values starting by
KUBERNETESin env vars key.
This is a special CloudEnv and can be considered as a fake one. This is cloud env is always triggered if none cloud env was found.
You can also use it to be able to use a config file directly without pain (but it's not 12 factors).
You can set the env var CLOUD_FILE which contains the path of a configuration files containing services.
You can create a config file called config.yml in your current working directory or set the env var CONFIG_FILE
which contains the path of your config file. It can contain anything you want, this will register by itself a service
named config with tag config which contains you configuration from the file.
A config file or a cloud file can be a yml, json, toml or hcl file.
A config file can contains anything you want, this will register by itself a service named config with tag config
which contains you configuration from the file.
A cloud file must follow this pattern (example in yml):
app_name: "myapp" # set the app name you want (it can be not set)
services:
- name: myelephantsql
tags: [postgresql, service] # ...
credentials:
uri: postgres://seilbmbd:PHxTPJSn@babar.elephantsql.com:5432/seilbmbd
host: babar.elephantsql.com
# ... you can have other credentialsYou can see how to follow the same pattern with other format here: /cloudenv/local_cloudenv_test.go#L13-L86.
- Cloud Detection: Always detected, used as fallback when no cloud env found.
- Service detection by name: Look if a service in the config file match the name required by a connector.
- Service detection by tags: Look if a service in the config file match one of tag required by a connector.
- App information id: random uuid
- App information name: The name given in the config file, if not set it will be
<unknown> - App information properties: None
NOTE: A config service is always created which permit to use ConfigFileInterceptor
this is a great interceptor to use with a generic config connector.
Gautocloud have a lot of black magics but in fact the concept is quite simple.
- Loader: It has the responsibility to find the CloudEnv where your program run, store Connectors and retrieve services from CloudEnv which corresponds to one or many Connector, and finally it will pass to Connector the service and store the result from connector.
- Gautocloud facade: This facade was made to make things easier for users. It stores one instance of a Loader
and give the ability to make lazy loading (this is why to register a Connector you only need to do
import _ "a/connector") - CloudEnv: Each CloudEnv correspond to a real cloud. It manages the detection of the environment but also the detections of services asked by the Loader.
- Connector: A connector register itself on the loader when using Gautocloud Facade. It handles the conversion of a service to a real client or structure which can be manipulated after by user.
- CloudDecoder: This decoder do the conversion of a service to an expected schema. In Gautocloud context this decoder is used to convert a given service to an expected schema given by a connector. This decoder can be used in other context. (see: /decoder/decoder.go to know about it)
The best way is to look at an example here: /connectors/connector_example_test.go.
You can also want to see how to create connector with interceptor here: /connectors/intercepter_example_test.go.
Note:
- An interceptor work like a http middleware. This permit to intercept data which will be given back by gautocloud and modified it before giving back to user. Interceptor should be used in a connector, to do so, connector have to implement ConnectorIntercepter:
type ConnectorIntercepter interface {
Intercepter() interceptor.Intercepter
}- Add your connector on the dedicated wiki page: https://github.com/cloudfoundry-community/gautocloud/wiki/Connectors
The best way to implement yourself a cloud environment, is to look at the following interface: /cloudenv/cloudenv.go.
You will need to load you cloud env after by use gautocloud without facade, you can either do a pull request to had your cloud environment as builtin by doing a pull request.
We will take the same example as we see in Usage by example, but we will not use the facade this time:
package main
import (
"fmt"
"github.com/cloudfoundry-community/gautocloud/loader"
"github.com/cloudfoundry-community/gautocloud/cloudenv"
log "github.com/sirupsen/logrus"
"os"
"github.com/cloudfoundry-community/gautocloud/connectors/databases/client/mysql" // this register the connector mysql to gautocloud
"github.com/cloudfoundry-community/gautocloud/connectors/databases/dbtype"
)
func main() {
// Gautocloud use logrus as logger, see: https://github.com/sirupsen/logrus
// In this case reload connectors is not necessary to see logs.
log.SetLevel(log.DebugLevel)
ld := loader.NewLoader(
[]cloudenv.CloudEnv{
cloudenv.NewCfCloudEnv(),
cloudenv.NewHerokuCloudEnv(),
cloudenv.NewKubernetesCloudEnv(),
cloudenv.NewLocalCloudEnv(),
},
)
ld.RegisterConnector(mysql.NewMysqlConnector()) // you need to manually register connectors
appInfo := ld.GetAppInfo() // retrieve all informations about your application instance
fmt.Println(appInfo.Name) // give the app name
fmt.Println(appInfo.Port) // give the port to listen to
// by injection
var c *dbtype.MysqlDB // this is just a wrapper of *net/sql.DB you can use as normal sql.DB client
err := ld.Inject(&c) // you can also use gautocloud.InjectFromId("mysql", &c) where "mysql" is the id of the connector to use
if err != nil {
panic(err)
}
defer c.Close()
}If you need to write your tests with a mocked gautocloud (and use the facade), you can ask to go compiler to get a mocked version.
To perform this, simply run your test with the tags gautocloud_mock (e.g.: go test -tags gautocloud_mock).
The facade will load a gomock version of the loader. You can find, for example, how to perform injection with this mocked version here: /test-mock/test_mock_test.go (see also gomock documentation to learn more)
Requirements:
Requirements for integration tests:
Note: We need docker and docker-compose for integrations to run services and ensure clients works.
Simply run in a terminal bin/test.sh.
Any PR or/and issues are welcomes.
Don't be shy to send a PR to add another cloud environment as a builtin one.
Why do I need to import a connector even if it's a builtin one ?
You need to import it because if you didn't have to it, it will require to load all default connectors with associated dependencies to the connector which can make a huge binary.
In our case, it will compile only what you need by importing the connector.
Why the CloudEnv interface works with a concept of tags and name?
This concept comes directly from Cloud Foundry and the way it gives services. Cloud Foundry has an API called service brokers this api return services with tags and name.
This concept will now be used in the future by a lot of cloud environment (PaaS and CaaS) because this api have now a dedicated governance managed by people from Google (Kubernetes in mind), Pivotal (Cloud Foundry), Red Hat (Openshift) ... You can found their website here: https://www.openservicebrokerapi.org/