Skip to content

config: Add support for YAML #336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 33 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ Available Commands:
compile Statically check SQL for syntax and type errors
generate Generate Go code from SQL
help Help about any command
init Create an empty sqlc.json settings file
init Create an empty sqlc.yaml settings file
version Print the sqlc version number

Flags:
Expand All @@ -267,24 +267,19 @@ Use "sqlc [command] --help" for more information about a command.

## Settings

The `sqlc` tool is configured via a `sqlc.json` file. This file must be
The `sqlc` tool is configured via a `sqlc.yaml` file. This file must be
in the directory where the `sqlc` command is run.

```json
{
"version": "1",
"packages": [
{
"name": "db",
"emit_json_tags": true,
"emit_prepared_queries": false,
"emit_interface": true,
"path": "internal/db",
"queries": "./sql/query/",
"schema": "./sql/schema/"
}
]
}
```yaml
version: "1"
packages:
- name: "db",
emit_json_tags: true
emit_prepared_queries: false
emit_interface: true
path: "internal/db"
queries: "./sql/query/"
schema: "./sql/schema/"
```

Each package document has the following keys:
Expand Down Expand Up @@ -315,17 +310,12 @@ If a different Go package for UUIDs is required, specify the package in the
`overrides` array. In this case, I'm going to use the `github.com/gofrs/uuid`
instead.

```
{
"version": "1",
"packages": [...],
"overrides": [
{
"go_type": "github.com/gofrs/uuid.UUID",
"db_type": "uuid"
}
]
}
```yaml
version: "1"
packages: [...]
overrides:
- go_type: "github.com/gofrs/uuid.UUID"
db_type: "uuid"
```

Each override document has the following keys:
Expand All @@ -345,37 +335,25 @@ This may be configured by specifying the `column` property in the override defin
should be of the form `table.column` buy you may be even more specify by specifying `schema.table.column`
or `catalog.schema.table.column`.

```
{
"version": "1",
"packages": [...],
"overrides": [
{
"column": "authors.id",
"go_type": "github.com/segmentio/ksuid.KSUID"
}
]
}
```yaml
version: "1"
packages: [...]
overrides:
- column: "authors.id"
go_type: "github.com/segmentio/ksuid.KSUID"
```

### Package Level Overrides

Overrides can be configured globally, as demonstrated in the previous sections, or they can be configured on a per-package which
scopes the override behavior to just a single package:

```
{
"version": "1",
"packages": [
{
...
"overrides": [...]
}
],
}
```yaml
version: "1"
packages:
- overrides: [...]
```


### Renaming Struct Fields

Struct field names are generated from column names using a simple algorithm:
Expand All @@ -392,14 +370,11 @@ If you're not happy with a field's generated name, use the `rename` dictionary
to pick a new name. The keys are column names and the values are the struct
field name to use.

```json
{
"version": "1",
"packages": [...],
"rename": {
"spotify_url": "SpotifyURL"
}
}
```yaml
version: "1"
packages: [...]
rename:
spotify_url: "SpotifyURL"
```

## Downloads
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ require (
golang.org/x/sys v0.0.0-20191220220014-0732a990476f // indirect
google.golang.org/genproto v0.0.0-20191223191004-3caeed10a8bf // indirect
google.golang.org/grpc v1.26.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71
vitess.io/vitess v0.0.0-20200119095853-bd8205ebca4a
)
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,8 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7 h1:VUgggvou5XRW9mHwD/yXxIYSMtY0zoKQf/v226p2nyo=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 h1:Xe2gvTZUJpsvOWUnvmL/tmhVBZUmHSvLbMjRj6NUUKo=
gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
Expand Down
10 changes: 5 additions & 5 deletions internal/cmd/cmd.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package cmd

import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
Expand All @@ -10,6 +9,7 @@ import (
"path/filepath"

"github.com/spf13/cobra"
yaml "gopkg.in/yaml.v3"

"github.com/kyleconroy/sqlc/internal/config"
)
Expand Down Expand Up @@ -47,16 +47,16 @@ var versionCmd = &cobra.Command{

var initCmd = &cobra.Command{
Use: "init",
Short: "Create an empty sqlc.json settings file",
Short: "Create an empty sqlc.yaml settings file",
RunE: func(cmd *cobra.Command, args []string) error {
if _, err := os.Stat("sqlc.json"); !os.IsNotExist(err) {
if _, err := os.Stat("sqlc.yaml"); !os.IsNotExist(err) {
return nil
}
blob, err := json.MarshalIndent(config.Config{Version: "1"}, "", " ")
blob, err := yaml.Marshal(config.V1GenerateSettings{Version: "1"})
if err != nil {
return err
}
return ioutil.WriteFile("sqlc.json", blob, 0644)
return ioutil.WriteFile("sqlc.yaml", blob, 0644)
},
}

Expand Down
30 changes: 29 additions & 1 deletion internal/cmd/generate.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ package cmd

import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"

Expand Down Expand Up @@ -34,7 +36,33 @@ func printFileErr(stderr io.Writer, dir string, fileErr dinosql.FileErr) {
}

func Generate(dir string, stderr io.Writer) (map[string]string, error) {
blob, err := ioutil.ReadFile(filepath.Join(dir, "sqlc.json"))
var yamlMissing, jsonMissing bool
yamlPath := filepath.Join(dir, "sqlc.yaml")
jsonPath := filepath.Join(dir, "sqlc.json")

if _, err := os.Stat(yamlPath); os.IsNotExist(err) {
yamlMissing = true
}
if _, err := os.Stat(jsonPath); os.IsNotExist(err) {
jsonMissing = true
}

if yamlMissing && jsonMissing {
fmt.Fprintln(stderr, "error parsing sqlc.json: file does not exist")
return nil, errors.New("config file missing")
}

if !yamlMissing && !jsonMissing {
fmt.Fprintln(stderr, "error parsing sqlc.json: both files present")
return nil, errors.New("sqlc.json and sqlc.yaml present")
}

configPath := yamlPath
if yamlMissing {
configPath = jsonPath
}

blob, err := ioutil.ReadFile(configPath)
if err != nil {
fmt.Fprintln(stderr, "error parsing sqlc.json: file does not exist")
return nil, err
Expand Down
56 changes: 29 additions & 27 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ package config

import (
"bytes"
"encoding/json"
"errors"
"fmt"
"go/types"
Expand All @@ -11,6 +10,8 @@ import (
"strings"

"github.com/kyleconroy/sqlc/internal/pg"

yaml "gopkg.in/yaml.v3"
)

const errMessageNoVersion = `The configuration file must have a version number.
Expand All @@ -29,7 +30,7 @@ The only supported version is "1".
const errMessageNoPackages = `No packages are configured`

type versionSetting struct {
Number string `json:"version"`
Number string `json:"version" yaml:"version"`
}

type Engine string
Expand All @@ -40,57 +41,57 @@ const (
)

type Config struct {
Version string `json:"version"`
SQL []SQL `json:"sql"`
Gen Gen `json:"overrides,omitempty"`
Version string `json:"version" yaml:"version"`
SQL []SQL `json:"sql" yaml:"sql"`
Gen Gen `json:"overrides,omitempty" yaml:"overrides"`
}

type Gen struct {
Go *GenGo `json:"go,omitempty"`
Go *GenGo `json:"go,omitempty" yaml:"go"`
}

type GenGo struct {
Overrides []Override `json:"overrides,omitempty"`
Rename map[string]string `json:"rename,omitempty"`
Overrides []Override `json:"overrides,omitempty" yaml:"overrides"`
Rename map[string]string `json:"rename,omitempty" yaml:"rename"`
}

type SQL struct {
Engine Engine `json:"engine,omitempty"`
Schema string `json:"schema"`
Queries string `json:"queries"`
Gen SQLGen `json:"gen"`
Engine Engine `json:"engine,omitempty" yaml:"engine"`
Schema string `json:"schema" yaml:"schema"`
Queries string `json:"queries" yaml:"queries"`
Gen SQLGen `json:"gen" yaml:"gen"`
}

type SQLGen struct {
Go *SQLGo `json:"go,omitempty"`
Go *SQLGo `json:"go,omitempty" yaml:"go"`
}

type SQLGo struct {
EmitInterface bool `json:"emit_interface"`
EmitJSONTags bool `json:"emit_json_tags"`
EmitPreparedQueries bool `json:"emit_prepared_queries"`
Package string `json:"package"`
Out string `json:"out"`
Overrides []Override `json:"overrides,omitempty"`
Rename map[string]string `json:"rename,omitempty"`
EmitInterface bool `json:"emit_interface" yaml:"emit_interface"`
EmitJSONTags bool `json:"emit_json_tags" yaml:"emit_json_tags"`
EmitPreparedQueries bool `json:"emit_prepared_queries" yaml:"emit_prepared_queries":`
Package string `json:"package" yaml:"package"`
Out string `json:"out" yaml:"out"`
Overrides []Override `json:"overrides,omitempty" yaml:"overrides"`
Rename map[string]string `json:"rename,omitempty" yaml:"rename"`
}

type Override struct {
// name of the golang type to use, e.g. `github.com/segmentio/ksuid.KSUID`
GoType string `json:"go_type"`
GoType string `json:"go_type" yaml:"go_type"`

// fully qualified name of the Go type, e.g. `github.com/segmentio/ksuid.KSUID`
DBType string `json:"db_type"`
Deprecated_PostgresType string `json:"postgres_type"`
DBType string `json:"db_type" yaml:"db_type"`
Deprecated_PostgresType string `json:"postgres_type" yaml:"postgres_type"`

// for global overrides only when two different engines are in use
Engine Engine `json:"engine,omitempty"`
Engine Engine `json:"engine,omitempty" yaml:"engine"`

// True if the GoType should override if the maching postgres type is nullable
Null bool `json:"null"`
Null bool `json:"null" yaml:"null"`

// fully qualified name of the column, e.g. `accounts.id`
Column string `json:"column"`
Column string `json:"column" yaml:"column"`

ColumnName string
Table pg.FQN
Expand Down Expand Up @@ -202,8 +203,9 @@ func ParseConfig(rd io.Reader) (Config, error) {
var buf bytes.Buffer
var config Config
var version versionSetting

ver := io.TeeReader(rd, &buf)
dec := json.NewDecoder(ver)
dec := yaml.NewDecoder(ver)
if err := dec.Decode(&version); err != nil {
return config, err
}
Expand Down
3 changes: 2 additions & 1 deletion internal/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ func TestBadConfigs(t *testing.T) {
},
{
"unknown fields",
"json: unknown field \"foo\"",
`yaml: unmarshal errors:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not happy about this error changing. Going forward, we shouldn't be exposing an error message directory from a dependency.

line 3: field foo not found in type config.V1GenerateSettings`,
unknownFields,
},
} {
Expand Down
Loading