Skip to content

feat: add support for Postgres multi dimensional arrays in Go #2309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 30 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
c33cb8e
feat: pg multi dim arrays
colli173 May 30, 2022
4630b30
add e2e test
colli173 May 30, 2022
e80ace6
update array dims on table alter
colli173 May 30, 2022
a9aa0f9
remove accidental insert statement
colli173 May 30, 2022
cdaa60c
Merge branch 'main' of https://github.com/colli173/sqlc into feature/…
colli173 Jul 2, 2022
9e78e66
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Jul 2, 2022
4e389d1
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Aug 4, 2022
2000d6d
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Aug 12, 2022
a007b7f
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Aug 27, 2022
f5d1ad9
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Sep 2, 2022
5cc5c5f
regenerate to fix mismatch in testdata generated code version
colli173 Sep 3, 2022
1b954c3
regenerate codegen json
colli173 Sep 3, 2022
a3bc887
fix merge conflicts
colli173 Feb 14, 2023
ddc0b2a
madd check for pgxv4 driver in go type
colli173 Feb 14, 2023
cb169fb
initial attempt at pgtype.Array vs pgtype.FlatArray
colli173 Feb 14, 2023
b8b0bc5
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Feb 16, 2023
21e5c0b
use pgxtypes flatarray and array
colli173 Feb 16, 2023
0ff26c0
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Feb 23, 2023
f2b511a
busing same approach with non-pgx types as pull/2090
colli173 Feb 23, 2023
1a62698
remove newline
colli173 Feb 23, 2023
6f2e193
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Feb 27, 2023
ceb0626
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Mar 1, 2023
84eda98
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Mar 7, 2023
1ac031e
fix merge conflicts
colli173 Apr 13, 2023
fc5ef8e
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Apr 20, 2023
8ed8493
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Apr 21, 2023
7adb257
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 Apr 24, 2023
a5a66e3
Merge branch 'main' of https://github.com/kyleconroy/sqlc into featur…
colli173 May 1, 2023
dbf287e
Merge branch 'main' into fork-colli173-feature/pg-multi-dim-arrays
kyleconroy Jun 7, 2023
be4a324
Regenerate proto files
kyleconroy Jun 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions internal/cmd/shim.go
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,11 @@ func pluginCatalog(c *catalog.Catalog) *plugin.Catalog {
Schema: c.Type.Schema,
Name: c.Type.Name,
},
Comment: c.Comment,
NotNull: c.IsNotNull,
IsArray: c.IsArray,
Length: int32(l),
Comment: c.Comment,
NotNull: c.IsNotNull,
IsArray: c.IsArray,
ArrayBounds: int32(c.ArrayBounds),
Length: int32(l),
Table: &plugin.Identifier{
Catalog: t.Rel.Catalog,
Schema: t.Rel.Schema,
Expand Down Expand Up @@ -246,6 +247,7 @@ func pluginQueryColumn(c *compiler.Column) *plugin.Column {
Comment: c.Comment,
NotNull: c.NotNull,
IsArray: c.IsArray,
ArrayBounds: int32(c.ArrayBounds),
Length: int32(l),
IsNamedParam: c.IsNamedParam,
IsFuncCall: c.IsFuncCall,
Expand Down
6 changes: 5 additions & 1 deletion internal/codegen/golang/go_type.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,11 @@ func goType(req *plugin.CodeGenRequest, col *plugin.Column) string {
}
typ := goInnerType(req, col)
if col.IsArray || col.IsSqlcSlice {
return "[]" + typ
dims := ""
for i := int32(0); i < col.ArrayBounds; i++ {
dims += "[]"
}
return dims + typ
}
return typ
}
Expand Down
51 changes: 27 additions & 24 deletions internal/compiler/output_columns.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@ func (c *Compiler) OutputColumns(stmt ast.Node) ([]*catalog.Column, error) {
catCols := make([]*catalog.Column, 0, len(cols))
for _, col := range cols {
catCols = append(catCols, &catalog.Column{
Name: col.Name,
Type: ast.TypeName{Name: col.DataType},
IsNotNull: col.NotNull,
IsArray: col.IsArray,
Comment: col.Comment,
Length: col.Length,
Name: col.Name,
Type: ast.TypeName{Name: col.DataType},
IsNotNull: col.NotNull,
IsArray: col.IsArray,
ArrayBounds: col.ArrayBounds,
Comment: col.Comment,
Length: col.Length,
})
}
return catCols, nil
Expand Down Expand Up @@ -248,15 +249,16 @@ func outputColumns(qc *QueryCatalog, node ast.Node) ([]*Column, error) {
cname = *res.Name
}
cols = append(cols, &Column{
Name: cname,
Type: c.Type,
Scope: scope,
Table: c.Table,
TableAlias: t.Rel.Name,
DataType: c.DataType,
NotNull: c.NotNull,
IsArray: c.IsArray,
Length: c.Length,
Name: cname,
Type: c.Type,
Scope: scope,
Table: c.Table,
TableAlias: t.Rel.Name,
DataType: c.DataType,
NotNull: c.NotNull,
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Length: c.Length,
})
}
}
Expand Down Expand Up @@ -545,15 +547,16 @@ func outputColumnRefs(res *ast.ResTarget, tables []*Table, node *ast.ColumnRef)
cname = *res.Name
}
cols = append(cols, &Column{
Name: cname,
Type: c.Type,
Table: c.Table,
TableAlias: alias,
DataType: c.DataType,
NotNull: c.NotNull,
IsArray: c.IsArray,
Length: c.Length,
EmbedTable: c.EmbedTable,
Name: cname,
Type: c.Type,
Table: c.Table,
TableAlias: alias,
DataType: c.DataType,
NotNull: c.NotNull,
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Length: c.Length,
EmbedTable: c.EmbedTable,
})
}
}
Expand Down
1 change: 1 addition & 0 deletions internal/compiler/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ type Column struct {
DataType string
NotNull bool
IsArray bool
ArrayBounds int
Comment string
Length *int
IsNamedParam bool
Expand Down
15 changes: 8 additions & 7 deletions internal/compiler/query_catalog.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,14 @@ func buildQueryCatalog(c *catalog.Catalog, node ast.Node, embeds rewrite.EmbedSe

func ConvertColumn(rel *ast.TableName, c *catalog.Column) *Column {
return &Column{
Table: rel,
Name: c.Name,
DataType: dataType(&c.Type),
NotNull: c.IsNotNull,
IsArray: c.IsArray,
Type: &c.Type,
Length: c.Length,
Table: rel,
Name: c.Name,
DataType: dataType(&c.Type),
NotNull: c.IsNotNull,
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Type: &c.Type,
Length: c.Length,
}
}

Expand Down
4 changes: 4 additions & 0 deletions internal/compiler/resolve.go
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ func (comp *Compiler) resolveCatalogRefs(qc *QueryCatalog, rvs []*ast.RangeVar,
DataType: dataType(&c.Type),
NotNull: p.NotNull(),
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Length: c.Length,
Table: table,
IsNamedParam: isNamed,
Expand Down Expand Up @@ -274,6 +275,7 @@ func (comp *Compiler) resolveCatalogRefs(qc *QueryCatalog, rvs []*ast.RangeVar,
DataType: dataType(&c.Type),
NotNull: p.NotNull(),
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Table: table,
IsNamedParam: isNamed,
},
Expand Down Expand Up @@ -445,6 +447,7 @@ func (comp *Compiler) resolveCatalogRefs(qc *QueryCatalog, rvs []*ast.RangeVar,
DataType: dataType(&c.Type),
NotNull: p.NotNull(),
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Table: &ast.TableName{Schema: schema, Name: rel},
Length: c.Length,
IsNamedParam: isNamed,
Expand Down Expand Up @@ -552,6 +555,7 @@ func (comp *Compiler) resolveCatalogRefs(qc *QueryCatalog, rvs []*ast.RangeVar,
DataType: dataType(&c.Type),
NotNull: c.IsNotNull,
IsArray: c.IsArray,
ArrayBounds: c.ArrayBounds,
Table: table,
IsNamedParam: isNamed,
IsSqlcSlice: p.IsSqlcSlice(),
Expand Down
9 changes: 5 additions & 4 deletions internal/compiler/to_column.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ func toColumn(n *ast.TypeName) *Column {
panic("toColumn: " + err.Error())
}
return &Column{
Type: typ,
DataType: strings.TrimPrefix(astutils.Join(n.Names, "."), "."),
NotNull: true, // XXX: How do we know if this should be null?
IsArray: isArray(n),
Type: typ,
DataType: strings.TrimPrefix(astutils.Join(n.Names, "."), "."),
NotNull: true, // XXX: How do we know if this should be null?
IsArray: isArray(n),
ArrayBounds: len(n.ArrayBounds.Items),
}
}
32 changes: 32 additions & 0 deletions internal/endtoend/testdata/multidimension_array/pgx/v4/go/db.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
CREATE TABLE bar (tags text[][] not null);

-- name: TextArray :many
SELECT * FROM bar;
13 changes: 13 additions & 0 deletions internal/endtoend/testdata/multidimension_array/pgx/v4/sqlc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"version": "1",
"packages": [
{
"path": "go",
"engine": "postgresql",
"sql_package": "pgx/v4",
"name": "querytest",
"schema": "query.sql",
"queries": "query.sql"
}
]
}
32 changes: 32 additions & 0 deletions internal/endtoend/testdata/multidimension_array/pgx/v5/go/db.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
CREATE TABLE bar (tags text[][] not null);

-- name: TextArray :many
SELECT * FROM bar;
13 changes: 13 additions & 0 deletions internal/endtoend/testdata/multidimension_array/pgx/v5/sqlc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"version": "1",
"packages": [
{
"path": "go",
"engine": "postgresql",
"sql_package": "pgx/v5",
"name": "querytest",
"schema": "query.sql",
"queries": "query.sql"
}
]
}
Loading