raw-dogging postgresql with pgx and sqlc

content/posts/custom-type-gotcha-in-go-validator/index.en.md

@@ -152,7 +152,9 @@ the box {{< emoji "canny" >}}. It also explains why:
    struct, it tried to check every field of that struct, the check will pass if
    one of them is not zero-value. This is also why I put a non-empty string on
    purpose to make the test failing, proving that `Valid = false` means
-   nothing.
+   nothing.<br>
+   * Have a read about `required` tag here:
+     [docs](https://pkg.go.dev/github.com/go-playground/validator/v10#hdr-Required)
 
 2. With the knowledge above, it's easier to understand why [second
    test](#2nd-test) is failing, since now it tried to check **"the length of the

content/posts/raw-dogging-postgresql-with-pgx-and-sqlc-in-go/index.en.md

@@ -0,0 +1,273 @@
+---
+title: Raw-dogging PostgreSQL with pgx and sqlc in Go
+date: "2024-09-25T17:32:18+07:00"
+draft: false
+showComments: true
+description: "Raw-dogging PostgreSQL made easier and less error-prone using the
+combination of pgx and sqlc in Go"
+tags:
+- golang
+- database
+- postgres
+- sql
+---
+
+## What is pgx and sqlc? 
+
+- [pgx](https://github.com/jackc/pgx): a robust toolkit and PostgreSQL driver
+  for Golang. This module also provides some useful tool for handling complex
+      queries easier and less error-prone.
+- [sqlc](https://github.com/sqlc-dev/sqlc): a code generator tool that turns
+  your SQL queries in `.sql` files into Go code with type-safe for both query
+  params and query result. Check out an example here: [sqlc
+  playground](https://play.sqlc.dev/). sqlc also <mark>supports pgx out of the
+  box</mark>, which makes this a great combination for your database's need.
+
+## Why the combination of pgx and sqlc? 
+
+Although you may want to use some alternative solution like
+[GORM](https://github.com/go-gorm/gorm) (A Database ORM in Go), It seems like
+an easy choice to implement and use, plus you don't have to write SQL. Sounds
+too good to be true... but here's the catch:
+
+Based on my experience, almost all `ORM` I have ever used only perform well in
+easy scenarios (eg `CRUD operation`). The more complex your query gets, the
+harder to implement properly in these `ORM`, sometime it's even harder than
+writing raw query manually (try adding an upsert or
+[CTE](https://www.postgresql.org/docs/current/queries-with.html)),
+to the point you have to pull out the big gun... yes, you grab the database
+driver under the wrapper and start raw-dogging query, map the types manually
+and questioning yourself why you chose to use an ORM in the first place. {{<
+emoji "beat_shot"
+>}}
+
+Another foot gun of `ORM` is that you generally can't control the `SQL query`
+they produce, they may do dumb things and write terrible queries. Here's a funny 
+story about `Prisma ORM` in javascript world: [video](https://youtu.be/jqhHXe746Ns)
+
+A balance spot between `error-prone, untyped raw query` and `ORM` is `query
+builder`. They provide some sort of type safety and the flexibility to build
+and optimize complex query.
+
+### Usage of sqlc
+
+`sqlc` is not exactly a query builder but a code generator that reads your
+queries and schema in `.sql` files and turns it into type-safe code for both
+query params and query result, for example:
+
+It turns this query: (note that the comment is mandatory)
+```sql
+-- name: GetAuthor :one
+SELECT * FROM authors
+WHERE id = $1 LIMIT 1;
+```
+
+In to this type-safe Go code, ready to use:
+```go
+const createAuthor = `-- name: CreateAuthor :one
+INSERT INTO authors (
+  name, bio
+) VALUES (
+  $1, $2
+)
+RETURNING id, name, bio
+`
+
+type CreateAuthorParams struct {
+	Name string
+	Bio  sql.NullString
+}
+
+func (q *Queries) CreateAuthor(ctx context.Context, arg CreateAuthorParams) (Author, error) {
+	row := q.db.QueryRowContext(ctx, createAuthor, arg.Name, arg.Bio)
+	var i Author
+	err := row.Scan(&i.ID, &i.Name, &i.Bio)
+	return i, err
+}
+```
+
+So it should provide a similar experience to using a `query builder`: You can
+write things that are close to SQL queries and have the types mapped
+automatically
+
+### Usage of pgx
+
+There's even more complex query that neither `query builder` nor `sqlc` can
+handle. This is where you can use `pgx` to handle these specific complex
+query. 
+
+`pgx` also comes with plenty of useful tools which made it easier to
+write raw SQL: 
+
+#### Named argument & collect rows:
+```go
+func pgxInsert(db *database.Database, name string, bio pgtype.Text) (Author, error) {
+	// use named arguments instead $1, $2, $3...
+	query := `INSERT INTO author (name, bio) VALUES (@name, @bio) RETURNING *`
+	args := pgx.NamedArgs{
+		"name": name,
+		"bio":  bio,
+	}
+	rows, err := db.Pool.Query(context.Background(), query, args)
+	if err != nil {
+		return Author{}, nil
+	}
+	defer rows.Close()
+
+	// use collect helper function instead of scanning rows
+	return pgx.CollectOneRow(rows, pgx.RowToStructByName[Author])
+}
+```
+
+#### Bulk insert with [Postgres's COPY](https://www.postgresql.org/docs/current/sql-copy.html):
+```go
+func pgxCopyInsert(db *database.Database, authors []Author) (int64, error) {
+	rows := [][]any{}
+	columns := []string{"name", "bio"}
+	tableName := "author"
+
+	for _, author := range authors {
+		rows = append(rows, []any{author.Name, author.Bio})
+	}
+
+	return db.Pool.CopyFrom(
+		context.Background(),
+		pgx.Identifier{tableName},
+		columns,
+		pgx.CopyFromRows(rows),
+	)
+}
+```
+
+Implementation detail can be found in the tutorial [down
+here](#pgx-and-sqlc-tutorial)
+
+### Summary 
+
+In a typical Golang project I will use:
+- `sqlc` for `CRUD` operation and simple query.
+- `pgx` for some specific complex query that `sqlc` can't parse. 
+
+## Pgx and sqlc tutorial
+
+Source code of this tutorial can be found here: [Github
+repo](https://github.com/remvn/go-pgx-sqlc)
+
+### 1. Add pgx and install sqlc
+
+You will use `sqlc` as a cli tool for generating go codes, please install
+`sqlc` using this following guide from official docs: [install
+sqlc](https://docs.sqlc.dev/en/stable/overview/install.html)
+
+Add pgx package to your Go module
+```bash
+go get github.com/jackc/pgx/v5
+```
+
+### 2. Create a directory to store query and schema files.
+
+`./sqlc/schema.sql`
+```sql
+CREATE TABLE author (
+    id   SERIAL PRIMARY KEY,
+    name VARCHAR(100) NOT NULL UNIQUE,
+    bio  TEXT
+);
+```
+
+`./sqlc/query.sql`
+```sql
+-- name: GetAuthor :one
+SELECT *
+FROM author
+WHERE id = $1
+LIMIT 1;
+
+-- name: ListAuthors :many
+SELECT *
+FROM author
+ORDER BY name;
+
+-- name: CreateAuthor :one
+INSERT INTO author (name, bio)
+VALUES (lower(@name), @bio)
+RETURNING *;
+```
+
+### 3. Create sqlc config and generate code: 
+
+`sqlc.yaml` at project's root 
+```yaml
+version: "2"
+sql:
+  - engine: "postgresql"
+    queries: "sqlc/query.sql"
+    schema: "sqlc/schema.sql"
+    gen:
+      go:
+        package: "sqlc" # Package name
+        out: "database/sqlc" # Output folder
+        sql_package: "pgx/v5" # Use sql types provided by pgx
+        emit_json_tags: true
+        emit_db_tags: true # this helps pgx scan struct using types generated by sqlc
+```
+
+Run this command to generate code: 
+```bash
+sqlc generate
+```
+
+Check the files generated by sqlc: 
+![sqlc generate](sqlc-generate.jpg)
+
+### 4. Create a database package to wrap pgx and sqlc
+
+`./database/database.go`
+```go
+package database
+
+import (
+	"context"
+	"log"
+
+	"github.com/jackc/pgx/v5/pgxpool"
+	"github.com/remvn/go-pgx-sqlc/database/sqlc"
+)
+
+type Database struct {
+	Pool  *pgxpool.Pool
+	Query *sqlc.Queries
+}
+
+func NewDatabase(connStr string) *Database {
+	// this only create pgxpool struct, you may need to ping the database to
+	// grab a connection and check availability
+	pool, err := pgxpool.New(context.Background(), connStr)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	// this is generated by sqlc cli
+	query := sqlc.New(pool)
+
+	database := Database{
+		Pool:  pool,
+		Query: query,
+	}
+	return &database
+}
+```
+
+### 5. Pgx and sqlc in action  
+
+Please check out this code: 
+[database_test.go](https://github.com/remvn/go-pgx-sqlc/blob/main/database/database_test.go)
+
+If you has docker installed, clone the repo and run this command:
+```bash
+go test -v ./... 
+```
+
+With the implementation of [testcontainer for
+go](https://golang.testcontainers.org/), it will create a postgres container on
+the fly and run integration test on that database!

content/posts/some-great-golang-book-im-reading/index.en.md

@@ -0,0 +1,25 @@
+---
+title: Some great Golang book that I'm reading
+date: "2024-09-26T04:06:09+07:00"
+draft: false
+showComments: true
+description: "Some great Golang book that I'm reading"
+tags:
+- golang
+- random
+- book
+---
+
+[Let's Go by Alex Edwards](https://lets-go.alexedwards.net/)
+[<img class="mt-1" src="lets-go.jpg" width="300px"/>](https://lets-go.alexedwards.net/)
+
+[Let's Go Further by Alex Edwards](https://lets-go-further.alexedwards.net/)
+[<img class="mt-1" src="lets-go-further.jpg" width="300px"/>](https://lets-go-further.alexedwards.net/)
+
+[100 Go Mistakes and How to Avoid Them by teivah](https://100go.co/book/)
+[<img class="mt-1" src="100-mistakes.jpg" width="300px"/>](https://100go.co/book/)
+
+[Concurrency in Go by Cox-Buday](https://www.oreilly.com/library/view/concurrency-in-go/9781491941294/)
+[<img class="mt-1" src="concurrency.jpg" width="300px"/>](https://www.oreilly.com/library/view/concurrency-in-go/9781491941294/)
+
+