Skip to content

Commit

Permalink
Merge fork with enhancements/fixes (#456)
Browse files Browse the repository at this point in the history
* Add a check to ensure created_at and updated_at are Time

If a column is named `created_at` or `updated_at` is not a type `Time?`, then the following occurs:

```There was a problem expanding macro 'macro_4801706624'

Code in lib/granite/src/granite/transactions.cr:82:5

 82 | {% if @type.instance_vars.select { |ivar| ivar.annotation(Granite::Column) }.map(&.name.stringify).includes? "created_at" %}
      ^
Called macro defined in lib/granite/src/granite/transactions.cr:82:5

 82 | {% if @type.instance_vars.select { |ivar| ivar.annotation(Granite::Column) }.map(&.name.stringify).includes? "created_at" %}

Which expanded to:

 > 1 |
 > 2 |       if mode == :create
 > 3 |         @created_at = time.at_beginning_of_second
                                  ^---------------------
```

this simply adds a type check.

* fix allowing enum columns

* ameba fixes

* fix build_find_by_clause for bool values

* minor update

* Fix issue where primary key can be pushed twice on create

* fix infinate recusion when searching with enum

* add retry to adapter base

* add retry to adapters

* minor update

* add support for querying with Array(UUID)

* format docs

* FIX: add converter to read_attribute

* Fix for type declaration in has_one

* Fix callback runs on #import

* more has_one fixes; fixes for through associations

---------

Co-authored-by: Seth T <crimsonknightstudios@gmail.com>
  • Loading branch information
kalinon and crimson-knight authored Apr 24, 2023
1 parent 51e41d4 commit 5996066
Show file tree
Hide file tree
Showing 18 changed files with 271 additions and 167 deletions.
4 changes: 4 additions & 0 deletions docs/crud.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ end
post = Post.find! 1 # raises when no records found
```

### find_by

Finds the record(s) that match the given criteria
Expand All @@ -52,6 +53,7 @@ end
post = Post.find_by!(slug: "foo") # raises when no records found.
other_post = Post.find_by(slug: "foo", type: "bar") # Also works for multiple arguments.
```

### first

Returns the first record.
Expand Down Expand Up @@ -99,6 +101,7 @@ post.update(name: "Granite Really Rocks!") # Assigns attributes and calls save
post = Post.find 1
post.update!(name: "Granite Really Rocks!") # Assigns attributes and calls save!. Will throw an exception when the save failed
```

## Delete

Delete a specific record.
Expand All @@ -111,6 +114,7 @@ puts "deleted" if post.destroyed?
post = Post.find 1
post.destroy! # raises when delete failed
```

Clear all records of a model

```crystal
Expand Down
217 changes: 111 additions & 106 deletions docs/imports.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,124 +2,129 @@

## Import

> **Note:** Imports do not trigger callbacks automatically. See [Running Callbacks](#running-callbacks).
> **Note:** Imports do not trigger callbacks automatically. See [Running Callbacks](#running-callbacks).
Each model has an `.import` method that will save an array of models in one bulk insert statement.
```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
```

```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
```

## update_on_duplicate

The `import` method has an optional `update_on_duplicate` + `columns` params that allows you to specify the columns (as an array of strings) that should be updated if primary constraint is violated.
```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
Model.find!(1).name # => Fred
models = [
Model.new(id: 1, name: "George", age: 14),
]
Model.import(models, update_on_duplicate: true, columns: %w(name))
Model.find!(1).name # => George
```

**NOTE: If using PostgreSQL you must have version 9.5+ to have the on_duplicate_key_update feature.**
The `import` method has an optional `update_on_duplicate` + `columns` params that allows you to specify the columns (as an array of strings) that should be updated if primary constraint is violated.

```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
Model.find!(1).name # => Fred
models = [
Model.new(id: 1, name: "George", age: 14),
]
Model.import(models, update_on_duplicate: true, columns: %w(name))
Model.find!(1).name # => George
```

**NOTE: If using PostgreSQL you must have version 9.5+ to have the on_duplicate_key_update feature.**

## ignore_on_duplicate

The `import` method has an optional `ignore_on_duplicate` param, that takes a boolean, which will skip records if the primary constraint is violated.
```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
Model.find!(1).name # => Fred
models = [
Model.new(id: 1, name: "George", age: 14),
]
Model.import(models, ignore_on_duplicate: true)
Model.find!(1).name # => Fred
```

```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
]
Model.import(models)
Model.find!(1).name # => Fred
models = [
Model.new(id: 1, name: "George", age: 14),
]
Model.import(models, ignore_on_duplicate: true)
Model.find!(1).name # => Fred
```

## batch_size

The `import` method has an optional `batch_size` param, that takes an integer. The batch_size determines the number of models to import in each INSERT statement. This defaults to the size of the models array, i.e. only 1 INSERT statement.
```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
Model.new(id: 3, name: "Bill", age: 66),
]
Model.import(models, batch_size: 2)
# => First SQL INSERT statement imports Fred and Joe
# => Second SQL INSERT statement imports John and Bill
```
The `import` method has an optional `batch_size` param, that takes an integer. The batch_size determines the number of models to import in each INSERT statement. This defaults to the size of the models array, i.e. only 1 INSERT statement.

```Crystal
models = [
Model.new(id: 1, name: "Fred", age: 14),
Model.new(id: 2, name: "Joe", age: 25),
Model.new(id: 3, name: "John", age: 30),
Model.new(id: 3, name: "Bill", age: 66),
]
Model.import(models, batch_size: 2)
# => First SQL INSERT statement imports Fred and Joe
# => Second SQL INSERT statement imports John and Bill
```

## Running Callbacks

Since the `import` method runs on the class level, callbacks are not triggered automatically, they have to be triggered manually. For example, using the Item class with a UUID primary key:
```Crystal
require "uuid"
class Item < Granite::Base
connection mysql
table items
column item_id : String, primary: true, auto: false
column item_name : String
before_create :generate_uuid
def generate_uuid
@item_id = UUID.random.to_s
end
end
```

```Crystal
items = [
Item.new(item_name: "item1"),
Item.new(item_name: "item2"),
Item.new(item_name: "item3"),
Item.new(item_name: "item4"),
]
# If we did `Item.import(items)` now, it would fail since the item_id wouldn't get set before saving the record, violating the primary key constraint.
# Manually run the callback on each model to generate the item_id.
items.each(&.before_create)
# Each model in the array now has a item_id set, so can be imported.
Item.import(items)
# This can also be used for a single record.
item = Item.new(item_name: "item5")
item.before_create
item.save
```

> **Note:** Manually running your callbacks is mainly aimed at bulk imports. Running them before a normal `.save`, for example, would run your callbacks twice.
Since the `import` method runs on the class level, callbacks are not triggered automatically, they have to be triggered manually. For example, using the Item class with a UUID primary key:

```Crystal
require "uuid"
class Item < Granite::Base
connection mysql
table items
column item_id : String, primary: true, auto: false
column item_name : String
before_create :generate_uuid
def generate_uuid
@item_id = UUID.random.to_s
end
end
```

```Crystal
items = [
Item.new(item_name: "item1"),
Item.new(item_name: "item2"),
Item.new(item_name: "item3"),
Item.new(item_name: "item4"),
]
# If we did `Item.import(items)` now, it would fail since the item_id wouldn't get set before saving the record, violating the primary key constraint.
# Manually run the callback on each model to generate the item_id.
items.each(&.before_create)
# Each model in the array now has a item_id set, so can be imported.
Item.import(items)
# This can also be used for a single record.
item = Item.new(item_name: "item5")
item.before_create
item.save
```

> **Note:** Manually running your callbacks is mainly aimed at bulk imports. Running them before a normal `.save`, for example, would run your callbacks twice.
3 changes: 3 additions & 0 deletions docs/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ dependencies:
```
Update shards
```sh
$ shards update
```
Expand All @@ -36,6 +37,7 @@ Micrate::Cli.run
```

Make it executable:

```sh
$ chmod +x bin/micrate
```
Expand Down Expand Up @@ -71,6 +73,7 @@ DROP TABLE posts;
```

And now let's run the migration

```sh
$ bin/micrate up
```
Expand Down
27 changes: 14 additions & 13 deletions docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ class Bar < Granite::Base
end
```

In this example, we defined two connections. One to a MySQL database named "legacy_db", and another to a PG database named "new_db". The connection name given in the model maps to the name of a registered connection.
In this example, we defined two connections. One to a MySQL database named "legacy_db", and another to a PG database named "new_db". The connection name given in the model maps to the name of a registered connection.

> **NOTE:** How you store/supply each connection's URL is up to you; Granite only cares that it gets registered via `Granite::Connections << adapter_object`.
Expand Down Expand Up @@ -51,7 +51,7 @@ end

## Primary Keys

Each model is required to have a primary key defined. Use the `column` macro with the `primary: true` option to denote the primary key.
Each model is required to have a primary key defined. Use the `column` macro with the `primary: true` option to denote the primary key.

> **NOTE:** Composite primary keys are not yet supported.
Expand Down Expand Up @@ -90,7 +90,7 @@ end

### Natural Keys

Primary keys are defined as auto incrementing by default. For natural keys, you can set `auto: false` option.
Primary keys are defined as auto incrementing by default. For natural keys, you can set `auto: false` option.

```crystal
class Site < Granite::Base
Expand All @@ -103,7 +103,7 @@ end

### UUIDs

For databases that utilize UUIDs as the primary key, the type of the primary key can be set to `UUID`. This will generate a secure UUID when the model is saved.
For databases that utilize UUIDs as the primary key, the type of the primary key can be set to `UUID`. This will generate a secure UUID when the model is saved.

```crystal
class Book < Granite::Base
Expand All @@ -119,6 +119,7 @@ book.isbn # => nil
book.save
book.isbn # => RFC4122 V4 UUID string
```

## Default values

A default value can be defined that will be used if another value is not specified/supplied.
Expand All @@ -137,7 +138,7 @@ book.name # => "DefaultBook"

## Generating Documentation

By default, running `crystal docs` will **not** include Granite methods, constants, and properties. To include these, use the `granite_docs` flag when generating the documentation. E.x. `crystal docs -D granite_docs`.
By default, running `crystal docs` will **not** include Granite methods, constants, and properties. To include these, use the `granite_docs` flag when generating the documentation. E.x. `crystal docs -D granite_docs`.

Doc block comments can be applied above the `column` macro.

Expand All @@ -148,7 +149,7 @@ column published : Bool

## Annotations

Annotations can be a powerful method of adding property specific features with minimal amounts of code. Since Granite utilizes the `property` keyword for its columns, annotations are able to be applied easily. These can either be `JSON::Field`, `YAML::Field`, or third party annotations.
Annotations can be a powerful method of adding property specific features with minimal amounts of code. Since Granite utilizes the `property` keyword for its columns, annotations are able to be applied easily. These can either be `JSON::Field`, `YAML::Field`, or third party annotations.

```crystal
class Foo < Granite::Base
Expand All @@ -168,18 +169,18 @@ end

## Converters

Granite supports custom/special types via converters. Converters will convert the type into something the database can store when saving the model, and will convert the returned database value into that type on read.
Granite supports custom/special types via converters. Converters will convert the type into something the database can store when saving the model, and will convert the returned database value into that type on read.

Each converter has a `T` generic argument that tells the converter what type should be read from the `DB::ResultSet`. For example, if you wanted to use the `JSON` converter and your underlying database column is `BLOB`, you would use `Bytes`, if it was `TEXT`, you would use `String`.
Each converter has a `T` generic argument that tells the converter what type should be read from the `DB::ResultSet`. For example, if you wanted to use the `JSON` converter and your underlying database column is `BLOB`, you would use `Bytes`, if it was `TEXT`, you would use `String`.

Currently Granite supports various converters, each with their own supported database column types:

- `Enum(E, T)` - Converts an Enum of type `E` to/from a database column of type `T`. Supported types for `T` are: `Number`, `String`, and `Bytes`.
- `Json(M, T)` - Converters an `Object` of type `M` to/from a database column of type `T.` Supported types for `T` are: `String`, `JSON::Any`, and `Bytes`.
- **NOTE:** `M` must implement `#to_json` and `.from_json` methods.
- `Json(M, T)` - Converters an `Object` of type `M` to/from a database column of type `T.` Supported types for `T` are: `String`, `JSON::Any`, and `Bytes`.
- **NOTE:** `M` must implement `#to_json` and `.from_json` methods.
- `PgNumeric` - Converts a `PG::Numeric` value to a `Float64` on read.

The converter is defined on a per field basis. This example has an `OrderStatus` enum typed field. When saved, the enum value would be converted to a string to be stored in the DB. Then, when read, the string would be used to parse a new instance of `OrderStatus`.
The converter is defined on a per field basis. This example has an `OrderStatus` enum typed field. When saved, the enum value would be converted to a string to be stored in the DB. Then, when read, the string would be used to parse a new instance of `OrderStatus`.

```crystal
enum OrderStatus
Expand All @@ -193,10 +194,10 @@ class Order < Granite::Base
table foos
# Other fields
column status : OrderStatus, converter: Granite::Converters::Enum(OrderStatus, String)
column status : OrderStatus, converter: Granite::Converters::Enum(OrderStatus, String)
end
```

## Serialization

Granite implements [JSON::Serializable](https://crystal-lang.org/api/JSON/Serializable.html) and [YAML::Serializable](https://crystal-lang.org/api/YAML/Serializable.html) by default. As such, models can be serialized to/from JSON/YAML via the `#to_json`/`#to_yaml` and `.from_json`/`.from_yaml` methods.
Granite implements [JSON::Serializable](https://crystal-lang.org/api/JSON/Serializable.html) and [YAML::Serializable](https://crystal-lang.org/api/YAML/Serializable.html) by default. As such, models can be serialized to/from JSON/YAML via the `#to_json`/`#to_yaml` and `.from_json`/`.from_yaml` methods.
Loading

0 comments on commit 5996066

Please sign in to comment.