Support nested catListTable (by representing nested arrays as text)
#242
Conversation
shane-circuithub
changed the title
catListTable (by represented nested arrays as text)catListTable (by representing nested arrays as text)
This is another possible "fix" to #168 (as opposed to #242). It doesn't really fix the problem, but it allows us to use two levels of `catListTable` instead of only one. Instead of trying to use Postgres's broken `.f1` syntax, we cast the anonymous record to text, remove the parentheses and quotes and unescape any escaped quotes or backslashes, and then cast the resulting text back to the appropriate type. The reason this only works one level deep is that if the type we cast the text back to is itself an anonymous record, then PostgreSQL doesn't know how to parse the text. It's kind of ugly and hacky but it does work and otherwise maintains the status quo. Comparison operators on nested lists continue to work as before and we don't need to burden `DBType` with parsing nonsense.
|
Is this basically a variant on what we used to do when encoding to JSON? |
|
I don't recall that we ever did actually use JSON for this but we did talk about it. As far as I know PostgreSQL doesn't have a way to reliably convert arbitrary things from JSON so I don't think JSON could really be used for this. There's also no guarantee that the |
shane-circuithub
force-pushed
the
profunctor-fold-aggregator
branch
from
144366d to
e28cc31
Compare
This is another possible "fix" to #168 (as opposed to #242). It doesn't really fix the problem, but it allows us to use two levels of `catListTable` instead of only one. Instead of trying to use Postgres's broken `.f1` syntax, we cast the anonymous record to text, remove the parentheses and quotes and unescape any escaped quotes or backslashes, and then cast the resulting text back to the appropriate type. The reason this only works one level deep is that if the type we cast the text back to is itself an anonymous record, then PostgreSQL doesn't know how to parse the text. It's kind of ugly and hacky but it does work and otherwise maintains the status quo. Comparison operators on nested lists continue to work as before and we don't need to burden `DBType` with parsing nonsense.
This is another possible "fix" to #168 (as opposed to #242). It doesn't really fix the problem, but it allows us to use two levels of `catListTable` instead of only one. Instead of trying to use Postgres's broken `.f1` syntax, we cast the anonymous record to text, remove the parentheses and quotes and unescape any escaped quotes or backslashes, and then cast the resulting text back to the appropriate type. The reason this only works one level deep is that if the type we cast the text back to is itself an anonymous record, then PostgreSQL doesn't know how to parse the text. It's kind of ugly and hacky but it does work and otherwise maintains the status quo. Comparison operators on nested lists continue to work as before and we don't need to burden `DBType` with parsing nonsense.
This is another possible "fix" to #168 (as opposed to #242). It doesn't really fix the problem, but it allows us to use two levels of `catListTable` instead of only one. Instead of trying to use Postgres's broken `.f1` syntax, we cast the anonymous record to text, remove the parentheses and quotes and unescape any escaped quotes or backslashes, and then cast the resulting text back to the appropriate type. The reason this only works one level deep is that if the type we cast the text back to is itself an anonymous record, then PostgreSQL doesn't know how to parse the text. It's kind of ugly and hacky but it does work and otherwise maintains the status quo. Comparison operators on nested lists continue to work as before and we don't need to burden `DBType` with parsing nonsense.
shane-circuithub
force-pushed
the
nested-array-text
branch
from
3e2b1be to
69e89e0
Compare
shane-circuithub
force-pushed
the
nested-array-text
branch
2 times, most recently
from
971d4d6 to
fcbab82
Compare
Yea, I had it working but couldn't find anything in the old rel8 branches. Oh well. Would have been around Mar 2021.
I think what I was doing previously was (for non-JSON things): casting to
|
shane-circuithub
force-pushed
the
nested-array-text
branch
from
fcbab82 to
adf0757
Compare
shane-circuithub
force-pushed
the
nested-array-text
branch
from
adf0757 to
3233807
Compare
This is one possible "fix" to #168. With this we can `catListTable` arbitrarily deep trees of `ListTable`s. It comes at a relatively high cost, however. Currently we represent nested arrays with anonymous records. This works reasonably well, except that we can't extract the field from the anonymous record when we need it (PostgreSQL [theoretically](https://www.postgresql.org/docs/13/release-13.html#id-1.11.6.16.5.6) suports `.f1` syntax since PG13 but it only works in very limited situations). But it does mean we can decode the results using Hasql's binary decoders, and ordering works how we expect ('array[row(array[9])] < array[row(array[10])]'. What this PR does is instead represent nested arrays as text. To be able to decode this, we need each 'DBType' to supply a text parser in addition to a binary decoder. It also means that ordering is no longer intuitive, because `array[array[9]::text] > array[array[10]::text]`. However, it does mean we can nest `catListTable`s to our heart's content and it will always just work.
shane-circuithub
force-pushed
the
nested-array-text
branch
from
3233807 to
9b0e5e2
Compare
This is a possible fix to #168. With this we can
catListTablearbitrarily deep trees ofListTables.It comes at a relatively high cost, however.
Currently we represent nested arrays with anonymous records. This works reasonably well, except that we can't extract the field from the anonymous record when we need it (PostgreSQL theoretically suports
.f1syntax since PG13 but it only works in very limited situations). But it does mean we can decode the results using Hasql's binary decoders, and ordering works how we expect to (array[row(array[9])] < array[row(array[10])]).What this PR does is instead represent nested arrays as text. To be able to decode this, we need each
DBTypeto supply a text parser in addition to a binary decoder. It also means that ordering is no longer intuitive, becausearray[array[9]::text] > array[array[10]::text]. However, it does mean we can nestcatListTables to our heart's content and it will always just work.