-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not return DATE fields as Javascript Date #50
Comments
I agree it can be clumsy to use JavaScript dates. This particular bike shed we've painted a few times in the past & kinda settled on the current behavior because it's easy enough to register your own type parser to do whatever you want with dates. I would be open to changing this behavior in the next major release for |
Would be nice if we could depend on this in Sequelize 4. We have not yet implemented the connection-level type parsers (registering global type parsers should be avoided). Of course the change should only be for DATE, not timestamp[tz]. |
Isn't this pg-types? 😄 |
I am super smart. |
Sorry was doing a bunch of node-postgres issues this morning and got confused. I'll delete the milestone & clean up the comments a bit - I'm only 1 of the maintainers here & my opinion is only 1 of several so we'll see what others weigh in on. |
okay tried to fix it up! Sorry for the confusion! Toooo many browser tabs! 😁 |
I also don't have strong feelings about this. I don't think returning a date as a |
The example might be a bit misleading. Consider the server is GMT+2. You query your database and then You could of course argue that the API should simply not have used |
That's my expectation here, despite the acknowledged need for extra boilerplate around serialization. Dates give you language level features like |
Thanks for the feedback. So far we only heard from people that a DATE (date-only) should be represented like a date-only and not as a timestamp with timezone, see the issue I linked. I actually never used this implicit object comparison, but always compared |
That's helpful but there's also a pretty significant confirmation bias in issues like that. You're far more likely to hear from the people who want strings and got tripped up by JS dates than the people who might be relying on a capital-D The common factor in other types that are not parsed is that they cannot be safely represented by a JS value type, e.g. int types with enough bits to be potentially greater than This change would make dates inconsistent with other types, including others with potentially treacherous locale behavior ( |
The problem is for a column that only contains date information and nothing about the time or a timezone (therefor representing a timezone-independent 24h range), an object is used for representation that includes date, time and timezone information (therefor representing a millisecond exact point in time in a specific meridian). Yes you can argue that you will hear more about people who complain, but I don't think that argument is 100% valid. In open source projects there are always also people who express their opinions against a proposed change, the sequelize issue is open for quite some time and so far you are the first one I met who considers this change undesirable. I think the use case of JSON.stringifying your database object is way greater than any native comparison capability. NodeJS is very often used to build RESTful JSON APIs, and it should be possible to recursively turn objects with data that somewhere comes from the database into a logical JSON representation without having to iterate and correct every single DATE(ONLY) property. In the common API you will have dozens of endpoints that need to send back a DATE(ONLY) in JSON, but I would argue having to compare two DATE(ONLY)s is very rare, and the additional code required to do it is way less than the additional code required to correct DATE(ONLY)s before JSON.stringifying them. Yes, custom type parsers can be used but to be honest I would feel weird making that decision inside the sequelize library and taking a high-abstraction object from the low-level database driver and converting it into a no-abstraction string inside the high-level ORM. Behavior should be consistent so the user is not confused if he comes from using plain pg. We should decide on one common way to represent DATE(ONLY) columns in Javascript, or at least make the low-level db driver delegate that decision to higher-level modules or the user by not using an abstraction. |
Ok, fair enough. I do want to throw out a third option here. |
Yes, I also thought about that. Basically introducing a new It only has to be decided wether this is something a low-level database driver should do, or if that should be left to the user / higher-level libraries. For example, it would be possible to always return I actually once published the package But honestly at least inside sequelize we always have to keep in mind that we have to support 3 other drivers too (mysql, tedious, sqlite3) and then normalize output for users. So in an ideal world the drivers just return low-level strings for stuff that doesn't map to native JS types (string, number, boolean, Date, Buffer, Array, Object) so we dont have to normalize (because those types are the only ones that all drivers can agree on easily). Just like DECIMALs are not returned as a BigNumber, even though everyone who actually uses DECIMALs probably is doing some monetary stuff and is using that library, because it is the only and best BigNumber implementation for JavaScript. So basically two options here, agree on a standard way to represent DateOnly values either with an interface or a class instance (that can still implement the object literal interface), or use a string and let users / libaries set custom type parsers. We also have to keep in mind that anything that is outputted also needs to be accepted as input. |
We are using pg-promises and ended up overriding pg-types's deserialization for Date types. |
We can make this change on a future major bump. Anyone care to do a PR for the right types/tests? |
@tomaspinho doing that in userland right now would break an ORM that relies on the dates being @bendrucker So what representation shall we use?
|
Let's go with |
Please see PR #53 |
I encountered the issue described by @felixfbecker and I share his opinion that a date-only field should not have any time information attached to it. The current behavior is very troublesome. |
What's the status of this? I see on #53 that it was decided to make sure this would be handled as a breaking change for node-postgres, but I don't see any mention on that repo. Are there further changes needed? |
I think it's planned for pg@7.0, https://github.com/brianc/node-postgres/milestone/3, but not totally sure about it. |
FWIW, we always have to set the type parser to return string. To me, returning ISO 8601 string seems most flexible. |
For future visitors. If you want to turn off node-postgres's date parsing....
You can add this code to override its behavior... import { types } from 'pg';
const TIMESTAMP_OID = 1114;
types.setTypeParser(TIMESTAMP_OID, (timestamp) => timestamp); |
To add to the above comment, the OID is different for const TYPE_TIMESTAMP = 1114
const TYPE_TIMESTAMPTZ = 1184
const noParse = v => v
types.setTypeParser(TYPE_TIMESTAMP, noParse)
types.setTypeParser(TYPE_TIMESTAMPTZ, noParse) |
OID for the DATE is if someone wants to return only date:
|
Given that this is a relatively frequent issue, might it be wise to add the workarounds provided here into the main documentation? |
This comment has been minimized.
This comment has been minimized.
Any update on this one. I have tried the workaround to disable the parser for Date OID 1082. However; I have the similar issue. I have a test case where I insert a Date say '2020-08-21', depending on which time of the day I run the test, I either get '2020-08-21' or '2020-08-22' (affected by timezone). I just need to get what I have in database '2020-08-21' with no manipulation. |
Please ignore my above comment. The issue was due to an oversight on my part. I am using a different workaround of using to_char(...) function to convert Date into String as an alternative to above. However; it would be great if Date are returned as strings and require no workaround solution |
Had a question relating to disabling the parsing, specifically for the DATE type (1082). What is the default parser / what would need to be applied to reenable the default (Javascript Date) behavior? Is it just using the parseTimestampUTC method defined in binaryParsers.js? Or is it as simple as (date) => new Date(date)? I couldn't find an explicit reference for 1082 except in the exports in builtins.js... |
A Javascript
Date
is always parsed in local timezone. A DATE column in Postgres or any other DB doesn't have any timezone associated with it, it is just a date. It begins and ends differently depending on where you are. Parsing this as a JSDate
that has timezone associated with it is a huge problem. Let's say your server is located in GMT+2 (Germany). You insert a date into your database, like2016-08-26
. When you query that in Germany, you will get theDate
objectFri Aug 26 2016 00:00:00 GMT+0200
. But runtoUTCString()
on it, and you getThu, 25 Aug 2016 22:00:00 GMT
! The date is one day off. This imposes a huge problem for distributed applications.TL;DR JavaScript objects are not suited to represent a DATE field, because they have a timezone and time associated, while a DATE has not.
postgres-date
should not be used to parse it and instead, the string should be returned (just like with a DECIMAL). Users can register their own type parser with their ownDateOnly
object if they want.This will be a breaking change and we would like to depend on this in Sequelize 4.
See sequelize/sequelize#4858
The text was updated successfully, but these errors were encountered: