[flash 411] use compatible new datetime/ date #224

hanfei1991 · 2019-09-08T06:00:04Z

No description provided.

hanfei1991 · 2019-09-08T06:46:39Z

/run-integration-tests

hanfei1991 · 2019-09-08T07:50:32Z

/run-integration-tests

hanfei1991 · 2019-09-08T08:46:25Z

/run-integration-tests

solotzg

Maybe you can try string_view and std::move.

solotzg · 2019-09-09T02:59:04Z

dbms/src/Common/MyTime.cpp

+    return idx;
+}
+
+std::vector<String> parseDateFormat(String format)


why not use const ref?

trimInPlace is defined as template<S> S& trimInPlace(S& str). If I pass a const ref, it will cause a compile error

dbms/src/Common/MyTime.cpp

dbms/src/DataTypes/DataTypeFactory.cpp

windtalker · 2019-09-09T06:42:00Z

dbms/src/Functions/FunctionsDateTime.h

+
+using FunctionToYYYYMM = FunctionDateOrDateTimeToSomething<DataTypeUInt32, ToYYYYMMImpl>;
+using FunctionToYYYYMMDD = FunctionDateOrDateTimeToSomething<DataTypeUInt32, ToYYYYMMDDImpl>;
+using FunctionToYYYYMMDDhhmmss = FunctionDateOrDateTimeToSomething<DataTypeUInt64, ToYYYYMMDDhhmmssImpl>;

 using FunctionAddSeconds = FunctionDateOrDateTimeAddInterval<AddSecondsImpl>;
 using FunctionAddMinutes = FunctionDateOrDateTimeAddInterval<AddMinutesImpl>;


looks like all the existing date/datetime udfs do not work for new datetime/date?

windtalker · 2019-09-09T06:49:05Z

dbms/src/IO/ReadHelpers.h

+{
+
+    const char * s = buf.position();
+    if (s + 19 <= buf.buffer().end())


where is the pessimistic path？

dbms/src/IO/ReadHelpers.h

windtalker · 2019-09-09T06:56:29Z

dbms/src/Storages/Transaction/Datum.cpp

-        UInt64 ymd = ((UInt64)values.year * 13 + values.month) << 5 | values.day_of_month;
-        UInt64 hms = (UInt64)hour << 12 | minute << 6 | second;
-        copy = (ymd << 17 | hms) << 24;
+        auto plain_text = orig.safeGet<String>();


Why get String from the orig filed? The orig field is the internal representation in TiFlash, and TiFlash does not use String to represent the date/datetime

It's only used by test. User insert datetime/date by string literal

windtalker · 2019-09-09T06:58:46Z

dbms/src/Storages/Transaction/TiDB.h

@@ -43,14 +43,14 @@ using DB::Timestamp;
    M(Float, 4, Float, Float32, false)               \
    M(Double, 5, Float, Float64, false)              \
    M(Null, 6, Nil, Nothing, false)                  \
-    M(Timestamp, 7, Int, DateTime, false)            \
+    M(Timestamp, 7, Int, MyDateTime, false)          \


Why timestamp is mapped to MyDateTime instead of MyTimestamp?

Has not implemented

windtalker · 2019-09-09T07:27:15Z

Does this pr support session level timezone? And in Mysql/TiDB only the values of the Timestamp data type is affected by time zone, does TiFlash follow this principle？

flowbehappy · 2019-09-10T04:14:55Z

dbms/src/Common/MyTime.h

+    String toString() const;
+};
+
+struct MyDate : public MyTimeBase


MyDate inherits from MyTimeBase may introduce too much overhead. Can we use a lightweight class?

TiDB use uint64 to store date type, we can optimize it in the future.

hanfei1991 · 2019-09-10T07:23:40Z

@windtalker I will support timestamp and timezone in next pr

…-datetime

hanfei1991 · 2019-09-11T07:12:10Z

/rebuild

hanfei1991 · 2019-09-11T08:32:09Z

/run-integration-tests tiflash=hanfei-new-datetime

hanfei1991 · 2019-09-11T18:29:05Z

/run-integration-tests tiflash=hanfei-new-datetime

hanfei1991 · 2019-09-12T05:58:51Z

@windtalker PTAL again

windtalker · 2019-09-12T06:22:31Z

dbms/src/Core/Types.h

+    MyDate,
+    MyDateTime,
+    MyTimeStamp,
+    MyTime


MyTime is mapping TypeDuration in MyTimeType enum? Better to use the same suffix, MyTime => MyDuration or TypeDuration => TypeTime

I'd like to keep consistent with MySQL Grammar

windtalker · 2019-09-12T07:09:01Z

dbms/src/Functions/FunctionsExternalDictionaries.h

-DECLARE_DICT_GET_TRAITS(UInt32, DataTypeDate)
-DECLARE_DICT_GET_TRAITS(Int64, DataTypeDateTime)
+DECLARE_DICT_GET_TRAITS(UInt16, DataTypeDate)
+DECLARE_DICT_GET_TRAITS(UInt32, DataTypeDateTime)


why new type is not declared here?

FunctionsExternalDictionaries.h has no relation with our own logic.

hanfei1991 · 2019-09-16T08:15:47Z

/run-integration-tests tiflash=hanfei-new-datetime

zanmato1984 · 2019-09-16T08:29:31Z

dbms/src/Storages/Transaction/Datum.cpp

@@ -17,96 +18,19 @@ using DB::Field;
 template <TP tp, typename = void>
 struct DatumOp
 {
-    static void unflatten(const Field &, std::optional<Field> &) {}


Don't remove unflattern/flattern methods, we are using them to deal with sign mismatch of Enum type between TiDB and TiFlash.

Just remove the unnecessary specialization version of Datum for Date/DateTime.

hanfei1991 · 2019-09-16T08:31:33Z

/run-integration-tests tiflash=hanfei-new-datetime

windtalker

LGTM

hanfei1991 · 2019-09-16T09:17:22Z

/run-integration-tests tiflash=hanfei-new-datetime

Signed-off-by: Wish <breezewish@outlook.com> Co-authored-by: JaySon <tshent@qq.com>

hanfei1991 added 2 commits September 3, 2019 19:17

turn back to old date types

f7713ff

support new date and datetime

91857bf

Merge branch 'master' into hanfei-datetime

db7cba9

solotzg reviewed Sep 9, 2019

View reviewed changes

windtalker reviewed Sep 9, 2019

View reviewed changes

flowbehappy reviewed Sep 10, 2019

View reviewed changes

hanfei1991 added 3 commits September 10, 2019 15:32

address comments

fba675a

Merge branch 'hanfei-datetime' of github.com:pingcap/tics into hanfei…

38f8677

…-datetime

Merge branch 'master' into hanfei-datetime

ff28222

add some tests

0b20720

hanfei1991 changed the title ~~[flash 441] use compatible new datetime/ date~~ [flash 411] use compatible new datetime/ date Sep 12, 2019

windtalker reviewed Sep 12, 2019

View reviewed changes

hanfei1991 added 2 commits September 12, 2019 17:25

Merge branch 'master' into hanfei-datetime

27f07f3

remove flatten and unflatten

381dd03

zanmato1984 reviewed Sep 16, 2019

View reviewed changes

hanfei1991 added 2 commits September 16, 2019 16:44

revert change

1362341

tiny change

1c74493

windtalker approved these changes Sep 16, 2019

View reviewed changes

hanfei1991 merged commit d8c631f into master Sep 16, 2019

hanfei1991 deleted the hanfei-datetime branch September 16, 2019 11:02

JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Aug 2, 2024

storage: Enable simsimd for usearch (pingcap#224)

628f40f

Signed-off-by: Wish <breezewish@outlook.com> Co-authored-by: JaySon <tshent@qq.com>

JaySon-Huang added a commit that referenced this pull request Aug 6, 2024

storage: Enable simsimd for usearch (#224)

ec8a3d0

Signed-off-by: Wish <breezewish@outlook.com> Co-authored-by: JaySon <tshent@qq.com>

Lloyd-Pottiger pushed a commit to Lloyd-Pottiger/tiflash that referenced this pull request Aug 26, 2024

storage: Enable simsimd for usearch (pingcap#224)

5e96b35

Signed-off-by: Wish <breezewish@outlook.com> Co-authored-by: JaySon <tshent@qq.com>

CalvinNeo mentioned this pull request Nov 7, 2024

update proxy of master to raftstore-proxy #9590

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[flash 411] use compatible new datetime/ date #224

[flash 411] use compatible new datetime/ date #224

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

solotzg left a comment

solotzg Sep 9, 2019

hanfei1991 Sep 10, 2019

windtalker Sep 9, 2019

hanfei1991 Sep 9, 2019

windtalker Sep 9, 2019

windtalker Sep 9, 2019

hanfei1991 Sep 10, 2019

windtalker Sep 9, 2019

hanfei1991 Sep 9, 2019

windtalker commented Sep 9, 2019

flowbehappy Sep 10, 2019

hanfei1991 Sep 10, 2019

hanfei1991 commented Sep 10, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 12, 2019

windtalker Sep 12, 2019

hanfei1991 Sep 12, 2019

windtalker Sep 12, 2019

hanfei1991 Sep 12, 2019

hanfei1991 commented Sep 16, 2019

zanmato1984 Sep 16, 2019

hanfei1991 commented Sep 16, 2019

windtalker left a comment

hanfei1991 commented Sep 16, 2019

[flash 411] use compatible new datetime/ date #224

[flash 411] use compatible new datetime/ date #224

Conversation

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

hanfei1991 commented Sep 8, 2019

solotzg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

windtalker commented Sep 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanfei1991 commented Sep 10, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 11, 2019

hanfei1991 commented Sep 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanfei1991 commented Sep 16, 2019

Choose a reason for hiding this comment

hanfei1991 commented Sep 16, 2019

windtalker left a comment

Choose a reason for hiding this comment

hanfei1991 commented Sep 16, 2019