WIP: bpo-1100942: Add datetime.time.strptime and datetime.date.strptime #5578

matrixise · 2018-02-07T10:28:07Z

Add datetime.date.strptime and datetime.time.strptime.

Fix the documentation of _strptime._strptime, the documentation was
wrong, return a 3-tuple and not a 2-tuple

Co-authored-by: Alexander Belopolsky alexander.belopolsky@gmail.com
Co-authored-by: Amaury Forgeot d'Arc amauryfa@gmail.com
Co-authored-by: Berker Peksag berker.peksag@gmail.com
Co-authored-by: Josh-sf josh-sf@users.sourceforge.net
Co-authored-by: Juarez Bochi jbochi@gmail.com
Co-authored-by: Maciej Szulik soltysh@gmail.com
Co-authored-by: Stéphane Wirtel stephane@wirtel.be
Co-authored-by: Matheus Vieira Portela matheus.v.portela@gmail.com

https://bugs.python.org/issue1100942

matrixise · 2018-02-07T10:29:03Z

This PR is for a very long issue, since 2005. We have a PR in 2018 👍

Mariatta · 2018-02-09T17:02:02Z

I restarted the travis job. It still did not do the full CPython test suite. So please rebase :)

matrixise · 2018-02-15T08:19:58Z

Thanks, I didn't see your message, works on this issue today.

matrixise · 2018-02-15T08:51:40Z

@Mariatta rebased and the tests pass on the CIs

pganssle · 2018-05-15T13:21:09Z

Lib/_strptime.py

    the number of microseconds based on the input string and the
-    format string."""
+    format string, and the GMT offset."""


We should probably consistently use "UTC offset", though I suppose it doesn't matter much since it's not a public-facing docstring.

@pganssle I am not really confident with the offsets and the datetime. Do you think we could keep it like that and propose an other bpo ?

I just meant use UTC instead of GMT everywhere.

Yeah, just replace "GMT" with "UTC". datetime.tzinfo has a utcoffset() method, so "UTC" is preferred in datetime.

(And there are some subtle differences between GMT and UTC that I forgot.)

pganssle · 2018-05-15T13:26:49Z

Lib/_strptime.py

@@ -565,6 +566,10 @@ def _strptime(data_string, format="%a %b %d %H:%M:%S %Y"):
            hour, minute, second,
            weekday, julian, tz, tzname, gmtoff), fraction, gmtoff_fraction

+date_specs = ('%a', '%A', '%b', '%B', '%c', '%d', '%j', '%m', '%U',


I think this is missing %G, %u and %V, the ISO 8601 week calendar directives.

pganssle · 2018-05-15T13:29:09Z

Lib/_strptime.py

@@ -565,6 +566,10 @@ def _strptime(data_string, format="%a %b %d %H:%M:%S %Y"):
            hour, minute, second,
            weekday, julian, tz, tzname, gmtoff), fraction, gmtoff_fraction

+date_specs = ('%a', '%A', '%b', '%B', '%c', '%d', '%j', '%m', '%U',
+              '%w', '%W', '%x', '%y', '%Y',)
+time_specs = ('%T', '%R', '%H', '%I', '%M', '%S', '%f', '%i', '%s',)


I don't see any references to %T, %R, %i or %s in the docs or elsewhere in the code. What do these represent?

pganssle · 2018-05-15T13:47:35Z

Lib/_strptime.py

+        _time = _strptime_datetime(datetime_datetime, data_string, format)
+        return _time.time()
+
+def _check_invalid_datetime_specs(fmt, specs, msg):


If I'm understanding this correctly, this function seems to have the inverted sense of what I would expect. It seems that it checks if fmt is a valid spec.

From the names of the variables and functions, I was thinking that this would be a whitelist not a blacklist. Does it make sense to switch to a whitelist approach? If not, can we maybe change specs to be blacklist_specs or something?

pganssle · 2018-05-15T13:50:31Z

Lib/_strptime.py

@@ -565,6 +566,10 @@ def _strptime(data_string, format="%a %b %d %H:%M:%S %Y"):
            hour, minute, second,
            weekday, julian, tz, tzname, gmtoff), fraction, gmtoff_fraction

+date_specs = ('%a', '%A', '%b', '%B', '%c', '%d', '%j', '%m', '%U',
+              '%w', '%W', '%x', '%y', '%Y',)
+time_specs = ('%T', '%R', '%H', '%I', '%M', '%S', '%f', '%i', '%s',)


I don't have strong opinions about this, but my intuition is that these should be set or frozenset rather than tuple. Are these tuples for performance reasons (I am not sure I know when exactly it's faster to use a set for "lookup membership" rather than a tuple or list).

pganssle · 2018-05-15T13:52:24Z

Lib/_strptime.py

+def _strptime_datetime_date(data_string, format):
+    """Return a date based on the input string and the format string."""
+    if not format:
+        raise ValueError("Date format is not valid.")


This line and its equivalent in _time are not being hit. If I understand correctly this branch is only hit if format is empty?

useless, thanks

pganssle · 2018-05-15T13:54:44Z

Lib/test/datetimetester.py

+        tests = [('2004-12-01 13:02:47.197',
+                  '%Y-%m-%d %H:%M:%S.%f'),
+                 ('2004-12-01', '%Y-%m-%d'),]
+        for date_string, date_format in tests:


I think it would be good to use with self.subTest for these parametrized tests.

Also, per the other comment I guess you need to add something like ('12:30:15', '')` to get full coverage.

I think you need two more test cases:

('1900-01-01 12:30', '%Y-%m-%d %H:%M'), ('12:30:15', ''),

date.strptime has similarly missing tests.

the test with ('1900-01-01 12:30', '%Y-%m-%d %H:%M') does not raise an exception but returns datetime.time(12, 30)

For the other test, yep, there is an issue.

The fact that it doesn't raise an exception is an issue. I'm a bit surprised that it doesn't raise an exception on pure Python, that's a bug, because I'm pretty sure that:

datetime.time.strptime("1901-01-01 12:30", "%Y-%m-%d %H:%M") does raise an exception.

pganssle · 2018-05-15T13:57:31Z

Modules/_datetimemodule.c

+     */
+    if (emptyDatetime == NULL) {
+        PyObject *emptyStringPair = Py_BuildValue("ss", "", "");
+        if (emptyStringPair == NULL)


If I'm interpreting PEP 7 correctly, these if statements need curly braces. The relevant section is Code layout.

matrixise · 2018-05-15T17:33:57Z

Hi @pganssle

thank you for your review, I am going to fix it asap but I am not the author of the code, just the author of the PR. so, maybe I would need your help. Thanks

matrixise · 2018-10-05T16:03:45Z

@pganssle I just rebased my branch with master. I am going to work on this PR. Do you want to help me because you are mister dateutil ;-)

pganssle · 2018-10-05T19:27:42Z

@matrixise Sorry this is on my list but probably can't get to it until the end of the month. 😟

matrixise · 2018-10-05T23:56:54Z

@pganssle ok, in this case, I will try to fix all the issues alone ;-) but I am not worried ;-)

matrixise · 2018-10-17T19:56:16Z

Hi, I just updated this PR with master.

matrixise · 2018-10-25T15:03:55Z

@pganssle when you have time, could you review this PR, we started together, just comment when you find a mistake, thanks

matrixise · 2018-11-04T06:31:20Z

ping @pganssle ;-)

pganssle

I haven't had a chance to look at the PR thoroughly, but I have a first-pass review of things that should be changed.

I also haven't checked exactly how you are doing it, but I believe we may want to / be able to refactor the tests a bit to take advantage of the existing test suite for datetime.strptime by separating out the date-only and time-only formats and reusing the tests with date, time and datetime.

Additionally, I should note that it's unfortunate that if this is merged, we'll have time.strptime and datetime.time.strptime, the first of which returns a timetuple (which is actually more like a datetime), and the second returning a datetime.time object. I don't see any way around this, but it will add confusion. :(

Doc/library/datetime.rst

pganssle · 2018-11-04T18:25:53Z

Doc/library/datetime.rst

+   Return a :class:`date` corresponding to *date_string*, parsed according to
+   *format*. :exc:`ValueError` is raised if the date string and format can't be
+   parsed by `time.strptime`, or if it returns a value where the time part is
+   nonzero.


Here and below, you have nonzero instead of non-zero. If we keep this wording, the hyphen needs to be added.

However, this PR does not parse to a timetuple or something and check if certain components were zero, it checks to see if the format string contains time components, and fails in that case (Edit: I was looking at just the pure python implementation - I now realize that this is precisely what the C implementation is doing, but I think it's the wrong thing to do anyway). The way the docs are currently worded, you would expect this to work:

from datetime import date date.strptime("2018-01-01 00:00", "%Y-%m-%d %H:%M")

But it will fail (rightly so, I think).

I believe you can change the last part of the last sentence as such:

- parsed by `time.strptime`, or if it returns a value where the time part is - nonzero. + parsed by `time.strptime`, or if time components are present in the format string.

Also, I think it needs to be

:meth:`time.strptime`

One other note on the documentation, the datetime.strptime documentation links to

:ref:`strftime-strptime-behavior`.

I think these should too.

pganssle · 2018-11-04T21:00:58Z

Modules/_datetimemodule.c

 {
    return new_date(GET_YEAR(self),
                    GET_MONTH(self),
                    GET_DAY(self));
 }

 static PyObject *
-datetime_gettime(PyDateTime_DateTime *self, PyObject *Py_UNUSED(ignored))


I don't know why this PyObject *Py_UNUSED(ignored) is here, so per Chesterton's Fence, I'm not comfortable removing it. Any insight as to why it is here and what the consequences will be in removing it?

Possibly it will be a breaking change in the C ABI?

According to git blame, this is to silence a warning in gcc8. Is this still relevant @siddhesh @serhiy-storchaka?

Edit: Looking closer, the merged PR is from April, so I'm guessing it is, but now I'm wondering if this will break the C ABI the other way.

Yes, datetime_gettime should have two arguments, the second is ignored.

pganssle · 2018-11-04T21:05:11Z

Modules/_datetimemodule.c

+        return NULL;
+    }
+
+    if (DATE_GET_HOUR(datetime) ||


Hm, if I understand correctly, this is inconsistent with the pure Python version. The pure Python version checks if there are any time components in the format string, whereas the C version parses to a datetime and then checks to see if there are any time components.

I think checking the format string is the better way to do this, as I mention in another comment, this approach seems to indicate that date.strptime("2018-01-01 00:00", "%Y-%m-%d %H:%M") would succeed, which is not the right thing to do. I will comment on the test suite to add a test for this.

ok, I will check the Python implementation, but in this case, we could migrate the Python implementation to the C layer?

Yeah, port the Python implementation to the C layer.

lol ;-) I am not an expert with the C-API, but I could try, it's a good exercise for my comprehension of the C-API of Python

Yeah, you have to reimplement _check_invalid_datetime_specs() in C.

Why does it have to be implemented in C? There are already calls out to the _strptime Python-language module here.

not sure, but because there is an other test in C for the date, and maybe we could put the verification process in only one function. @vstinner do you confirm?

pganssle · 2018-11-04T21:06:41Z

Lib/test/datetimetester.py

+        tests = [('2004-12-01 13:02:47.197',
+                  '%Y-%m-%d %H:%M:%S.%f'),
+                 ('2004-12-01', '%Y-%m-%d'),]
+        for date_string, date_format in tests:


I think you need two more test cases:

('1900-01-01 12:30', '%Y-%m-%d %H:%M'), ('12:30:15', ''),

date.strptime has similarly missing tests.

pganssle · 2018-11-04T21:09:18Z

Lib/test/datetimetester.py

+        tests = [
+            ('2004-12-01 13:02:47.197', '%Y-%m-%d %H:%M:%S.%f'),
+            ('01', '%M'),
+            ('02', '%H'),


This needs at least two more test cases:

('2018-01-01 00:00', '%Y-%m-%d %H:%M'), ('2018-01-01', ''),

Both should fail.

matrixise · 2018-11-04T21:15:42Z

@pganssle thanks for your review, I am going to update this PR asap.

matrixise · 2018-11-11T14:15:19Z

ok, rebased with the last master.

matrixise · 2018-11-11T17:35:06Z

ok, I will check and try to fix that, but because I don't know this part and the main patch was not mine, that will be difficult. but okay for the fix. thanks for the review.

matrixise · 2019-02-20T13:59:32Z

@pganssle Are you ready for a new review of this PR? I will continue my PR ;-)

vadmium

I suggest you check the discussion about the previous patches. I noticed I repeated some old comments and ideas.

vadmium · 2019-03-23T07:20:31Z

Doc/library/datetime.rst

+   Return a :class:`date` corresponding to *date_string*, parsed according to
+   *format*. :exc:`ValueError` is raised if the date string and format can't be
+   parsed by :meth:`time.strptime`, or if time components are present in the
+   format string. For a complete list of formatting directives, see


Perhaps clarify “time.strptime” does not refer to the strptime function in the time module. The reader may not see the meth RST code, HTML links, etc.

vadmium · 2019-03-23T07:21:47Z

Doc/library/datetime.rst

+   Return a :class:`time` corresponding to *date_string, parsed according to
+   *format*. :exc:`ValueError` is raised if the date string and format can't be
+   parsed by :meth:`time.strptime`, if it returns a value which isn't a time
+   tuple, or if the date part is nonzero. For a complete list of formatting


“By time.strptime” is redundant, unless you mean to refer to the strptime function in the time module.

What does “the date part is nonzero” mean? I would expect the time class to work without specifying a date, zero or otherwise.

vadmium · 2019-03-23T07:23:54Z

Doc/library/datetime.rst

-is substituted for the year, and ``1`` for the month and day.
+The :meth:`date.strptime` class method creates a :class:`date` object from a
+string representing a date and a corresponding format string. :exc:`ValueError`
+raised if the format codes for hours, minutes, seconds, and microseconds are used.


ValueError is raised . . .

And microseconds should be or microseconds, unless you must combine all four codes to get the error.

But wouldn’t these details be better placed directly under the date and time classes, not in this common section about format codes in general?

I have fixed the grammar issues, but for the details I don't know.

vadmium · 2019-03-23T07:31:30Z

Doc/library/datetime.rst

@@ -2023,13 +2046,13 @@ equivalent to ``datetime(*(time.strptime(date_string, format)[0:6]))``, except
 when the format includes sub-second components or timezone offset information,
 which are supported in ``datetime.strptime`` but are discarded by ``time.strptime``.


Need to clarify time.strptime refers to the time module, not your new strptime method.

vadmium · 2019-03-23T07:33:52Z

Doc/library/datetime.rst

+
+.. classmethod:: time.strptime(date_string, format)
+
+   Return a :class:`time` corresponding to *date_string, parsed according to


Why `time` here, but `.time` below (with a dot)?

👍 and there was an other erorr with *date_string*

vadmium · 2019-03-23T07:44:53Z

Doc/whatsnew/3.8.rst

+datetime
+--------
+
+Added :func:`~datetime.date.strptime` and :func:`~datetime.time.strptime`.


I expect you lose the reference to date and time, so all you are saying is you added two functions with the same name repeated.

@vadmium yep, here, we just add the class methods datetime.date.strptime and datetime.time.strptime

vadmium · 2019-03-23T07:59:07Z

Modules/_datetimemodule.c

+        return NULL;
+    }
+
+    assert(PyTuple_CheckExact(specs) || PyList_CheckExact(specs));


Why would it be a list? This seems like unused code.

the first version was only for the tuples. After, I wanted to accept the list for my tests. but I am not against to only accept the tuples.

vadmium · 2019-03-23T08:07:49Z

Modules/_datetimemodule.c

+        return NULL;
+    }
+
+    if (DATE_GET_HOUR(datetime) ||


Why does it have to be implemented in C? There are already calls out to the _strptime Python-language module here.

vadmium · 2019-03-23T08:10:00Z

Lib/_strptime.py

+    """Return a date based on the input string and the format string."""
+    msg = "'{!s}' {} not valid in date format specification."
+    from _datetime import _check_invalid_datetime_specs
+    if _check_invalid_datetime_specs(format, time_specs, msg):


Why not move the msg string into the check function, and just pass the bit that varies ('date' or 'time') as an argument? It would make the code easier to read.

Looks like the check function either raises an exception or returns True. It would be clearer to not use an if statement here.

at the beginning, this PR was a submitted patch by other contributors, I just wanted to convert it to a PR. and now, I try to fix all the issues.

but for this point, the first step was the conversion of _check_invalid_datetime_specs to C, and after start to improve the code with the other recommendations. (from @vstinner and @pganssle)

list

csabella · 2019-11-16T02:22:11Z

@matrixise, @pganssle There's been a lot of work done on this one. Is it something we should try to move forward on? Thanks!

github-actions · 2022-08-15T00:12:12Z

This PR is stale because it has been open for 30 days with no activity.

zitterbewegung · 2024-05-23T00:44:50Z

@matrixise, @pganssle, @csabella while there has been lots of work for this where are we at on this? Its around a bit over two years old for the PR but, these new features would be great!

vstinner · 2024-05-23T11:52:48Z

where are we at on this?

No activity since 2019. Someone has to step in and restart the work (ex: create a new PR and update the PR).

nineteendo · 2024-06-18T20:02:45Z

I'm going to try to resolve the merge conflicts and make the requested changes.

nineteendo · 2024-06-19T06:26:19Z

Looks like I'll have to start over.

vstinner · 2024-06-19T07:53:26Z

I close the inactive PR.

the-knights-who-say-ni added the CLA signed label Feb 7, 2018

bedevere-bot added the awaiting review label Feb 7, 2018

matrixise mentioned this pull request Feb 7, 2018

Travis has an issue with the C build? python/core-workflow#218

Closed

matrixise force-pushed the bpo-1100942 branch from ad32c42 to a7b624a Compare February 15, 2018 08:23

pganssle reviewed May 15, 2018

View reviewed changes

pganssle requested changes May 15, 2018

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels May 15, 2018

matrixise force-pushed the bpo-1100942 branch from fb6d098 to 1d7a5b0 Compare October 5, 2018 16:01

matrixise force-pushed the bpo-1100942 branch from 1d7a5b0 to c286708 Compare October 17, 2018 19:56

matrixise force-pushed the bpo-1100942 branch from a6722e7 to 7526de9 Compare October 25, 2018 14:02

pganssle requested changes Nov 4, 2018

View reviewed changes

matrixise force-pushed the bpo-1100942 branch from 1ca284e to 43a77af Compare November 11, 2018 14:15

matrixise added 10 commits March 22, 2019 18:16

Fix the signatures

010ac8b

Update with the recommendation of @pganssle

5b58fc1

Update

8353a65

Add use case where we generate an exception

cff74a9

Add the Py_UNUSED(ignore) for datetime_gettime & datetime_getdate

f92a799

Add a link to the strtime() and strptime() behaviour

4c4a7cc

Update the blurb entry and add an entry in the Doc/whatsnew/3.8.rst

b006172

Fix doc

ff7487f

Convert the _check_invalid_datetime_spec to the C-API

a048f0e

Support tuple and list for the specs

9ac0af4

matrixise force-pushed the bpo-1100942 branch from 70c125f to 9ac0af4 Compare March 22, 2019 17:17

vadmium reviewed Mar 23, 2019

View reviewed changes

matrixise changed the title ~~bpo-1100942: Add datetime.time.strptime and datetime.date.strptime~~ WIP: bpo-1100942: Add datetime.time.strptime and datetime.date.strptime Mar 24, 2019

matrixise added 2 commits March 24, 2019 14:20

_datetimemodule._check_invalid_datetime_specs only accepts tuple and not

a952bb7

list

Fix the code with the recommendation of Martin Panter

412aea2

josh-sf mannequin mentioned this pull request Apr 9, 2022

Add datetime.time.strptime and datetime.date.strptime #41431

Closed

ezio-melotti removed the CLA signed label Jul 13, 2022

github-actions bot added the stale Stale PR or inactive for long period of time. label Aug 15, 2022

abalkin marked this pull request as draft February 8, 2023 14:59

abalkin self-assigned this Feb 8, 2023

abalkin linked an issue Feb 8, 2023 that may be closed by this pull request

Add datetime.time.strptime and datetime.date.strptime #41431

Closed

github-actions bot removed the stale Stale PR or inactive for long period of time. label May 1, 2023

vstinner closed this Jun 19, 2024

		@@ -2023,13 +2046,13 @@ equivalent to ``datetime(*(time.strptime(date_string, format)[0:6]))``, except
		when the format includes sub-second components or timezone offset information,
		which are supported in ``datetime.strptime`` but are discarded by ``time.strptime``.


		.. classmethod:: time.strptime(date_string, format)

		Return a :class:`time` corresponding to *date_string, parsed according to

WIP: bpo-1100942: Add datetime.time.strptime and datetime.date.strptime #5578

WIP: bpo-1100942: Add datetime.time.strptime and datetime.date.strptime #5578

Conversation

matrixise commented Feb 7, 2018 • edited by bedevere-bot Loading

matrixise commented Feb 7, 2018

Mariatta commented Feb 9, 2018

matrixise commented Feb 15, 2018

matrixise commented Feb 15, 2018

pganssle May 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matrixise commented May 15, 2018

matrixise commented Oct 5, 2018

pganssle commented Oct 5, 2018

matrixise commented Oct 5, 2018

matrixise commented Oct 17, 2018

matrixise commented Oct 25, 2018

matrixise commented Nov 4, 2018

pganssle left a comment

Choose a reason for hiding this comment

pganssle Nov 4, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pganssle Nov 4, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matrixise commented Nov 4, 2018

matrixise commented Nov 11, 2018

matrixise commented Nov 11, 2018 via email

matrixise commented Feb 20, 2019

vadmium left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csabella commented Nov 16, 2019

github-actions bot commented Aug 15, 2022

zitterbewegung commented May 23, 2024

vstinner commented May 23, 2024

nineteendo commented Jun 18, 2024

nineteendo commented Jun 19, 2024

vstinner commented Jun 19, 2024

matrixise commented Feb 7, 2018 •

edited by bedevere-bot

Loading

pganssle May 15, 2018 •

edited

Loading

pganssle Nov 4, 2018 •

edited

Loading

pganssle Nov 4, 2018 •

edited

Loading