Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR offsets: rename 'Q' to 'QE' #55553

Merged
merged 19 commits into from
Oct 24, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions doc/source/user_guide/timeseries.rst
Original file line number Diff line number Diff line change
Expand Up @@ -890,7 +890,7 @@ into ``freq`` keyword arguments. The available date offsets and associated frequ
:class:`~pandas.tseries.offsets.CBMonthBegin` or :class:`~pandas.tseries.offsets.CustomBusinessMonthBegin`, ``'CBMS'``, "custom business month begin"
:class:`~pandas.tseries.offsets.SemiMonthEnd`, ``'SM'``, "15th (or other day_of_month) and calendar month end"
:class:`~pandas.tseries.offsets.SemiMonthBegin`, ``'SMS'``, "15th (or other day_of_month) and calendar month begin"
:class:`~pandas.tseries.offsets.QuarterEnd`, ``'Q'``, "calendar quarter end"
:class:`~pandas.tseries.offsets.QuarterEnd`, ``'QE'``, "calendar quarter end"
:class:`~pandas.tseries.offsets.QuarterBegin`, ``'QS'``, "calendar quarter begin"
:class:`~pandas.tseries.offsets.BQuarterEnd`, ``'BQ``, "business quarter end"
:class:`~pandas.tseries.offsets.BQuarterBegin`, ``'BQS'``, "business quarter begin"
Expand Down Expand Up @@ -1254,7 +1254,7 @@ frequencies. We will refer to these aliases as *offset aliases*.
"SMS", "semi-month start frequency (1st and 15th)"
"BMS", "business month start frequency"
"CBMS", "custom business month start frequency"
"Q", "quarter end frequency"
"QE", "quarter end frequency"
"BQ", "business quarter end frequency"
"QS", "quarter start frequency"
"BQS", "business quarter start frequency"
Expand Down Expand Up @@ -1373,18 +1373,18 @@ For some frequencies you can specify an anchoring suffix:
"W\-THU", "weekly frequency (Thursdays)"
"W\-FRI", "weekly frequency (Fridays)"
"W\-SAT", "weekly frequency (Saturdays)"
"(B)Q(S)\-DEC", "quarterly frequency, year ends in December. Same as 'Q'"
"(B)Q(S)\-JAN", "quarterly frequency, year ends in January"
"(B)Q(S)\-FEB", "quarterly frequency, year ends in February"
"(B)Q(S)\-MAR", "quarterly frequency, year ends in March"
"(B)Q(S)\-APR", "quarterly frequency, year ends in April"
"(B)Q(S)\-MAY", "quarterly frequency, year ends in May"
"(B)Q(S)\-JUN", "quarterly frequency, year ends in June"
"(B)Q(S)\-JUL", "quarterly frequency, year ends in July"
"(B)Q(S)\-AUG", "quarterly frequency, year ends in August"
"(B)Q(S)\-SEP", "quarterly frequency, year ends in September"
"(B)Q(S)\-OCT", "quarterly frequency, year ends in October"
"(B)Q(S)\-NOV", "quarterly frequency, year ends in November"
"(B)Q(E)(S)\-DEC", "quarterly frequency, year ends in December. Same as 'QE'"
"(B)Q(E)(S)\-JAN", "quarterly frequency, year ends in January"
"(B)Q(E)(S)\-FEB", "quarterly frequency, year ends in February"
"(B)Q(E)(S)\-MAR", "quarterly frequency, year ends in March"
"(B)Q(E)(S)\-APR", "quarterly frequency, year ends in April"
"(B)Q(E)(S)\-MAY", "quarterly frequency, year ends in May"
"(B)Q(E)(S)\-JUN", "quarterly frequency, year ends in June"
"(B)Q(E)(S)\-JUL", "quarterly frequency, year ends in July"
"(B)Q(E)(S)\-AUG", "quarterly frequency, year ends in August"
"(B)Q(E)(S)\-SEP", "quarterly frequency, year ends in September"
"(B)Q(E)(S)\-OCT", "quarterly frequency, year ends in October"
"(B)Q(E)(S)\-NOV", "quarterly frequency, year ends in November"
"(B)Y(S)\-DEC", "annual frequency, anchored end of December. Same as 'Y'"
"(B)Y(S)\-JAN", "annual frequency, anchored end of January"
"(B)Y(S)\-FEB", "annual frequency, anchored end of February"
Expand Down Expand Up @@ -1692,7 +1692,7 @@ the end of the interval.
.. warning::

The default values for ``label`` and ``closed`` is '**left**' for all
frequency offsets except for 'ME', 'Y', 'Q', 'BME', 'BY', 'BQ', and 'W'
frequency offsets except for 'ME', 'Y', 'QE', 'BME', 'BY', 'BQ', and 'W'
which all have a default of 'right'.

This might unintendedly lead to looking ahead, where the value for a later
Expand Down
14 changes: 11 additions & 3 deletions doc/source/whatsnew/v0.18.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -808,11 +808,19 @@ Upsampling operations take you from a lower frequency to a higher frequency. The
performed with the ``Resampler`` objects with :meth:`~Resampler.backfill`,
:meth:`~Resampler.ffill`, :meth:`~Resampler.fillna` and :meth:`~Resampler.asfreq` methods.

.. ipython:: python
.. code-block:: ipython

s = pd.Series(np.arange(5, dtype='int64'),
In [89]: s = pd.Series(np.arange(5, dtype='int64'),
index=pd.date_range('2010-01-01', periods=5, freq='Q'))
s

In [90]: s
Out[90]:
2010-03-31 0
2010-06-30 1
2010-09-30 2
2010-12-31 3
2011-03-31 4
Freq: Q-DEC, Length: 5, dtype: int64

Previously

Expand Down
22 changes: 22 additions & 0 deletions doc/source/whatsnew/v2.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,28 @@ For example:

pd.date_range('2020-01-01', periods=3, freq='ME')

Deprecate alias ``Q`` in favour of ``QE`` for offsets
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very similar to the previous section, perhaps we can combine them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I agree. I combined these two sections together.


The alias ``Q`` is deprecated in favour of ``QE`` for offsets, please use ``QE`` for "quarter end" instead of ``Q`` (:issue:`9586`)

For example:

*Previous behavior*:

.. code-block:: ipython

In [8]: pd.date_range('2020-01-01', periods=3, freq='Q-NOV')
Out[8]:
DatetimeIndex(['2020-02-29', '2020-05-31', '2020-08-31'],
dtype='datetime64[ns]', freq='Q-NOV')

*Future behavior*:

.. ipython:: python

pd.date_range('2020-01-01', periods=3, freq='QE-NOV')

Other Deprecations
^^^^^^^^^^^^^^^^^^
- Changed :meth:`Timedelta.resolution_string` to return ``h``, ``min``, ``s``, ``ms``, ``us``, and ``ns`` instead of ``H``, ``T``, ``S``, ``L``, ``U``, and ``N``, for compatibility with respective deprecations in frequency aliases (:issue:`52536`)
Expand Down
27 changes: 26 additions & 1 deletion pandas/_libs/tslibs/dtypes.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,19 @@ OFFSET_TO_PERIOD_FREQSTR: dict = {
"us": "us",
"ns": "ns",
"h": "h",
"Q": "Q",
"QE": "Q",
"QE-DEC": "Q-DEC",
"QE-JAN": "Q-JAN",
"QE-FEB": "Q-FEB",
"QE-MAR": "Q-MAR",
"QE-APR": "Q-APR",
"QE-MAY": "Q-MAY",
"QE-JUN": "Q-JUN",
"QE-JUL": "Q-JUL",
"QE-AUG": "Q-AUG",
"QE-SEP": "Q-SEP",
"QE-OCT": "Q-OCT",
"QE-NOV": "Q-NOV",
"W": "W",
"ME": "M",
"Y": "Y",
Expand All @@ -211,6 +223,19 @@ OFFSET_TO_PERIOD_FREQSTR: dict = {
}
OFFSET_DEPR_FREQSTR: dict[str, str]= {
"M": "ME",
"Q": "QE",
"Q-DEC": "QE-DEC",
"Q-JAN": "QE-JAN",
"Q-FEB": "QE-FEB",
"Q-MAR": "QE-MAR",
"Q-APR": "QE-APR",
"Q-MAY": "QE-MAY",
"Q-JUN": "QE-JUN",
"Q-JUL": "QE-JUL",
"Q-AUG": "QE-AUG",
"Q-SEP": "QE-SEP",
"Q-OCT": "QE-OCT",
"Q-NOV": "QE-NOV",
}
cdef dict c_OFFSET_TO_PERIOD_FREQSTR = OFFSET_TO_PERIOD_FREQSTR
cdef dict c_OFFSET_DEPR_FREQSTR = OFFSET_DEPR_FREQSTR
Expand Down
6 changes: 3 additions & 3 deletions pandas/_libs/tslibs/offsets.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -2767,7 +2767,7 @@ cdef class QuarterEnd(QuarterOffset):
Timestamp('2022-03-31 00:00:00')
"""
_default_starting_month = 3
_prefix = "Q"
_prefix = "QE"
_day_opt = "end"

cdef readonly:
Expand Down Expand Up @@ -4585,7 +4585,7 @@ prefix_mapping = {
Second, # 's'
Minute, # 'min'
Micro, # 'us'
QuarterEnd, # 'Q'
QuarterEnd, # 'QE'
QuarterBegin, # 'QS'
Milli, # 'ms'
Hour, # 'h'
Expand All @@ -4603,7 +4603,7 @@ opattern = re.compile(

_lite_rule_alias = {
"W": "W-SUN",
"Q": "Q-DEC",
"QE": "QE-DEC",

"Y": "Y-DEC", # YearEnd(month=12),
"YS": "YS-JAN", # YearBegin(month=1),
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/indexes/accessors.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ class DatetimeProperties(Properties):
2 2
dtype: int32

>>> quarters_series = pd.Series(pd.date_range("2000-01-01", periods=3, freq="q"))
>>> quarters_series = pd.Series(pd.date_range("2000-01-01", periods=3, freq="qe"))
>>> quarters_series
0 2000-03-31
1 2000-06-30
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/resample.py
Original file line number Diff line number Diff line change
Expand Up @@ -2101,7 +2101,7 @@ def __init__(
else:
freq = to_offset(freq)

end_types = {"ME", "Y", "Q", "BME", "BY", "BQ", "W"}
end_types = {"ME", "Y", "QE", "BME", "BY", "BQ", "W"}
rule = freq.rule_code
if rule in end_types or ("-" in rule and rule[: rule.find("-")] in end_types):
if closed is None:
Expand Down Expand Up @@ -2300,7 +2300,7 @@ def _adjust_bin_edges(
if self.freq.name in ("BME", "ME", "W") or self.freq.name.split("-")[0] in (
"BQ",
"BY",
"Q",
"QE",
"Y",
"W",
):
Expand Down
3 changes: 3 additions & 0 deletions pandas/io/json/_table_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from pandas._libs import lib
from pandas._libs.json import ujson_loads
from pandas._libs.tslibs import timezones
from pandas._libs.tslibs.dtypes import freq_to_period_freqstr
from pandas.util._exceptions import find_stack_level

from pandas.core.dtypes.base import _registry as registry
Expand Down Expand Up @@ -207,6 +208,8 @@ def convert_json_field_to_pandas_type(field) -> str | CategoricalDtype:
if field.get("tz"):
return f"datetime64[ns, {field['tz']}]"
elif field.get("freq"):
# GH#9586 rename frequency M to ME for offsets
field["freq"] = freq_to_period_freqstr(1, field["freq"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left over from #55581 (review)? should we wait for that one to be resolved first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it' related to #55581. I agree, it's better at first to merge #55581. I rolled my changes in pandas/io/json/_table_schema.py back , but now I have failures in test_read_json_table_period_orient, which will be resolved after merging.

# GH#47747 using datetime over period to minimize the change surface
return f"period[{field['freq']}]"
else:
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/arithmetic/test_datetime64.py
Original file line number Diff line number Diff line change
Expand Up @@ -1076,7 +1076,7 @@ def test_dt64arr_add_dtlike_raises(self, tz_naive_fixture, box_with_array):
# Note: freq here includes both Tick and non-Tick offsets; this is
# relevant because historically integer-addition was allowed if we had
# a freq.
@pytest.mark.parametrize("freq", ["h", "D", "W", "2ME", "MS", "Q", "B", None])
@pytest.mark.parametrize("freq", ["h", "D", "W", "2ME", "MS", "QE", "B", None])
@pytest.mark.parametrize("dtype", [None, "uint8"])
def test_dt64arr_addsub_intlike(
self, request, dtype, box_with_array, freq, tz_naive_fixture
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/arrays/test_datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@


# TODO: more freq variants
@pytest.fixture(params=["D", "B", "W", "ME", "Q", "Y"])
@pytest.fixture(params=["D", "B", "W", "ME", "QE", "Y"])
def freqstr(request):
"""Fixture returning parametrized frequency in string format."""
return request.param
Expand Down
19 changes: 15 additions & 4 deletions pandas/tests/arrays/test_datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -746,12 +746,23 @@ def test_iter_zoneinfo_fold(self, tz):
assert str(left) == str(right2)
assert left.utcoffset() == right2.utcoffset()

def test_date_range_frequency_M_deprecated(self):
depr_msg = "'M' will be deprecated, please use 'ME' instead."
@pytest.mark.parametrize(
"freq, freq_depr",
[
("2ME", "2M"),
("2QE", "2Q"),
("2QE-SEP", "2Q-SEP"),
],
)
def test_date_range_frequency_M_Q_deprecated(self, freq, freq_depr):
# GH#9586
depr_msg = (
f"'{freq_depr[1:]}' will be deprecated, please use '{freq[1:]}' instead."
)

expected = pd.date_range("1/1/2000", periods=4, freq="2ME")
expected = pd.date_range("1/1/2000", periods=4, freq=freq)
with tm.assert_produces_warning(UserWarning, match=depr_msg):
result = pd.date_range("1/1/2000", periods=4, freq="2M")
result = pd.date_range("1/1/2000", periods=4, freq=freq_depr)
tm.assert_index_equal(result, expected)


Expand Down
23 changes: 17 additions & 6 deletions pandas/tests/frame/methods/test_asfreq.py
Original file line number Diff line number Diff line change
Expand Up @@ -234,12 +234,23 @@ def test_asfreq_2ME(self, freq, freq_half):
result = DataFrame({"s": Series([0.0, 2.0, 4.0], index=index)})
tm.assert_frame_equal(result, expected)

def test_asfreq_frequency_M_deprecated(self):
depr_msg = "'M' will be deprecated, please use 'ME' instead."
@pytest.mark.parametrize(
"freq, freq_depr",
[
("2ME", "2M"),
("2QE", "2Q"),
("2QE-SEP", "2Q-SEP"),
],
)
def test_asfreq_frequency_M_Q_deprecated(self, freq, freq_depr):
# GH#9586
depr_msg = (
f"'{freq_depr[1:]}' will be deprecated, please use '{freq[1:]}' instead."
)

index = date_range("1/1/2000", periods=4, freq="ME")
index = date_range("1/1/2000", periods=4, freq=f"{freq[1:]}")
df = DataFrame({"s": Series([0.0, 1.0, 2.0, 3.0], index=index)})
expected = df.asfreq(freq="5ME")
expected = df.asfreq(freq=freq)
with tm.assert_produces_warning(UserWarning, match=depr_msg):
result = df.asfreq(freq="5M")
tm.assert_frame_equal(result, expected)
result = df.asfreq(freq=freq_depr)
tm.assert_frame_equal(result, expected)
2 changes: 1 addition & 1 deletion pandas/tests/frame/test_nonunique_indexes.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ def test_setattr_columns_vs_construct_with_columns(self):
check(df, expected)

def test_setattr_columns_vs_construct_with_columns_datetimeindx(self):
idx = date_range("20130101", periods=4, freq="Q-NOV")
idx = date_range("20130101", periods=4, freq="QE-NOV")
df = DataFrame(
[[1, 1, 1, 5], [1, 1, 2, 5], [2, 1, 3, 5]], columns=["a", "a", "a", "a"]
)
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/groupby/test_timegrouper.py
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@ def test_timegrouper_with_reg_groups(self):
)
tm.assert_frame_equal(result, expected)

@pytest.mark.parametrize("freq", ["D", "ME", "Y", "Q-APR"])
@pytest.mark.parametrize("freq", ["D", "ME", "Y", "QE-APR"])
def test_timegrouper_with_reg_groups_freq(self, freq):
# GH 6764 multiple grouping with/without sort
df = DataFrame(
Expand Down
19 changes: 18 additions & 1 deletion pandas/tests/indexes/datetimes/methods/test_to_period.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def test_to_period_quarterly(self, month):
def test_to_period_quarterlyish(self, off):
rng = date_range("01-Jan-2012", periods=8, freq=off)
prng = rng.to_period()
assert prng.freq == "Q-DEC"
assert prng.freq == "QE-DEC"

@pytest.mark.parametrize("off", ["BY", "YS", "BYS"])
def test_to_period_annualish(self, off):
Expand Down Expand Up @@ -89,6 +89,23 @@ def test_dti_to_period_2monthish(self, freq_offset, freq_period):

tm.assert_index_equal(pi, period_range("2020-01", "2020-05", freq=freq_period))

@pytest.mark.parametrize(
"freq, freq_depr",
[
("2ME", "2M"),
("2QE", "2Q"),
("2QE-SEP", "2Q-SEP"),
],
)
def test_to_period_freq_deprecated(self, freq, freq_depr):
# GH#9586
msg = f"'{freq_depr[1:]}' will be deprecated, please use '{freq[1:]}' instead."

rng = date_range("01-Jan-2012", periods=8, freq=freq)
prng = rng.to_period()
with tm.assert_produces_warning(UserWarning, match=msg):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I'd missed this last time, but this should be FutureWarning, not UserWarning #52064 (review)

could we address that in a separate PR please?

Copy link
Contributor Author

@natmokval natmokval Oct 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you, I corrected Warning class and made the separate PR #55636

assert prng.freq == freq_depr

def test_to_period_infer(self):
# https://github.com/pandas-dev/pandas/issues/33358
rng = date_range(
Expand Down
3 changes: 2 additions & 1 deletion pandas/tests/indexes/datetimes/test_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -1036,7 +1036,8 @@ def test_constructor_int64_nocopy(self):
assert (index.asi8[50:100] != -1).all()

@pytest.mark.parametrize(
"freq", ["ME", "Q", "Y", "D", "B", "bh", "min", "s", "ms", "us", "h", "ns", "C"]
"freq",
["ME", "QE", "Y", "D", "B", "bh", "min", "s", "ms", "us", "h", "ns", "C"],
)
def test_from_freq_recreate_from_data(self, freq):
org = date_range(start="2001/02/01 09:00", freq=freq, periods=1)
Expand Down
4 changes: 2 additions & 2 deletions pandas/tests/indexes/datetimes/test_date_range.py
Original file line number Diff line number Diff line change
Expand Up @@ -965,7 +965,7 @@ def test_3(self):
def test_precision_finer_than_offset(self):
# GH#9907
result1 = date_range(
start="2015-04-15 00:00:03", end="2016-04-22 00:00:00", freq="Q"
start="2015-04-15 00:00:03", end="2016-04-22 00:00:00", freq="QE"
)
result2 = date_range(
start="2015-04-15 00:00:03", end="2015-06-22 00:00:04", freq="W"
Expand All @@ -989,7 +989,7 @@ def test_precision_finer_than_offset(self):
"2015-06-21 00:00:03",
]
expected1 = DatetimeIndex(
expected1_list, dtype="datetime64[ns]", freq="Q-DEC", tz=None
expected1_list, dtype="datetime64[ns]", freq="QE-DEC", tz=None
)
expected2 = DatetimeIndex(
expected2_list, dtype="datetime64[ns]", freq="W-SUN", tz=None
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/indexes/datetimes/test_misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ def test_datetimeindex_accessors4(self):
def test_datetimeindex_accessors5(self):
freq_m = to_offset("ME")
bm = to_offset("BME")
qfeb = to_offset("Q-FEB")
qfeb = to_offset("QE-FEB")
qsfeb = to_offset("QS-FEB")
bq = to_offset("BQ")
bqs_apr = to_offset("BQS-APR")
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/indexes/datetimes/test_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ class TestDatetimeIndexOps:
"freq,expected",
[
("Y", "day"),
("Q", "day"),
("QE", "day"),
("ME", "day"),
("D", "day"),
("h", "hour"),
Expand Down
Loading
Loading