-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
side
option to define minute bounds / test suite overhaul
#71
Conversation
Adds side parameter to ExchangeCalendar to define the bounds of sessions´ trading minutes. updates ExchangeCalendar.minute_index_to_session_labels. updates ExchangeCalendar.minute_to_session_label. updates ExchangeCalendar.is_open_on_minute. Also: adds NoSessionsError.
Updates 24 hour calendars to have: same open time and close time. day offset as applicable. valid sides options as only 'left' or 'right'. Updates 24 hour calendar csv files to reflect new times. Introduces ExchangeCalendar.default_side. "right" for 24 hour calendars, "both" for all others.
Proposes a basis for a new ExchangeCalendarTestBase which accommodates testing ExchangeCalendar methods for calendars based on all valid 'side' options. ExchangeCalendarTestBaseProposal includes a limited number of tests to show proposed implementation. Answers class provides input for and expected returns from ExchangeCalendar test methods. Incorporated to '24-5', '24-7', 'CMES', 'XLON' and 'XHKG'. Native pytest.
TODOTests need speeding up.
Then (EDIT - FOLLOWING PROCESS MOVED TO / SUPERSEDED BY (#86))
|
Hey @maread99 - thanks for putting this up and documenting this really well. I think my high level comments would be:
|
@gerrymanoim, thanks for looking through this and for your comments.
I’m not able to reply fully at the moment although will do towards the end of the month when I expect to move this forward and incorporate your suggestions. Cheers. |
@gerrymanoim, thanks again for your comments from which I'm assuming you're happy, subject to the following, for me to progress with the proposed
I'm assuming the concern here is memory? The arrays received are always limited in size (to the number of the calendar's days) and the copy is then worked on directly to create the return, i.e. there's no intermediary object taking up memory.
I anticipate adding a
👍 I'll update the What times are considered open and closed? section of FAQs.
I hope to get through all of this over the next couple of weeks. To aid reviews I'll work on the side branch with sequential commits. To avoid conflicts with recent and any future merged PRs, I might request an intermediary merge before moving the calendar test modules to the new test base.
😄. I did give it some thought before settling on 'side'. I originally had it as the arguably worse 'closed'. I guess that, with usage, parameter names take on the significance of the functionality they provide. Of course, more than open to suggestions for an alternative name that might better indicate what the option does.
My intenion is to keep all the critical evaluations (test inputs / expected outputs) within |
Revisons to testing to increase speed and coverage.
7d2653d
to
f6da99a
Compare
adds ExchangeCalendar.is_trading_minute. adds ExchangeCalendar.is_break_minute. updates ExchangeCalendar.is_open_on_minute.
Extends ExchangeCalendarTestBaseProposal to include (and supporting fixtures): test_calculated_against_csv test_bound_start test_bound_end test_sanity_check_session_lengths test_adhoc_holidays_specification test_prev_next_open_close Extends Answers to support newly included tests. Adds parsing and bounds handling to ExchangeCalendar methods previous_open, previous_close, next_open, next_close.
Reduces setup time of cached properties.
Extends ExchangeCalendarTestBaseProposal to include following methods: test_next_prev_minute Extends Answers to support newly included test. Adds parsing and bounds handling to ExchangeCalendar methods previous_minute and next_minute.
Removes _minute_to_session_label_cache from ExchangeCalendar.
Yep - I think there's a lot here, but overall everything is reasonable.
That's a fair point, I think my instinct is to just use an enum where string constants are needed and do
Mostly performance actually, but I agree with your point that these are generally of a limited size so it should be fine.
What would be the benefit for these other methods?
Yep - that's fine - don't want you to have to keep having issues with other PRs coming in.
Yep I also can't think of anything particularly better here.
|
Extends ExchangeCalendarTestBaseProposal to include following methods: test_next_prev_session test_date_to_sesion_label test_minutes_in_range Extends Answers to support newly included test. Adds parsing and updates documentation for ExchangeCalendar methods next_session_label, previous_session_label, date_to_session_label, minutes_in_range.
Orders methods into sections. Identifies those properties that should, and should not, be overriden by a subclass to define a calendar.
Adds ExchangeCalendar minute properties/ methods: first_minutes last_minutes last_am_minutes first_pm_minutes session_first_minute() session_last_minute() session_last_am_minute() session_first_pm_minute() Adds tests to ExchangeCalendarTestBaseProposal: test_minutes_properties test_session_minute_methods Renames Answers properties: first_trading_minutes* to first_minutes* last_trading_minutes* to last_minutes* last_am_trading_minutes* to last_am_minutes* first_pm_trading_minutes* to first_pm_minutes*
@gerrymanoim, thanks for your further comments.
My own usage of exchange calendars led me to accommodate this functionality - very occassionally I found a need to treat 'side' differently from the default for a specific purpose (similar to how
👍 I anticipate having the new test base proposal completed this week, at which point this will be good for reviewing / merging. I'll offer a summary of changes made and what will be left to do over subsequent PRs. |
Yep - sounds good to me. |
Reorders test methods of ExchangeCalendarTestBaseProposal to follow order of corresponding methods of ExchangeCalendar.
Extends ExchangeCalendarTestBaseProposal to include following methods: test_minutes_for_session test_minutes_for_sessions_in_range Extends Answers to support newly included tests. Revises, adds parsing and updates documentation for ExchangeCalendar methods: minutes_for_session minutes_for_sessions_in_range Adds deprecate decorator (for deprecating ExchangeCalendar methods). Deprecates following methods from ExchangeCalendar and moves functionality exclusively to QuantopianUSFuturesCalendar: execution_minutes_for_session execution_minutes_for_sessions_in_range execution_time_from_open execution_time_from_close
Adds following methods to ExchangeCalendarTestBaseProposal: test_start_end test_daylight_savings test_has_breaks test_session_has_break test_minutes_count_for_sessions_in_range (not previously tested) Adds daylight_saving_days fixture to ExchangeCalendarTestBaseProposal. Extends Answers to support new tests. Adds parsing and updates documentation for ExchangeCalendar methods: has_breaks session_has_break Rewrites `ExchangeCalendar.minutes_count_for_sessions_in_range`.
Adds tests to ExchangeCalendarTestBaseProposal for previously untested methods/properties: `test_all_sessions` `test_opens_closes_break_starts_ends` `test_calendar_bounds_properties` `test_late_opens` `test_early_closes` `test_is_session` `test_minutes_window` Adds fixtures to ExchangeCalendarTestBaseProposal: late_opens early_closes Extends Answers to support new tests. Adds parsing (if applic) and updates documentation for ExchangeCalendar properties/methods: `opens` `closes` `break_starts` `break_ends` `late_opens` `early_closes` `is_session` `minutes_window` Fixes `ExchangeCalendar.minutes_window` (forward looking window would start prior to `start_dt` if `start_dt` not a trading minute). Implemented to have same behaviour as `sessions_window`. Adds to calendar_helpers.py: `TradingMinute` type `parse_trading_minute` function Adds `test_parse_trading_minute` test to `test_calendar_helpers.py`. Adds `NotTradingMinuteError` to errors.py.
side
option to define sessions' trading minute boundsside
option to define minute bounds / test suite overhaul
@gerrymanoim, I think I'm there, with this initial PR ready to merge pending your review. It's ended up as quite a bit more than implementing
TestingContinuing from my earlier notes... Generally tests have been made more comprehensive as a result of:
Speed I anticipate that the new test suite will run somewhat slower than the existing, possibly around 1.7x, maybe less. This is partly due to the more comprehensive testing and partly due to methods that are dependent on the evaluation of minutes now being tested against four versions of each calendar (one for each side). Other points of note
Moving OnI've laid out in #86 the path to moving calendars over to the new test base. I envisage this ideally being followed by some method renaming (#85 - what do you reckon?), a revised README and culminating with cutting 3.4(?). |
Updates calendar_helpers.py parse_timestamp function to handle input with a non-zero second (or more accurate) component.
I've added one small commit to cover something that became apparent to me when working on documentation. @gerrymanoim, you might want to check out the proposed doc - it ties together the side option covered by this PR and the proposed renaming (#85). It's on the 'readme' branch (I've overhauled the README and added a couple of .ipynb tutorials). |
@gerrymanoim, although I've added another small commit this is no longer a WIP (at least as far as this PR is concerned) and is good to review / merge as and when you're in a position to look at it... |
Awesome! Will take a look in the next few days. |
Great! Be sure to check out the proposed doc on the 'readme' branch - README overhaul and 4 ipynb tutorials. It's intended to be published alongside release 3.4 (assumes this PR is merged and that the renaming covered in #85 happens). |
|
||
|
||
@pytest.mark.skip |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this purposely skipped?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I included a note on this at the end of the first comment.
Basically, introducing the side option has allowed the 24 hour calendars to be truly 24 hour - the open times have been moved 'back' one minute to represent the actual open time. These calendars can only take side as "left" or "right". They default to "right" to maintain the original behaviour; each minute remains associated with the same session as perviously. Whilst the behvaiour hasn't changed (save for a change to the open time) some of the old tests fail as they were designed on the assumption that the open minute and close minute would both be a trading minutes - this assumption continues to hold for all the other calendars (which default to side "both") but not the 24 hour ones.
All the 24 hour calendars are subjected to and pass the new test suite.
@@ -532,71 +575,253 @@ def special_offsets_adhoc(self): | |||
""" | |||
return [] | |||
|
|||
# ----- | |||
# ------------------------------------------------------------------ | |||
# -- NO method below this line should be overriden on a subclass! -- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you wanted to, we could enforce this via an __init_subclass__
checking this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't want to impede users doing what they want with their own calendars. I've added a test test_base_integrity
to ensure the integrity of the pre-defined ones.
@@ -250,13 +333,21 @@ def __init__(self, start: Date | None = None, end: Date | None = None): | |||
self.last_trading_session = _all_days[-1] | |||
|
|||
self._late_opens = pd.DatetimeIndex( | |||
_special_opens.map(self.minute_index_to_session_labels) | |||
_special_opens.map(lambda x: self.minute_index_to_session_labels(x, "both")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this set to both?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It shouldn't be! _special_opens
should map to sessions based on minutes evaluated for side "left" and _special_closes
should map to sessions based on minutes evaluated for side "right". It could be left as "both" if not for the 24 hour calendars (which don't have a "both" option). I had it as "both" simply to maintain the original behaviour - the only reason this didn't result in a bug is that no existing 24 hour calendar has a late open. I would now change these to "left" and "right", but...
Having looked into what's going on here the whole mapping appears to be superfluous. _late_opens
as evaluated is actually already represented by _special_opens.index
and _early_closes
by _special_closes.index
, i.e. the assignment line under review could be replaced with a simple:
self._late_opens = _special_opens.index
I'm not sure where the need for the method call came from? Perhaps _special_opens
was originally defined differently? In any event, I'll add a commit to change these assignments to the direct assignment (as above). Let me know if you think the mapping is necessary and that I'm missing something here.
NB a consequence of losing the call to minute_index_to_session_labels
is that it's more likely that the *_minute_nanos
methods will end up being changed to properties (item 5 on #86).
Some minor questions/comments, but LGTM. Feel free to merge when you're ready. The new notebooks are great! As an FYI github will render the notebooks directly, but I don't have view on linking directly vs nbviewer. |
Great. I'm going to add a commit or two to cover the issues / points you've raised and then I'll merge and move forwards with moving the test modules over to the new suite.
Glad you like them. I just prefer the nbviewer rendering. The only issue is that (I believe) it'll be necessary to update the nbviewer links on the README every time the notebooks are updated (I might yet end up linking to the GitHub pages). |
Updates evaluation of `_late_opens` and `_early_closes`.
Adds test `test_base_integrity` to ensure a subclass does not override base properties or methods that are not intended to be overriden.
We can always transition to sphinx in the future |
@gerrymanoim, Hi Gerry, I feel the time's right for a review of the
side
option PR before I plough on any further!This draft PR includes the
side
implementation in full and a proposal for an overhaul of ExchangeCalendarTestBase.Of the three commits:
side
passed to the constructor.side
options for these calendars is limited to "left" or "right" in order that every trading minute belongs to one session only.(you'll notice I've pushed these commits to a 'side' branch and set up this draft PR from there).
Rather than replace the existing
ExchangeCalendarTestBase
, at this stage I've simply added a newExchangeCalendarTestBaseProposal
to the end oftest_exchange_calendar.py
and implemented it only to the test modules of the following selected calendars:ExchangeCalendarTestBaseProposal includes only the following methods for now (enough I hope to give you an appreciation of the proposed implementation):
test_all_minutes
- new test to test trading mintues at sessions' bounds.test_minute_to_session_label
- replacement for existing test.test_minute_index_to_session_labels
- replacement for existing test.test_is_open_on_minute
- replacement for existing test.test_invalid_input
- new test to test that constructor raises expected errors on invalid inputs.Main features of the proposed approach:
Answers
class that, from interrogation of the corresponding csv file, provides inputs and expected return values for methods under test. Importantly, it is not intended that this class does more than minimial evaluations. It should not in any way 'repeat the code' of the methods being tested. It does the same kind of work currently done ad-hoc within test methods.side
parameter that provides for its properties to reflect a specificside
option.all_calendars_and_answers
. Any test that inherits this fixture will be exectuted with a calendar / answers combo for each side option.If you were to be happy with the general approach then my intenion would be to:
ExchangeCalendarTestBaseProposal
to cover all tests ofExchangeCalendarTestBase
and eventually replace it.I look forward to hearing your thoughts!
NB with the exception of the 24 hour calendars the default side is "both" which fully replicates the existing behaviour. Accordingly all tests for non-24hour calendars continue to pass. I've skipped the standard ExchangeCalendar tests for the 24h calendars - some of these tests will fail given that these calendars are, intentionally, not acting as if side were "both" (i.e. the behaviour that the tests were designed for).