Error running zipline backtest with custom trading calendar

[richafrank]

From @vonpupp in #1800 (comment):

Hi @freddiev4 ,

I am new to zipline. I am trying to use a 24/7 calendar, so I am trying to use the feature of this PR already to do it in a proper way. I am using zipline installed via pip directly from github's egg (`pip install git+git://github.com/quantopian/zipline')

Unfortunately I am not able to use this feature properly, here is what I am doing:

I edited the extensions.py as follows:

# A bunch of imports removed for the sake of clarity

class TwentyFourSevenCalendar(TradingCalendar):
    """
    Exchange calendar for 24/7 trading.

    Open Time: 12am, UTC
    Close Time: 11:59pm, UTC

    """
    @property
    def name(self):
        return "TWENTYFOURSEVEN"

    @property
    def tz(self):
        return timezone("UTC")

    @property
    def open_time(self):
        return time(0, 0)

    @property
    def close_time(self):
        return time(23,59)

    @lazyval
    def day(self):
        return CustomBusinessDay(
            weekmask='Mon Tue Wed Thu Fri Sat Sun',
        )

start_session = pd.Timestamp('2014-01-07', tz='utc')
end_session = pd.Timestamp('2017-11-13', tz='utc')
assets = ['BTC_USD']

register_calendar(
    'TWENTYFOURSEVEN',
    TwentyFourSevenCalendar(
        start=start_session,
        end=end_session
    )
)
register(
    'custom-csvdir-bundle',
    csvdir_equities(["daily"], '/home/av/repos/ziplinetest/csvdir'),
    calendar_name='TWENTYFOURSEVEN',
    start_session=start_session,
    end_session=end_session
)

As you can see I am using csvdir bundle which has recently been merged. The ingestion process seems to work fine (no errors and I also verified the directory and the sqlite files exist, so I assume the ingestion is ok). I conclude therefore that the 24/7 calendar works properly during ingestion.

When I try to execute with:

zipline run -f csvdir_example.py \
    -b custom-csvdir-bundle \
    --start 2016-1-1 \
    --end 2017-1-1 \
    --data-frequency daily \
    --trading-calendar TWENTYFOURSEVEN \
    --capital-base 100000 \
    -o csvdir_example.pickle

I get:

Error: Invalid value for "--trading-calendar": invalid choice: TWENTYFOURSEVEN. (choose from BMF, CFE, CME, ICE, LSE, NYSE, TSX, us_futures)

Could you please help me?

Thanks a lot.

[freddiev4]

Ah looks like we're loading the options for --trading-calendar using:

default_calendar_names = sorted(_default_calendar_factories.keys())

in zipline.utils.calendars.calendar_utils.

And then passing that to the CLI zipline/__main__.py

@click.option(
    '--trading-calendar',
    metavar='TRADING-CALENDAR',
    type=click.Choice(default_calendar_names),
    default='NYSE',
    help="The calendar you want to use e.g. LSE. NYSE is the default."
)

And so when we register a new calendar, the new calendar doesn't appear in default_calendar_names.

I'm thinking there are a couple solutions here:

• get the global list of available calendars and then pass that as type=click.Choice(calendars) to the @click.option()
• remove type=click.Choice(default_calendar_names)
• check if the arg someone passes in is in default_calendar_names, and if not, then check our registered calendars

[freddiev4]

Also, a note to add:

If I create a 24/7 calendar I'll also have to call set_benchmark('<some_other_asset>') in the initialize() function of my algorithm because the SPY data doesn't map to a 24/7 calendar (as it's a NYSE asset)

[freddiev4]

Made some fake assets and ingested them (.head() of one below):

date,open,high,low,close,volume,dividend,split
2010-01-01 00:00:00+00:00,54.7428,61.066,75.3803,85.1145,86.4614,0.0,1.0
2010-01-02 00:00:00+00:00,33.9725,41.7909,43.2215,99.8412,56.173,0.0,1.0
2010-01-03 00:00:00+00:00,43.3257,46.8696,76.9452,36.877,67.5251,0.0,1.0
2010-01-04 00:00:00+00:00,34.4473,93.6755,63.3079,39.8946,55.5203,0.0,1.0
2010-01-05 00:00:00+00:00,87.7385,69.6116,76.7587,85.1788,33.3338,0.0,1.0

And then a toy algo:

from zipline.api import (
    order_target_percent,
    set_benchmark,
    symbol
)


def initialize(context):
    set_benchmark('FAKE')
    context.asset = symbol('FAKE')

def handle_data(context, data):
    order_target_percent(context.asset, 0.5)

Ran:

zipline run -f order_fake.py \
    -b custom-csvdir-bundle \
    --start 2014-1-1 \
    --end 2016-1-1 \
    --data-frequency daily \
    --trading-calendar TFS \
    --capital-base 100000 \
    -o twentyfourseven.pickle

And then ran into this:

[2017-11-24 12:04:05.683714] WARNING: Loader: Refusing to download new benchmark data because a download succeeded at 2017-11-24 11:30:34+00:00.
Traceback (most recent call last):
  File "pandas/index.pyx", line 567, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:10828)
  File "pandas/hashtable.pyx", line 303, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:6610)
  File "pandas/hashtable.pyx", line 309, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:6554)
KeyError: 1262304000000000000

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/freddiev4/.virtualenvs/zip35/bin/zipline", line 11, in <module>
    load_entry_point('zipline', 'console_scripts', 'zipline')()
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/__main__.py", line 101, in _
    return f(*args, **kwargs)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/__main__.py", line 254, in run
    environ=os.environ,
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/utils/run_algo.py", line 141, in _run
    adjustment_reader=bundle_data.adjustment_reader,
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/data/data_portal.py", line 296, in __init__
    if self._first_trading_day is not None else (None, None)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/utils/calendars/trading_calendar.py", line 649, in open_and_close_for_session
    sched.at[session_label, 'market_open'].tz_localize('UTC'),
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/pandas/core/indexing.py", line 1649, in __getitem__
    return self.obj.get_value(*key, takeable=self._takeable)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/pandas/core/frame.py", line 1838, in get_value
    return engine.get_value(series.get_values(), index)
  File "pandas/index.pyx", line 103, in pandas.index.IndexEngine.get_value (pandas/index.c:3332)
  File "pandas/index.pyx", line 111, in pandas.index.IndexEngine.get_value (pandas/index.c:3035)
  File "pandas/index.pyx", line 569, in pandas.index.DatetimeEngine.get_loc (pandas/index.c:10895)
KeyError: Timestamp('2010-01-01 00:00:00+0000', tz='UTC')

[vonpupp]

Thanks for pointing the set_benchmark @freddiev4. I also thought about that but didn't know how to overcome that issue.

Regarding this last error, I have no idea. I will reproduce what you posted here to see if I have the same error.

[freddiev4]

When we enter the DataPortal() setup in zipline/zipline/data/data_portal.py(296), and call

# Get the first trading minute
self._first_trading_minute, _ = (
    self.trading_calendar.open_and_close_for_session(
    self._first_trading_day
)

We error at this point (i.e. the KeyError: 1262304000000000000 from the above post) because we actually only pass the NYSE calendar to the DataPortal() in zipline/utils/run_algo.py , so I changed the line to pass the calendar received from the CLI:

--- a/zipline/utils/run_algo.py
+++ b/zipline/utils/run_algo.py
@@ -134,7 +134,8 @@ def _run(handle_data,
         first_trading_day =\
             bundle_data.equity_minute_bar_reader.first_trading_day
         data = DataPortal(
-            env.asset_finder, get_calendar("NYSE"),
+            env.asset_finder,
+            trading_calendar=trading_calendar,
             first_trading_day=first_trading_day,
             equity_minute_reader=bundle_data.equity_minute_bar_reader,
             equity_daily_reader=bundle_data.equity_daily_bar_reader,

Then I end up with this error (looks like something having to do with sids/symbols, and the FAKE asset I generated)

Traceback (most recent call last):
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/assets/assets.py", line 433, in retrieve_asset
    asset = self._asset_cache[sid]
KeyError: 'FAKE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/freddiev4/.virtualenvs/zip35/bin/zipline", line 11, in <module>
    load_entry_point('zipline', 'console_scripts', 'zipline')()
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/__main__.py", line 101, in _
    return f(*args, **kwargs)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/__main__.py", line 254, in run
    environ=os.environ,
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/utils/run_algo.py", line 186, in _run
    overwrite_sim_params=False,
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/algorithm.py", line 722, in run
    for perf in self.get_generator():
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/algorithm.py", line 607, in get_generator
    return self._create_generator(self.sim_params)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/algorithm.py", line 586, in _create_generator
    self._create_benchmark_source(),
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/algorithm.py", line 545, in _create_benchmark_source
    self.benchmark_sid)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/assets/assets.py", line 438, in retrieve_asset
    return self.retrieve_all((sid,), default_none=default_none)[0]
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/assets/assets.py", line 482, in retrieve_all
    type_to_assets = self.group_by_type(missing)
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/assets/assets.py", line 426, in group_by_type
    return invert(self.lookup_asset_types(sids))
  File "/Users/freddiev4/Documents/quantopian/zipline/zipline/assets/assets.py", line 401, in lookup_asset_types
    self.asset_router.c.sid.in_(map(int, assets))
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/operators.py", line 424, in in_
    return self.operate(in_op, other)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 739, in operate
    return op(self.comparator, *other, **kwargs)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/operators.py", line 744, in in_op
    return a.in_(b)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/operators.py", line 424, in in_
    return self.operate(in_op, other)
  File "<string>", line 1, in <lambda>
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/type_api.py", line 60, in operate
    return o[0](self.expr, op, *(other + o[1:]), **kwargs)
  File "/Users/freddiev4/.virtualenvs/zip35/lib/python3.5/site-packages/sqlalchemy/sql/default_comparator.py", line 129, in _in_impl
    for o in seq_or_selectable:
ValueError: invalid literal for int() with base 10: 'FAKE'

[freddiev4]

Ah, ValueError: invalid literal for int() with base 10: 'FAKE' came from:

set_benchmark('FAKE')

Changed that to

set_benchmark(symbol('FAKE'))

[freddiev4]

@vonpupp with the merge of #2026 you should be able to get your calendar working.

You can install the latest zipline master branch by running:

git clone git@github.com:quantopian/zipline.git
pip install zipline/

[vonpupp]

Thank you very much @freddiev4.

I got away due to medical issues. I am focusing on catalyst (zipline for crypto) but I am also interested in zipline itself so I will give it a try.

[freddiev4]

Just did a new release (1.2.0). You should be able to pip install zipline and run with a custom trading calendar.

Going to close this. Feel free to reopen if you still have issues.

[fxyw]

split=1! I set it 0 originally and find moving average not work which seems is related to tradingcalendar. Search for module 'zipline.utils.tradingcalendar' has no attribute 'session_distance' find @freddiev4's example, this set solved my moving average problem.