Time not updating on data fetches. Stops trading.

[gerardbos]

System information

• Have I written custom code (as opposed to using zenbot vanilla): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Docker
• Zenbot version (commit ref, or version): d1cac34
• Zenbot branch: unstable
• NodeJS version: from latest docker version from unstable tag on hub.docker
• Python version (when using a python script): from latest docker version from unstable tag on hub.docker
• Exact command to reproduce (include everything): Start new trade (paper or not) without additional parameters (—debug can be used). Settings, see changes to conf-sample.js
• Did I make any changes to conf-sample.js?: Fill in selector as gdax.BTC-EUR and configure the gdax API key.

Describe the problem

Bug found:
When starting a trade (paper or live), most of the time, the bot starts properly, but after a while, this can be minutes or hours, the bot stops trading, and the graphs in the webinterface are not updated anymore. As it seems now, the root cause of the problem seems to be that the time is not updated properly and the system keeps retrieving the last retrieved dataset.

Logs included below.

Logs are from gdax, but it can also be reproduced on bitfinex.

Source code / Error logs

This error log contains multiple runs.
logs.log

[nmagill123]

Hi! This bug pretty much renders the bot useless - trading will not work and the graph won't render nor show info. The time stamp remains the same after executing a trade. I'm assuming the two problems are somehow linked.

There is two pictures attached below illustrating the difference between an older version and the latest version on the branch.

This is very frustrating as the bot will not work at all. I hope this can get resolved quickly.

Thank you.

[dlasher]

(I'm not sure if this is directly related, so if it's not, I'll move it off to a new issue.)

Since the #2425 fix, sims are -way- off on their profitability calc, and I think it has to do with time.

Example:
2020-10-14 09:11:56 11.2063 LINK-USD -0.07% 28 50.66 + 11.2047 -0.0007 0.0101 50.6 sold 0.22999997 LINK 10162.24 USD+836.09%+675.42%

Started this run with $10k currency, finishing with 10,162 is NOT 675% profit.

2020-10-11 04:08:52 6.25370 BAND-USD +0.16% 65 59.08 ++ 6.2204 0.0026 -0.0202 59.0 sold 0.00999995 BAND 10051.99 USD-785.73%-753.60

Started this run with $10k currency, finishing with 10,051 is NOT a 785% loss.

[dlasher]

Another example, fresh from a darwin backtest. This is not right.

{
"commandString": "./zenbot.sh sim gdax.BTC-USD --period_length=1m --min_periods=110 --markdown_buy_pct=0.75 --markup_sell_pct=0.75 --order_type=maker --sell_stop_pct=48.6 --buy_stop_pct=4.4 --profit_stop_enable_pct=1.5 --profit_stop_pct=1.7 --rsi_periods=14 --stoch_periods=27 --stoch_k=1 --stoch_k_ma_type=EMA --stoch_d=4 --stoch_k_sell=15 --stoch_k_buy=25 --stoch_d_ma_type=T3 --bollinger_size=20 --bollinger_updev=1.8 --bollinger_dndev=1.3 --bollinger_dType=TEMA --bollinger_upper_bound_pct=85 --bollinger_lower_bound_pct=41 --backtester_generation=2 --strategy=ta_stoch_bollinger --asset_capital=0 --currency_capital=10000 --runGenerations=5 --buy_pct=99 --sell_pct=99 --quarentine_time=1 --period=1m --max_buy_loss_pct=100 --max_sell_loss_pct=100 --days=5 --maxCores=36 --filename=none",
"queryStart": "2020-10-08T23:04:59.000Z",
"queryEnd": "2020-10-14T16:21:49.748Z",
"iteration": 2,
"startTime": "2020-10-14T16:21:49.749Z",
"currentTimeString": "2020-10-14 09:19:57 ",
"endTime": "2020-10-14T16:26:40.193Z",
"result": {
"params": "module.exports = {"avg_slippage_pct":0.045,"backtester_generation":2,"bollinger_dType":"TEMA","bollinger_dndev":1.3,"bollinger_lower_bound_pct":41,"bollinger_size":20,"bollinger_updev":1.8,"bollinger_upper_bound_pct":85,"buy_stop_pct":4.4,"markdown_buy_pct":0.75,"markup_sell_pct":0.75,"max_buy_loss_pct":"100","max_sell_loss_pct":"100","max_slippage_pct":5,"min_periods":110,"order_poll_time":5000,"order_type":"maker","period":"1m","period_length":"1m","profit_stop_enable_pct":1.5,"profit_stop_pct":1.7,"quarentine_time":1,"rsi_periods":14,"runGenerations":5,"selector":"gdax.BTC-USD","sell_stop_pct":48.6,"show_options":true,"stoch_d":4,"stoch_d_ma_type":"T3","stoch_k":1,"stoch_k_buy":25,"stoch_k_ma_type":"EMA","stoch_k_sell":15,"stoch_periods":27,"strategy":"ta_stoch_bollinger","symmetrical":false,"net_currency":60665.9412525922,"start":"202010081700"}",
"assetPriceLastBuy": "11391.66",
"lastAssestValue": 11338.79,
"profit": 6.080468689790999,
"assetCapital": 10133.9345471456,
"startCapital": 10000,
"endBalance": 70804.68689791,
"buyHold": 10416.540733112426,
"vsBuyHold": 579.7332119369902,
"wins": 1,
"losses": 1,
"errorRate": 0.5,
"days": 5,
"period_length": "1m",
"min_periods": 110,
"markdown_buy_pct": 0.75,
"markup_sell_pct": 0.75,
"order_type": "maker",
"wlRatio": 1,
"roi": 608.047,
"selector": "gdax.BTC-USD",
"strategy": "ta_stoch_bollinger",
"frequency": 0.4
}
}

There's no way you have a 608% profit from 1 win 1 loss, trading BTC, over the last 5 days.

[nmagill123]

No your exactly right - when I ran simulation the numbers were all over the place, often varying by thousands of dollars with a few trades. Also, when I ran the trade sim, I used the --days command, but often it wouldn't run for that exact amount of days....

I can't narrow down the problem exactly but there seems to be something wrong effecting the entire bot and its operations.

[dlasher]

Any love for this issue? All the bots that used to work, all hang, are completely unusable.

[jorisw]

Please have a look at the changes I submitted in #2345 and #2334 and see if reversing any of it fixes your problem. These PRs fixed it for me, but introduced the same problem for others.

Easiest would be to checkout the HEAD to the commit before #2334 was merged.