Position and transaction data is wrong if the same security is used in multiple sub-strategies

[MDDietz1]

Hi all,
I have just been hunting down an issue where positioning data (and transactions calculated from the positions) did not agree with target weights. For example, my strategy showed a positive weight (get_security_weights) but a negative position (or vice versa), each of these cases would imply negative prices which I do not have.

As it turns out, this only happens with more complicated tree-like strategies with nodes and sub-strategies that use the same security in more than one of the underlying strategies. The below code to extract aggregate positions (sum across all strategies) fails in this case - the problem is that the dictionary only holds the position data from one sub-strategy. If the same security is included multiple time in self.securities then all but one of them are ignored.

bt/bt/core.py

Line 1096 in abf3269

positions = pd.DataFrame({x.name: x.positions for x in self.securities})

bt/bt/core.py

Lines 543 to 545 in abf3269

	vals = pd.DataFrame(
	{x.name: x.positions for x in self.members if isinstance(x, SecurityBase)}
	)

Please note that self.securities looks as follows, so securities held in different strategies are shown separately, but for position or transaction data, the strategy level is removed and only securities are shown (as expected):

[<Security TopLevelStrategy>Strategy 1>Security 1>,
 <Security TopLevelStrategy>Strategy 1>Security 2>,
 <Security TopLevelStrategy>Strategy 2>Security 1>,
 <Security TopLevelStrategy>Strategy 2>Security 2>,
 ...]

For comparison, the calculation of total fund value and then weights is robust and checks if there are duplicate names, see here:

bt/bt/backtest.py

Lines 313 to 316 in abf3269

	if m.name in vals:
	vals[m.name] += m_values
	else:
	vals[m.name] = m_values

So the aggregation of positions needs to be done in a similar way: check first if the security is already in the DataFrame. If not, add a column. If yes, aggregate the values.

pos=pd.DataFrame()
for x in self.securities:
    if x.name in pos.columns:
        pos[x.name]+=x.positions
    else:
        pos[x.name]=x.positions

The above code fixes the problems for me, but please let me know if you have any other ideas or suggestions?

Many thanks
Martin

[timkpaine]

Thanks for the detailed report. The solution proposed looks fine to me, would you like to do a PR with it? Otherwise I will take a look in the next week or two

[MDDietz1]

OK, I will aim to send that through later this week. I've had a look through the rest of the file and there is one other place where the same fix seems required:

bt/bt/core.py

Line 532 in abf3269

return pd.DataFrame({x.name: x.outlays for x in self.securities})