How to Manage a Portfolio of Trading Strategies (Without Overfitting) — Transcript
Full transcript
- 0:00In the previous videos, we talked about
- 0:02how to build, backtest, and optimize a
- 0:04single strategy, but nobody serious
- 0:06trade a single strategy. Today, we talk
- 0:09about what happens after, how to put
- 0:11many strategies together into one
- 0:13portfolio, why you should diversify, how
- 0:15to make strategies comparable in terms
- 0:18of risk, how to assign weights, why
- 0:20optimizing the portfolio can be just
- 0:22another way to overfit, and a
- 0:24walk-forward technique to check whether
- 0:26your portfolio optimization actually
- 0:28works out of sample. We will start from
- 0:31the theory of diversification, then we
- 0:33move to the most important practical
- 0:35step, which is normalizing the risk for
- 0:37each strategy, then waiting methods, the
- 0:40formula I use in my live portfolio, the
- 0:43overfitting problem, a walk-forward
- 0:45approach to evaluate portfolio
- 0:46optimization, how to balance across
- 0:49instruments and strategy families, and
- 0:51finally some practical problems I faced
- 0:54running this live. Diversification is
- 0:57the only free lunch in finance, and
- 0:59these two formulas show why. If you
- 1:01combine N strategies with the same
- 1:03Sharpe ratio and an average correlation
- 1:05rho, the portfolio volatility goes down
- 1:08while the expected returns stay the
- 1:10same. So, the Sharpe ratio goes up. With
- 1:1310 strategies at Sharpe 0.5 and 0
- 1:16correlation, you get a portfolio Sharpe
- 1:19of about 1.6. But, look at the last row.
- 1:22When N goes to infinity, the Sharpe
- 1:24ratio converges to Sharpe ratio divided
- 1:27by the square root of rho. Correlation
- 1:29sets a ceiling. For example, with
- 1:32correlation 0.2, you can never go above
- 1:34about 1.1. So, the real goal is not more
- 1:38strategies, it's more uncorrelated
- 1:40strategies. This table shows the
- 1:43diversification multiplier, which is how
- 1:45much the Sharpe ratio improves as a
- 1:48function of the number of strategies
- 1:50with correlation 0.2. Going from one to
- 1:52five strategies gives you plus 67%
- 1:56from 10 to 50, only plus 14. Most of the
- 1:59benefit comes from the first 10 to 15
- 2:02different strategies. After that, every
- 2:05new strategy costs you development,
- 2:07porting, and monitoring time, and adds
- 2:10very little. This is why I target 10 to
- 2:1315 strategies live, and why a new
- 2:15strategy that's high correlated to what
- 2:18I already have usually doesn't make it
- 2:21to the portfolio. Before talking about
- 2:23weights, there is a step most people
- 2:25skip. Look at these three equity curves.
- 2:28If you just put them together with equal
- 2:30capital, strategy B with 25% volatility
- 2:34will drive almost all the portfolios ups
- 2:37and downs. You think you're diversified
- 2:40across three strategies, but in terms of
- 2:42risk, you're basically have one. So, the
- 2:45first rule is you can't compare or
- 2:47combine strategies until they have the
- 2:50same risk. And the measure of risk we
- 2:52will use is annualized volatility.
- 2:55Volatility targeting means rescaling
- 2:58each strategy so that stand-alone, it
- 3:00has the same annual volatility. You take
- 3:03the daily returns, compute the standard
- 3:05deviation, annualize it, and compute a
- 3:08scaling factor. Target volatility
- 3:10divided by realized volatility. If my
- 3:13target is 10% and a strategy runs at 20,
- 3:17I trade it half size. If it runs at
- 3:20five, I double it. Use daily returns,
- 3:22not trade returns, so that holding
- 3:25period and trade frequency are already
- 3:27accounted for. Now, how you actually
- 3:30apply the scaling depends on how the
- 3:32strategy sizes positions. So, let's see
- 3:35the two cases. If you use a stop-loss
- 3:37and size every trade as a fixed
- 3:40percentage of equity at risk, the risk
- 3:42per trade is not the same thing as the
- 3:45strategy risk. The annual volatility
- 3:47depends on how many trades you take per
- 3:49year, and how dispersed the outcomes are
- 3:52in our multiples. Two strategies both
- 3:55risking 1% per trade can have a
- 3:57completely different annual volatility.
- 4:00One with 50 trades per year sits around
- 4:0210% one with 200 trades sits around 20%.
- 4:06The formula is only an approximation
- 4:09because it assumes independent trades
- 4:11and ignores overlapping positions. So in
- 4:13practice I do it empirically. Run the
- 4:16back test at 1% risk measure the annual
- 4:18volatility of the equity curve and
- 4:21rescale the risk per trade linearly to
- 4:23hit the target. Returns scale roughly
- 4:26linearly with risk so this works well
- 4:29apart from lot rounding which we will
- 4:32see later. If you don't use stop losses
- 4:34a risk is controlled through position
- 4:36size directly. Here I use a simple
- 4:39discrete signal plus one long minus one
- 4:41short zero is flat. When the signal is
- 4:44on the position notional value is the
- 4:46account equity times the ratio between
- 4:48the target volatility and the
- 4:50instruments volatility. That ratio is
- 4:53your leverage. If the instrument moves
- 4:5520% a year and your target is 10% the
- 4:58notional is half your equity. If it
- 5:01moves 5% a year the notional is twice
- 5:03your equity. Then you convert notional
- 5:06to lots dividing by price times point
- 5:08value which is the value of one lot. Be
- 5:11careful not to confuse the two. Equity
- 5:14is what you have in the account notional
- 5:16is the exposure you get as output. One
- 5:19caveat is that this gives you the target
- 5:22volatility only while you're in a
- 5:24position. If the strategy is in the
- 5:27market say 40% of the time it's realized
- 5:30annual volatility is roughly the target
- 5:33times the square root of 0.4. So about
- 5:3563% of it. So also here the final step
- 5:39is the same. Measure the strategies
- 5:41realized annual volatility and rescale.
- 5:43Both approaches end in the same place.
- 5:46Every strategy at the same annual risk.
- 5:49Once the strategies at the same risk,
- 5:51you need weights. Here are the main
- 5:53families. Equal weight use no
- 5:55information at all. Inverse volatility
- 5:58is pointless once you've done volatility
- 6:00targeting because all volatilities are
- 6:02already equal. Equal risk contribution
- 6:05and minimum variance use the covariance
- 6:07metrics, but not expected returns.
- 6:09Markovitz max sharp uses expected
- 6:12returns, and that's why the trouble
- 6:15begins. Hierarchical risk parity from
- 6:17Marcos Lopez de Prado cluster strategies
- 6:20by correlation and allocates top-down
- 6:23avoiding matrix inversion, so it's more
- 6:25stable. And Carvers and Crafting groups
- 6:27similar strategies and assigns weights
- 6:30by hand to the groups. Moving down the
- 6:32table, you use more information from the
- 6:34data, which sounds better, but every
- 6:37number you estimate is a number you can
- 6:39get wrong. This is what I use in my life
- 6:41portfolio. Every strategy is already at
- 6:44the same target volatility, so equal
- 6:47weight means equal risk. If I simply
- 6:49divided by n, the portfolio volatility
- 6:52would end up well below target because
- 6:54the strategies aren't perfectly
- 6:56correlated. This formula fixes that.
- 6:59It's the weight that brings the whole
- 7:01portfolio exactly back to the target
- 7:03volatility given n and the average
- 7:05correlation. It's mathematically the
- 7:07same as Carver's diversification
- 7:09multiplier divided by n. With 10
- 7:12strategies and average correlation of
- 7:140.2, each strategy gets 18.9 of its
- 7:18stand-alone size instead of 10%. In
- 7:20practice, if a strategy risks 0.5 per
- 7:24trade on its own, inside the portfolio,
- 7:26it risks about 0.095%
- 7:30and the portfolio as a whole runs at 10%
- 7:32volatility. Why not just run an
- 7:34optimizer and maximize the sharp ratio?
- 7:37Well, because portfolio optimization is
- 7:39itself a fit. Classic research showed
- 7:42that errors in the means hurt about 10
- 7:45times more than errors in the variances.
- 7:47With a sharp 0.5 over 10 years of data,
- 7:50the standard error is about 0.34.
- 7:54So, a strategy with backtest sharp 0.5
- 7:57and one with 0.9 are statistically
- 8:01almost indistinguishable. The optimizer
- 8:03doesn't know that. It puts big weights
- 8:06on whatever got lucky in sample. De
- 8:08Miguel and co-authors showed that
- 8:11mean-variance optimization out of sample
- 8:14doesn't consistently beat simple 1/n.
- 8:17Plus, there is a double-dipping problem.
- 8:19You selected these strategies because
- 8:22their backtest was good, and now you're
- 8:24using the same backtest again to weight
- 8:26them. That's a selection bias on top of
- 8:28selection bias. So, how do we know if an
- 8:31optimization method is actually helping?
- 8:34The same way we validate a strategy,
- 8:35walk forward. You take a window of
- 8:37history, estimate the weights with
- 8:39method you want to test, then apply
- 8:41those weights frozen to the next unseen
- 8:43period. Then you roll forward and
- 8:45repeat. At the end, you concatenate all
- 8:47the out-of-sample periods, and you get
- 8:49an honest equity curve of the allocation
- 8:51method. The key is the benchmark.
- 8:54Compare it with the 1/n on exactly the
- 8:57same out-of-sample periods. If your
- 8:59method doesn't beat equal weights out of
- 9:01sample, the optimization is just fitting
- 9:04noise. This idea exists in the
- 9:06literature, is essentially how De Miguel
- 9:08evaluated allocation methods. Also, look
- 9:11at the walk forward efficiency and at
- 9:13how much the weights jump between
- 9:15windows. Unstable weights are a red flag
- 9:17on their own. There is also a trap that
- 9:19makes most portfolio walk forwards
- 9:22useless. If the strategy returns you
- 9:24feed are in-sample backtests, it doesn't
- 9:27matter how carefully you walk forward
- 9:29the weights. The inputs are already
- 9:31overfitted. The fix is to use for each
- 9:33strategy the out-of-sample returns from
- 9:36its own walk forward analysis or live
- 9:39returns. Basically, a walk forward on
- 9:41top on a walk forward. Also, a strategy
- 9:43should only enter the portfolio from the
- 9:46date you could actually have known about
- 9:48it. And remember that if you test six
- 9:51allocation methods and pick the best
- 9:53one, you've just introduced a new
- 9:55selection bias. So, check whether your
- 9:57improvement over equal weight is
- 9:59significant, for example, by
- 10:01bootstrapping the difference in Sharpe
- 10:03ratios. Equal weight has a weakness. It
- 10:06gives equal weight to strategies, not to
- 10:09ideas. If I have eight trend following
- 10:11strategies and two mean reversion ones,
- 10:13equal weight means 80% of my risk is on
- 10:16trend following, just because I happen
- 10:18to develop more of them. The solution is
- 10:21decide a priori the percentage for each
- 10:24strategy family, for example, 40% to
- 10:26trend following, 40% to mean reversion,
- 10:2920% for everything else. Inside each
- 10:32bucket, equal weight. Trend following
- 10:34and mean reversion tend to make money in
- 10:36different market regimes, so they're
- 10:38natural diversifiers. These percentages
- 10:40are not estimated from data, so they
- 10:43can't be over fitted, and that's the
- 10:45point. This is the core idea of Carvers
- 10:47and Crafting. The same logic applies to
- 10:49instruments. You can have 10 different
- 10:51strategies, but if four of them trade
- 10:54the S&P 500, you're much more
- 10:56concentrated than you think. On a bad
- 10:58day for equities, they can lose all
- 11:00together, even if their average
- 11:02correlation looked low. So, I look at
- 11:05the portfolio also by instrument and by
- 11:07asset class, and I put caps. No single
- 11:10instrument or asset class above a
- 11:11certain share of my risk. And part of
- 11:14the job is building new strategy that
- 11:16fill the gaps where you're under
- 11:18weighted, rather than adding a fifth
- 11:20strategy on the same index. The
- 11:23correlation formula has a hidden risk. A
- 11:25lower correlation gives a higher weight
- 11:27per strategy, which means more leverage
- 11:30on each one. That's fine as long as
- 11:32correlations stay low, but correlations
- 11:34are not stable. So, they tend to rise
- 11:37exactly in market stress when everything
- 11:40sells off together. That's when your
- 11:42portfolio volatility can blow past the
- 11:44target. So, I use safeguards. I never
- 11:46let the average correlation go below
- 11:48zero. Carver caps the diversification
- 11:51multiplier at 2.5, and you can use a
- 11:54conservative correlation estimate, for
- 11:56example, a high quantile of the rolling
- 11:58correlation rather than the full sample
- 12:01average. Now, some problems I see in
- 12:03real life. First, when you only have a
- 12:04few strategies live, for example, after
- 12:07pulling for some maintenance, the
- 12:08formula would give each one of them a
- 12:10very large weight. With three
- 12:12strategies, each one carries a lot of
- 12:14risk, and you have no real
- 12:16diversification to justify it. So, I set
- 12:18a minimum n of 10. Until the portfolio
- 12:21is full, it simply runs below its
- 12:23volatility target. Second, lot rounding.
- 12:26When you divide the risk among many
- 12:28strategies, the risk per trade gets very
- 12:30small. And minimum lot of 0.01, some
- 12:33trades run to zero and get skipped or
- 12:36get rounded up and are bigger than
- 12:38intended. Your live results then drift
- 12:41from the backtest. Check what the
- 12:43minimum position means in terms of risk
- 12:45for each instrument given your capital.
- 12:48Third, keep track of which sizing
- 12:50configuration was active for each trade.
- 12:53I encode it with magic number, so I can
- 12:55always reconstruct what was running.
- 12:58Last decision, what volatility target to
- 13:00use. On a personal account, it's about
- 13:02the drawdown you can tolerate
- 13:04emotionally and financially. On a prop
- 13:06firm account, it's driven by the rules,
- 13:09daily loss limit and maximum drawdown.
- 13:11The right way is not to guess, but to
- 13:13bootstrap the portfolio returns and
- 13:15simulate the challenge many times,
- 13:17estimating the probability of passing,
- 13:19the probability of hitting the daily
- 13:21loss limit, and the expected value per
- 13:23attempt. To give you an idea, 10% annual
- 13:26volatility is about 0.63%
- 13:29daily. 20% is about 1.26%.
- 13:33Fat tails and correlation spikes make
- 13:35the bad days worse than a normal
- 13:37distribution suggests, which is why
- 13:39simulation on real returns beats the
- 13:41formula. Now we will go through the full
- 13:44pipeline of managing the portfolio of
- 13:47strategies with this Jupiter notebook,
- 13:49and we will go step by step following
- 13:52the presentation. We start by defining
- 13:54these parameters. So, the number of bars
- 13:58per year, the target volatility of the
- 14:00strategies and the portfolio, the mean
- 14:03Sharpe ratio of each strategy, even
- 14:05though each one will have a different
- 14:07performance, the number of years
- 14:09simulated and the random seed. Then we
- 14:12initialize a basket of strategies from
- 14:15different styles and applied on
- 14:17different asset classes and instruments.
- 14:20So, for example, we have this trend
- 14:21following strategy on the S&P 500 with a
- 14:25win rate of 38% exposure to the market
- 14:28of 70% volatility raw of 14% and true
- 14:32Sharpe ratio of 0.75. And as you can
- 14:35see, each strategy has a different
- 14:37statistical profile in terms of win
- 14:40rate, exposure, volatility, and Sharpe
- 14:42ratio. Now a bit of explanation of how
- 14:45we build the strategies because since we
- 14:47need to know the truth, we need to
- 14:49generate the data ourselves in two
- 14:51steps. First, we generate each strategy
- 14:54on its own. So, every day it's either in
- 14:56the market or flat. If it's in the
- 14:58market, it wins with a given
- 15:00probability, which is the win rate. Then
- 15:02we pick the size of wins and losses so
- 15:05that each strategy is has exactly the
- 15:07Sharpe ratio that was defined for them.
- 15:10And as a second step, we define how they
- 15:13move together. So, we use a correlation
- 15:15matrix. So, strategies on the same
- 15:18instrument, the same asset class, and
- 15:20the same family are more correlated by
- 15:22definition. So, we use a tool called a
- 15:25Gaussian copula, which ties each
- 15:27strategy to the others using that matrix
- 15:30without changing the shape of its
- 15:32returns. Finally, since it's also the
- 15:34case as in real life, we let the
- 15:36correlation drift slowly over time and
- 15:39add a few stress periods when everything
- 15:42moves together to simulate the market
- 15:44crashes. And after we created every
- 15:47strategy profile, we can also see that
- 15:49for example, trend following strategies
- 15:52have a low win rate, but the average win
- 15:54to average loss is greater, and we can
- 15:57also check the skewness, which is
- 15:59positive for trend following and
- 16:01negative for mean reverting ones. Here
- 16:03we can see the distribution of daily
- 16:05returns from different strategies. So,
- 16:08in blue we have the trend following with
- 16:10positive skew and low win rate. In the
- 16:12red, we have the mean reverting strategy
- 16:15with negative skew and high win rate.
- 16:17And in green, we have our carry strategy
- 16:20with win rate about 55%. Here we can see
- 16:23the equity curves of all the strategies,
- 16:26and it's so clear that every strategy
- 16:29has a very different risk profile. For
- 16:31example, this one has a much higher
- 16:33volatility than this one. Here we have
- 16:36the correlation matrix of daily returns
- 16:38over the full sample, and we can see
- 16:41that the strategies within the same
- 16:43family as a higher correlation by
- 16:46construction of ours. And also in
- 16:49respect to the same instruments. For
- 16:51example, here we have the S&P 500 and
- 16:53also here, and we can see the
- 16:55correlation is high even though this is
- 16:57a trend following and this is a
- 16:58seasonality strategy. Here we can see
- 17:01the three-month pairwise correlation for
- 17:04four couples of strategies, and by
- 17:07definition, we have this correlation
- 17:09changing over time slowly. And also, we
- 17:12can see the moments of crisis when all
- 17:16the correlations tend to spike
- 17:18simultaneously.
- 17:19Now we see the formula for calculating
- 17:22the Sharpe ratio of the portfolio that
- 17:24given each strategy having 0.5 Sharpe
- 17:27ratio and a correlation of 0, 0.2, 0.5,
- 17:31we have the same values we obtain in the
- 17:34presentation. So we can see that as long
- 17:36as the correlation of the portfolio is
- 17:38low, the Sharpe ratio goes to infinity,
- 17:41while if the strategies are correlated,
- 17:44the value added is almost none. Here is
- 17:46the plot of that equation. So with
- 17:48highly correlated strategies, the value
- 17:51of adding new strategies is almost zero,
- 17:54while for totally uncorrelated
- 17:56strategies, the Sharpe ratio of the
- 17:58portfolio continues to grow up to
- 18:01infinity. This in particular is our
- 18:03case, so we have an average Sharpe ratio
- 18:06of 0.74 and an average correlation of
- 18:090.15. So we can see that we get the most
- 18:12benefit in the beginning and then the
- 18:14performance tend to stale here. This is
- 18:17the same concept in terms of
- 18:19diversification multiplier. So how much
- 18:22you are gaining in terms of Sharpe ratio
- 18:25of performance by adding new strategies.
- 18:28And as you can see, for correlated
- 18:30strategies, the the gain is zero, while
- 18:34for and the uncorrelated strategies is
- 18:36much higher, but also you can see that
- 18:39after a while the benefit is smoothed.
- 18:42So we get the most value in terms of
- 18:45number of strategies employed around 10
- 18:48to 15. This plot shows the increase in
- 18:51Sharpe ratio of the portfolio given the
- 18:54correlation of the new strategy. So if
- 18:57we have a strategy uncorrelated from the
- 18:59portfolio, we get an 8% gain, while if
- 19:03the strategy is very much correlated, we
- 19:05get a diminishing Sharpe ratio of minus
- 19:089%. Here we see three strategies from
- 19:11the portfolio and we can immediately
- 19:14notice that this trend following
- 19:15strategy on gold is way more volatile
- 19:18than the others. Indeed, it has a annual
- 19:21volatility of 25% and if we look the
- 19:24strategies in terms of risk, we can also
- 19:27see that 70% of the risk is allocated in
- 19:31this strategy. So, to compare different
- 19:33strategies, we need first to standardize
- 19:36each one of them to the same level of
- 19:39annual volatility and you can clearly
- 19:41see the difference once we do that
- 19:44because now all the strategies are on
- 19:46the same level of risk, which is also
- 19:49confirmed by this plot. So, they're all
- 19:52around 35%
- 19:54of risk and now we will dive through
- 19:56each case for volatility targeting your
- 20:00strategy. So, the first case is using a
- 20:02stop loss per trade. So, in this case,
- 20:05the annual volatility is the risk per
- 20:07trade multiplied the standard deviation
- 20:10of the R multiple multiplied the square
- 20:12root of the number of trades per year.
- 20:14So, this means that two strategies
- 20:17having the same risk per trade, if one
- 20:19trades way more often than the other, it
- 20:22will have a much higher volatility over
- 20:25the year and that's why it will have a
- 20:27lower risk per trade once it's scaled
- 20:30and we can see effectively once the
- 20:33scale is applied, the two strategies are
- 20:36almost identical. So, in this case, the
- 20:39first strategy trades 50 times per year
- 20:41with a 10% volatility. The second one
- 20:44trades 200 times per year with a 20.9%
- 20:48volatility. So, the scale the risk per
- 20:51trade becomes 0.93%
- 20:54for the first strategy and for the
- 20:56second one is 0.48 and as you can see,
- 20:59the final volatility is 10%. Instead, if
- 21:03you use no stop loss, you have to size
- 21:05your positions according to the market
- 21:07volatility. So, assuming that your
- 21:09signal is either -1, 0, or 1, so it's
- 21:13binary, but you can also do that with
- 21:16continuous signals. Your notional, so
- 21:18the value of the position, is your
- 21:20signal times the equity times the
- 21:23leverage, which is the ratio between the
- 21:25volatility target and the volatility of
- 21:28the instrument, and the number of lots
- 21:30will be the notional divided by the
- 21:32price of the contract, and you can
- 21:34estimate this instrument volatility
- 21:37using a rolling estimate of the
- 21:39volatility over the instrument. Here you
- 21:42can see what happens when you don't
- 21:44scale your positions according to
- 21:46volatility. So, this is the volatility
- 21:48of the underlying instrument, and this
- 21:51is the volatility of your strategy when
- 21:54you have a fixed notional. So, it will
- 21:56follow precisely the market volatility,
- 21:59while if you size the positions
- 22:01according to an estimate of market
- 22:03volatility, you will be always around
- 22:06your volatility target or lower, and I
- 22:09say lower because in this case you are
- 22:12exposed 67% of the time. So, our actual
- 22:16volatility becomes 8.2% instead of 10%.
- 22:20So, again, after we sized according to
- 22:24volatility, we have to rescale back
- 22:26according to our exposure. So, using a
- 22:29fixed notional, we have the same
- 22:31volatility of the market, while with
- 22:33volatility scaled we have less than 10%,
- 22:36so 8.3%, which is this one given this
- 22:40exposure, but once we scaled back, we go
- 22:43to our target of 10% volatility. Now,
- 22:46we'll go through the main methods for
- 22:48weighting the portfolio. So, we have the
- 22:50equal weights, the inverse volatility,
- 22:53the equal risk contribution, the minimum
- 22:55variance, the max sharp, and the
- 22:57hierarchical risk parity. We can see
- 22:59here the weights over the whole
- 23:02portfolio given different waiting
- 23:04methods and over the full time sample.
- 23:07So, what we see is that most methods are
- 23:11balanced except for the max sharp ratio
- 23:14which tends to concentrate all the
- 23:17power, all the weights into the most
- 23:20proficient strategies. And if we look at
- 23:23the detail of the weights here, of
- 23:25course equal weight and inverse
- 23:27volatility are constant about 8% the ERC
- 23:31is very conservative in the delta of
- 23:35weights. The mean variance as some
- 23:37weights close to zero. For example, this
- 23:40trend following strategy gets a 0.8
- 23:43weight which is almost none, but no
- 23:45weight is extremely high. While for the
- 23:48max sharp ratio, many strategies gets a
- 23:52weight around zero and a few strategies
- 23:54get a weight around 20 to 30%. And
- 23:58finally, the hierarchical risk parity is
- 24:02also balanced just like ERC. This is a
- 24:04very important plot in finance and it's
- 24:07called the efficient frontier. It shows
- 24:09the annualized volatility against the
- 24:13annualized return. So, here we have all
- 24:15our single strategies operating at a
- 24:19volatility of 10% given different rate
- 24:22of returns because they have different
- 24:24sharp ratios. And here we have the
- 24:26performance of the various kinds of
- 24:28portfolio techniques and this clearly
- 24:31shows that the portfolio aggregation
- 24:34maintains the same level of return, but
- 24:37minimizes a lot the variance or
- 24:40annualized volatility of the returns.
- 24:43Here we can see the zoom into this
- 24:46cluster and we can see that all these
- 24:49methods are very similar and close to
- 24:52each other in this region while the max
- 24:55sharp ratio wants a higher return. So,
- 24:58it's more distant in the plot from the
- 25:00others. And we can also notice that the
- 25:03one And of course, this is the
- 25:06performance in sample. So, we know
- 25:08nothing about the performance out of
- 25:10sample of this portfolio configuration.
- 25:14This is the formula I use for my
- 25:16waiting. So, it uses both the number of
- 25:18strategies and the average correlation
- 25:21of the portfolio. And it's equal to the
- 25:23diversification multiplier divided by
- 25:25the number of strategies. For example,
- 25:28we have a 12 strategies with average
- 25:30correlation of 0.15. The weight becomes
- 25:330.17,
- 25:34while the equal weight would be just
- 25:370.08. So, if the risk per trade on a
- 25:40singular level is 0.5% stand alone, it
- 25:43becomes 0.088
- 25:46in the portfolio. And we can see here
- 25:48just the equal weight and the equal
- 25:50weight plus correction for the
- 25:52correlation. In particular, if we just
- 25:55scale according to the number of assets,
- 25:57the volatility is reduced a lot, while
- 26:00we want to go back to our target of 10%
- 26:04volatility. Now, we see why optimizing
- 26:06the portfolio overfits. So, this is the
- 26:08formula for the standard error of the
- 26:11Sharpe ratio. And from this, we can
- 26:13calculate the standard error of our 0.5
- 26:16Sharpe ratio over 10 years, which is
- 26:190.34.
- 26:21So, basically, the 95% confidence
- 26:23intervals are something between minus
- 26:260.1 and 1.1. So, even using 10 years of
- 26:31data, the edge is not still
- 26:34statistically indistinguishable from
- 26:37noise. And if we just look at 3 years of
- 26:40data, the standard error is 0.63.
- 26:44So, this could be easily be zero, but
- 26:48the optimizer see that the strategy is
- 26:52profitable, okay? So, the the error is
- 26:55huge, but the optimizer sees that the
- 26:58strategy is profitable and allocates a
- 27:01greater weight to the strategy. And
- 27:03also, we can see that even though the
- 27:05strategies have a true Sharpe ratio
- 27:08between 0.2 and 1 by construction, if we
- 27:11estimate the performance on a yearly
- 27:13basis, the variance is huge because we
- 27:16have estimates between -1.34
- 27:20up to 2.4, 2.6, 3, -2. So, the
- 27:26optimizer, using just a few years of
- 27:29data, it's very difficult to tell real
- 27:32edge from just luck on that specific
- 27:35year or in-sample fault. And that's why
- 27:38it's so easy to overfit the optimization
- 27:42of the portfolio. And if we use the max
- 27:44Sharpe ratio method, we can see that the
- 27:47weights change enormously from one fold
- 27:50to the other. But what changed actually
- 27:53was just the estimate of the performance
- 27:56of the strategies because they had
- 27:58constant Sharpe ratio by construction.
- 28:00So, the optimizer just see the noise in
- 28:03the estimate and changes almost
- 28:06completely the weights from one fold to
- 28:08the other. Here, you can see the true
- 28:10Sharpe ratio of the strategy against the
- 28:14average weight picked by the optimizer.
- 28:17And we can see that there are a lot of
- 28:19very good strategies with high Sharpe
- 28:21ratios, which get very low weights on
- 28:24certain folds just because the optimizer
- 28:27saw the noise and not the signal. So, a
- 28:30good way to estimate the efficiency of
- 28:33your portfolio optimization is to use
- 28:36walk-forward analysis. So, we will use a
- 28:383-year in-sample window, estimate the
- 28:41weights using each method, and evaluate
- 28:44out of sample for the next year.
- 28:46Finally, we will concatenate all these
- 28:49out of sample returns and compare all
- 28:52the methods against the equal weight.
- 28:54And we will use as metrics the out of
- 28:56sample sharp, the in sample sharp, the
- 28:59efficiency, which is just the ratio
- 29:01between the out of sample and in sample,
- 29:04and the turnover, which is the change of
- 29:07the weights over time. Here is the
- 29:09result. So, the max sharp, which wants
- 29:12to optimize, actually performed worse,
- 29:15while the equal weight performed around
- 29:18the middle of all the performers. And
- 29:21the best one was actually the mean
- 29:23variance method. The results are in this
- 29:25table. So, the equal weight has an
- 29:28efficiency of around one, having done no
- 29:31optimization at all. Inverse volatility
- 29:34and ERC about the same efficiency. The
- 29:37mean variance was the best with 1.06
- 29:40efficiency. So, the out of sample was
- 29:43even better than the in sample on
- 29:45average. The max sharp was the worst,
- 29:48absolute worst, having a 0.59
- 29:50efficiency. And the hierarchical risk
- 29:53parity with a good performance of 1.04.
- 29:56Also, we can see the turnover of
- 29:58weights, which is kind of low, except
- 30:00from the mean variance and the max sharp
- 30:03ratio. Now, to understand if this
- 30:06performance is real or not, we bootstrap
- 30:09the delta increments in sharp ratio
- 30:12against the equal weight method. And
- 30:15what we get here is the max sharp method
- 30:19in green. The dotted line is the equal
- 30:22weight. This is the ERC, and this is the
- 30:25mean variance method. We can see the
- 30:27numbers in this table. So, the three
- 30:30methods based on covariances, which is
- 30:32the HRP, the ERC, and the mean variance,
- 30:35are statistically superior in this
- 30:38simulation compared to to equal weight,
- 30:40while the max sharp is inferior to the
- 30:43equal weight method, but not as
- 30:45statistically significant as the others.
- 30:49Here we can also see the stability of
- 30:51our increment against the equal weight
- 30:55over time, and we can see that the ERC,
- 30:59mean variance, and HRP are above the
- 31:02equal weight about 80 to 90% of the
- 31:05time, even though we saw the increment
- 31:08is not huge, while the max sharp is
- 31:11above the equal weight just 39% of the
- 31:14time. So, it's very unstable how it
- 31:16compares against the equal weight. Here
- 31:19we see the out-of-sample sharp ratio by
- 31:21calendar year and how many years the
- 31:24methods beat the equal weight. So, for
- 31:27inverse volatility is nine, ERC is nine,
- 31:30mean variance is nine, the HRP is eight,
- 31:33and the max sharp is four out of 10
- 31:35years. So, in conclusion, given our
- 31:38synthetic data, which has slowly
- 31:41changing correlation over time, the max
- 31:43sharp ratio method has the highest
- 31:46in-sample performance and the worst
- 31:48out-of-sample performance, while the
- 31:50covariance-based methods, so the ERC,
- 31:54mean variance, and HRP are significantly
- 31:57superior than the equal weight method,
- 32:00and the inverse volatility is just
- 32:02slightly different because the actual
- 32:05volatility is not exactly 10% per year.
- 32:09This means that these three methods
- 32:11efficiently estimate the correlation in
- 32:14the in-sample windows and exploit it in
- 32:17the out-of-sample test. This result,
- 32:19however, is not guaranteed in real life
- 32:22using your own strategies and given
- 32:25their rolling correlation over time. In
- 32:27this experiment, we create 40 strategies
- 32:30without any edge, and in the first case,
- 32:33we select 10 of them looking at the
- 32:35whole sample and running the equal
- 32:38weight portfolio on them. And as you can
- 32:40see, the expected sharp ratio is 0.81,
- 32:45even though the strategies have no edge.
- 32:47While in this case, we only select the
- 32:50strategies using the in-sample fold and
- 32:53evaluate the performance out of sample.
- 32:55And in this case, the performance is
- 32:57zero, of course. So, this clearly shows
- 33:00how selecting based on the full sample
- 33:04can inflate your results. So, the
- 33:06walk-forward analysis is correct, but
- 33:08the inputs are totally wrong. So, here
- 33:10we can see how we balance the risk
- 33:13between different strategy types. So, we
- 33:16can see that using equal weight, we have
- 33:18a higher risk allocated to trend
- 33:20following just because I have more trend
- 33:22following strategies in my portfolio.
- 33:24While I would prefer to allocate certain
- 33:27buckets of risk towards certain strategy
- 33:30families. So, for example, 40% to mean
- 33:34reversion, 40% to trend following, and
- 33:3620% to the other. So, they are more
- 33:39balanced independently on the number of
- 33:42strategies within each bucket. In this
- 33:44case, you define the percentage of total
- 33:48risk of the portfolio and then apply the
- 33:50equal weight or optimization method to
- 33:53that specific bucket of strategies which
- 33:56share the percentage you decided upon
- 33:59the total portfolio. Balancing the risk
- 34:02among instruments and asset classes is
- 34:05basically the same thing. So, we can see
- 34:07that if we have four strategies on the
- 34:09S&P 500, the risk allocated in this
- 34:12instrument is way higher than the other
- 34:15instruments. And so, the risk by asset
- 34:18class in this case for indices is too
- 34:21large. So, in this case, what we want to
- 34:24do is just fix a cap of share for each
- 34:28asset class and then rebalance all the
- 34:30others according to the margin left. So,
- 34:33for example, in this case, we want to
- 34:35cut the risk at 25% and then split this
- 34:39amount of risk within the asset classes
- 34:42that have less, so metals, agriculture,
- 34:45and energy.
- 34:46Another very important factor to
- 34:48consider is that average correlation of
- 34:51your portfolio and strategies changes
- 34:54over time. So, the problem is that when
- 34:56there are market crashes or crises, the
- 35:00average correlation tend to spike, and
- 35:02that's the moment when we most need the
- 35:05diversified portfolio. So, in this case,
- 35:08what do we do? We measure the volatility
- 35:11of the portfolio using a conservative
- 35:13estimate. So, for example, we measure
- 35:15the rolling volatility over 6 months, 1
- 35:19month, and then take the 90 or 95% of
- 35:24that as a conservative value. So, we can
- 35:27see that if we use the standard one,
- 35:29which is the average correlation, which
- 35:31is 0.15, the diversification multiplier
- 35:35becomes 2.11, and the volatility during
- 35:39stress periods becomes 0.16,
- 35:43so almost the double as what we have
- 35:45normally. While using a conservative
- 35:48value of raw, which is 0.28,
- 35:51so it's almost the double of the
- 35:54average, we have a diversification
- 35:56multiplier of 1.7, and the volatility in
- 36:00a stress period becomes 0.13.
- 36:03And we can see, for example, in this
- 36:05period of crash, our rolling volatility
- 36:08of the portfolio is way lower than what
- 36:12we would have using the average
- 36:13correlation. Two other important factors
- 36:16to consider are lot rounding and number
- 36:19of strategies. So, if you have a low
- 36:22number of strategies, the weight per
- 36:24strategy would be very high, And this
- 36:27means that anything that goes wrong with
- 36:30the strategy, being that alpha decay
- 36:33execution, it will have a drastic effect
- 36:36on your portfolio. And this is very
- 36:38dangerous to me. That's why I decided to
- 36:41use an effective N, which is the maximum
- 36:45between the number of strategies used
- 36:48and 10 or 15. You can decide the number.
- 36:51So, if my portfolio just have three to
- 36:54four strategies, the risk will be kept
- 36:57low and just when the portfolio is full,
- 37:01the volatility will become the target.
- 37:04So, for example, 10%. Before that, I
- 37:07will just have a few strategies with
- 37:09lower risk, but also lower volatility of
- 37:12the portfolio. And this is done to
- 37:14protect me against any kind of problem
- 37:17regarding the single strategies.
- 37:20Another problem is lot rounding. So, the
- 37:24more strategies you have, the lower the
- 37:27risk per trade per strategy. And this
- 37:29means that, especially if you have a
- 37:32small account, the lot size will be
- 37:35rounded down to the minimum value or
- 37:38even become zero. So, all the trades
- 37:41will be skipped just because you don't
- 37:43have enough capital to trade. And we can
- 37:45simulate this, for example, using the
- 37:49risk per trade of 0.5%
- 37:51a correlation of 0.2, a cost per trade
- 37:55of 1,000 euro or dollar, and a stop
- 37:59distance of 0.4 ATR. This means that,
- 38:02given two accounts, one of 10,000 and
- 38:05one of 100,000, with 10 strategies,
- 38:08almost 47% of the trades are skipped on
- 38:13the smaller account. And with 30
- 38:16strategies, all the trades are skipped.
- 38:19While for the bigger account, in all
- 38:22these situations, no trade is skipped.
- 38:25Just because the the account is so
- 38:27large, you can trade small and so having
- 38:31much more strategies. If I run this with
- 38:34a lower risk per trade, also the bigger
- 38:38account will start to having problems
- 38:40because you're risking so little per
- 38:42trade, you have so many strategies, the
- 38:45lot size tend to zero. So, you also have
- 38:49to balance this problem when deciding
- 38:51the number of strategies. Finally,
- 38:54defining your target volatility, which
- 38:57is the first step you need to do for
- 38:59balancing your portfolio. And this
- 39:01depends on your situation. So, if it's
- 39:05your personal account, it's just
- 39:07dependent on your tolerance on drawdowns
- 39:10and volatility, while if you're trading
- 39:12on a prop firm accounts, this is defined
- 39:15clearly by the prop firm rules. So, I
- 39:18already made a video on how to estimate
- 39:21the probability of passing prop firms
- 39:23challenges. I will leave the link in
- 39:25description, but in this case, we will
- 39:27use a simplified function for simulating
- 39:30challenges. And we can see here, for
- 39:33example, we are estimating our
- 39:35probability of passing a challenge
- 39:38having a 10% profit target, 10% loss,
- 39:42and 120 days for our time limit. And we
- 39:46can see that having a 5% annual
- 39:49volatility gives us a 4% probability of
- 39:52passing the challenge within 120 days,
- 39:56which is 6 months, while having 10%
- 40:00increases the probability to 46, and 15%
- 40:03increases to 68. But, if you increase
- 40:06too much, of course, the probability of
- 40:08breaching the max loss or daily loss
- 40:11will be also higher, so the probability
- 40:14of passing will be lower. So, you need
- 40:17to run these simulations given your
- 40:20constraints and ultimately defining your
- 40:23optimal annual volatility. To recap,
- 40:25diversification works, but correlation
- 40:28sets the ceiling. So, look for generally
- 40:31different strategies. Before weighting
- 40:33anything, normalize risk with volatility
- 40:35targeting. Equal weight with the
- 40:37correlation formula is simple, robust,
- 40:39and keeps the portfolio at the target
- 40:41volatility. If you want to use an
- 40:43optimizer, prove it beats one over n out
- 40:45of sample with a walk forward and using
- 40:48out of sample strategy returns as input.
- 40:51Balance strategy families and
- 40:53instruments with buckets decided in
- 40:55advance and protect yourself from
- 40:57unstable correlations and the practical
- 40:59issues of live execution. The notebook
- 41:02with the volatility targeting waiting
- 41:04formula and portfolio walk forward is on
- 41:06GitHub, so I will put the link into
- 41:08description. Let me know in the comments
- 41:10how you size your strategies, equal
- 41:12weight, risk parity, something else. And
- 41:15if you found this video useful,
- 41:17subscribe and share with your friends so
- 41:19I can keep making more videos like this
- 41:21one. See you in the next one.
About this transcript
This page contains the full transcript of How to Manage a Portfolio of Trading Strategies (Without Overfitting) by SharpeEdge, generated from the public captions YouTube serves with the video. The transcript has 6,253 words across 947 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.