YT YouTube2Text

YouTube2Text → Transcripts → How I Decide If a Strategy Is Ready — Not Just a Good Backtest

How I Decide If a Strategy Is Ready — Not Just a Good Backtest — Transcript

by Algo-trading with Saleh · 4,512 words · 618 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00There are two questions that every algo
0:02trader needs to answer before going live
0:04with their strategy. Number one, is my
0:06strategy's results actually good enough
0:08to bother with? And that includes
0:10metrics such as the P&L, max drawdown,
0:12or the win rate. And second question,
0:14and this is the one that most people
0:16fool themselves with, is that are the
0:17results that I'm seeing actually real?
0:20Or has the strategy just overfit itself
0:22on historical data? Which means even
0:24though the results look good on the back
0:26test, they are not going to be as good
0:28on the actual live trading session. In
0:30this video, I will show you how I
0:32personally answer both of these
0:33questions. Now, before we continue, I
0:35got to say I am not a financial advisor,
0:37and this video is for educational
0:39purposes only. So, with that out of the
0:41way, let's get right into it.
0:44To begin, I'm going to show you the
0:45results of a strategy that I recently
0:47found good enough in order to run as
0:50live. So, let's begin by backtesting it.
0:51Now, for the exchange, I choose Binance
0:53Perpetual Futures, even though in live
0:55that's not actually where I trade, but
0:57Binance Perpetual Futures usually offers
1:00really high-quality data for
1:01backtesting. And for the route, I'm
1:03choosing SOL/USDT because that's the
1:06route that I optimized the strategy for,
1:09which is really another topic that we
1:10need to discuss. And that is you don't
1:12need one a strategy to perform well in
1:15all the markets and symbols. It's
1:17perfectly fine to optimize your strategy
1:19for one a specific symbol. In fact, in
1:21my experience, that's what happens all
1:23the time. So, I never found a one a
1:25strategy that performs well everywhere.
1:28And this is the name of the strategy.
1:29And in case you're wondering which
1:30strategy this is, well, if you go to our
1:33website, this is actually the strategy
1:36that I'm talking about, and I even made
1:37a video about it like a year ago. And
1:39not only that, the optimization session
1:41that I run, I also ran like months ago,
1:43and the fact that it is still performing
1:45well gives me a lot of confidence in the
1:47strategy itself. But we don't always
1:49have the luxury of forward testing our
1:51strategy, right? Because when you
1:53develop it, in that moment, you don't
1:55have access to the future data, right?
1:57And that's really the point of this
1:59video. We want to see how we can
2:00evaluate it in the moment. Anyways, as
2:02for the duration, I've picked since the
2:05beginning of 2022 up until just this
2:08month, and I've enabled the fast one and
2:10the benchmark feature. But before I show
2:11you the result for it, first I want to
2:13show you the result for the past
2:15approximately 1 year, and here it is.
2:17And as you can see, the strategy is
2:19performing well, even though it hasn't
2:21seen the data for this date. Now, this
2:23is the equity curve of our portfolio
2:25against the sole USDT itself. This is
2:28the log scale of the same chart. And
2:30this is the drawdown, and this is the
2:33monthly return. But anyways, so it is
2:35producing 43%. All right, now let's move
2:37on to the longer back test. Here's the
2:39result for it. You see, since the
2:41beginning of 2022 up until this month,
2:44and you can say that it is relatively
2:46upward like all the time, which is good.
2:49Now, this is the log scale version. The
2:51PNL was 722%,
2:53which is great. This is how much trading
2:55fees we are paying. This is the max
2:57drawdown, which is minus 17%. The annual
2:59return is 62%. The mean rate is 53%, and
3:03the sharp ratio is 1.76.
3:06And here's the monthly returns. But
3:07anyways, I want to show you the first
3:09thing that I do. So, you see, the
3:11trading fees that we are paying here is
3:13what is the default number for taker
3:16fees on Binance. Now, firstly, I'm not
3:18going to trade this on Binance, but
3:20where I am going to trade it, the
3:21trading fees are approximately the same
3:22number. First of all, you can see that
3:24we are we are only taking into account
3:26the taker fees, not the maker fees,
3:28which are usually lower. And yes, the
3:30strategy is using the limit order at
3:32some point. So, that means we're going
3:34to pay a little bit less fees in the
3:36actual environment. And that's great.
3:38But you see, I'm not going to lower this
3:40number to make it more real just to
3:43consider the maker fees. In fact, not
3:46only I don't do that, what I like to do,
3:48and you don't have to do this, but this
3:50is what I like to do, is to double the
3:52size of my trading fees. And you might
3:55be wondering, why is he doing this?
3:56Well, you see, in the actual
3:58environment, yes, the trading fees are
4:00going to be a little bit less because of
4:01the limit orders. But something else
4:03that is also going to happen is the
4:05slippage. That means when we use a
4:07market order, which the strategy is
4:09using some part of it, the price that we
4:11ask for is not going to be exactly the
4:13price that we actually get to fill. And
4:15this is a real problem in live trading
4:17environments, which you cannot really
4:19see in the back test. And to adjust for
4:20that, what I do is that I double the
4:22size of my fees. Now, doubling it is a
4:25bit too much, but I like doing this
4:27because if my strategy is performing
4:29good enough even with double the fees,
4:32it gives me really good confidence when
4:34I want to run it live. So, that's one
4:35thing that I do. And if I go back and
4:38run the same back test, but this time
4:39with a higher fees, this is going to be
4:42the result. So, you see, instead of
4:43722%,
4:45we're getting 470%,
4:48which is significantly lower. But when I
4:50look at this equity curve, what I see is
4:52that it is a still good enough for me.
4:55Now, you might be wondering, okay, what
4:56is good enough? Well, a max drawdown of
4:58minus 20% is good enough for me because
5:01my risk tolerance allows me up to minus
5:0330% in my entire portfolio to be down.
5:07For you, this number might be different.
5:09So, you have to consider that first.
5:10Next, if we look at the win rate, it is
5:1253%. So, that means approximately half
5:15of the times my trades are going to
5:17lose. So, this is also something that
5:19you have to consider yourself. Like,
5:21what is your risk tolerance? What is
5:23your mental tolerance in this case?
5:25Because if, for example, the win rate of
5:27the strategies is 30%, that means you're
5:29going to lose seven out of 10 times. So,
5:32are you ready to take that mentally or
5:34are you just going to accept everything
5:36and just quit trading? So, this is
5:38important. And for some people, 53% is
5:41low,
5:41but like 70% is enough. So, you have to
5:43consider that. But in In
5:45if you just want to be profitable, the
5:47formula is this. You shouldn't only
5:49consider the win rate by itself. You
5:51should combine that number with another
5:53number, which we call it the average win
5:56to loss ratio, or some people call it
5:58simply the R. And in this strategy, it
6:00is 1.07.
6:02So, that means it is above one, and the
6:05win rate is 53%. So, here's what it
6:08means. If the average win to loss ratio
6:10in our strategy was 0.5, we would have
6:13needed at least 67% in order to break
6:16even. And after that, everything would
6:17have been profit. But in our case, it's
6:19actually slightly more than one, which
6:20means we would have needed at least 50%
6:22of win rate in order to break even. So,
6:24that extra 3 to 4% of win rate that we
6:27have is the edge of our strategy. All
6:29right, the next thing that I'd like to
6:30take a look at is the average holding
6:32time of the strategy. And in this case,
6:34it is 10 hours. Now, why am I pointing
6:37to this? Well, because some strategies
6:39can give you good numbers, but if the
6:41average holding time for it is, let's
6:43say, like a few days or a few weeks,
6:45first going to have to also consider the
6:47funding fees if you are trading futures.
6:49And second, you have to again consider
6:51your mental tolerance. What does it
6:54mean? It means that if I'm going to have
6:56to wait for a few days just for a losing
6:58trade, it's going to be hard for me
7:00mentally. Someone else might be
7:02perfectly comfortable with that. Or
7:04maybe this 10 hours here is too much for
7:07some people. Maybe they are comfortable
7:09with just a few minutes. So, this again
7:11is something that you have to decide for
7:12yourself. But for me, 10 hours on
7:15average for each trade is a good number.
7:17The Sharpe ratio of 1. almost 5 is
7:20pretty good for me. Generally speaking,
7:22anything above one, I would consider it
7:23good enough. But I don't really look at
7:26the Sharpe ratio. I usually look at the
7:28win rate and the average win to loss. If
7:30both of those two things are good, then
7:32I'm going to be happy. Another metric
7:34that actually combines the two of these
7:36two is the expectancy of the trade. So,
7:38you see, that's going to be 0.93%
7:42per each trade, which is also a positive
7:44number. And also the annual return of
7:46the strategy is about 50%. Now, some
7:49people want to like double or triple
7:51their entire capital in just 1 year. And
7:54even though that is actually possible
7:56sometimes, especially if you are in a
7:57bull market or something, I never
7:59actually target for that. Because even
8:0150% is actually a good enough number for
8:04me to continue trading, especially if
8:06it's going to be automated, which is the
8:08case for us algo traders. And when you
8:10compound 50% a year, you actually get a
8:12huge number over the years. But we have
8:14to also consider that this is the
8:16average over multiple years. So, if I go
8:19and take a look at this month's return
8:20heat map chart, which also shows us the
8:23return over that entire year, you see on
8:25the first year it produced 47%.
8:28On the next year it was 137%,
8:32which means you were going to make your
8:33money more than double, which is of
8:35course like awesome. But on the third
8:37year, on 2024, I was actually losing
8:411.5%
8:42and on the next year it was 30% again.
8:44So, that means this 50% number that we
8:47are getting here is actually the average
8:49and not what we are going to get every
8:52year. So, the numbers are going to be
8:53significantly different. So, over this
8:56month it was 9.5%
8:58but then it was minus 7%, 3.6, 2%, 3.5,
9:0229% and again a losing month, right? So,
9:06and this is for a good year. So, even
9:09for a good year, we are not going to be
9:11profitable every single month. And in
9:13this year, for example, which we made a
9:16really good number,
9:18over 3 months in a row, the strategy was
9:21losing money. So, now imagine if you
9:23actually assorted trading the strategy
9:25here and over 3 months in a row, your
9:28strategy was losing money. Are you going
9:31to stop trading it? Well, there's a high
9:33chance. Most people actually do that.
9:35But in reality, if you run a backtest,
9:37for example, you're going to see that
9:39this was actually an expected outcome of
9:41the strategy. And in some other years,
9:43it was even worse. But you see, minus 5%
9:46or 6%, this is not something that's
9:48going to make me go broke. And that's
9:50because of the position sizing that I'm
9:52doing in my strategy and the max
9:54drawdown that I am preparing for. Now,
9:57this brings me to another lesson that is
9:59really important. So, if we come up and
10:01look at the equity curve of the
10:03strategy, actually let's look at this
10:04chart. So, you see this is the drawdown
10:07chart and it also highlights the five
10:09worst drawdown periods. Now, we had here
10:12minus 16%, we had here minus 20%, minus
10:1618%. So, this one was supposedly the
10:18worst one if we consider it
10:20percentage-wise. But in my eyes, this is
10:23actually the worst one. Not because we
10:25lost the most amount of money, but but
10:27because it took the longest time. And if
10:30we go and read it here, you see the max
10:32underwater period, it was 3 5 4 days.
10:36So, that's almost a year, right? So,
10:38that means if I was just trading this
10:40one single strategy on my portfolio, and
10:43if someone looked at my trading results,
10:45they would have called me a really bad
10:46trader. But if you remember, I said this
10:49to me is a good enough strategy for
10:51trading. So, why is that and how do I
10:53look at it? Well, here's the thing.
10:55Again, look at this chart. You see here,
10:57my portfolio is almost flat. Here it's
11:00going down and here again it is almost
11:02flat, right? But in other times, it is
11:04going up, which are actually most of the
11:06times. Now, here's why I actually like
11:08this strategy. Because in reality, I'm
11:11not going to be trading just this one
11:12single strategy. What I want is a basket
11:15of multiple strategies that are going to
11:18perform well in different parts of the
11:21market. Now, what does it mean? It means
11:23that okay, this strategy is performing
11:25well here and that's awesome, but not so
11:27much here, right? So, after this
11:29strategy, I'm going to find another
11:31strategy that will trade well during
11:34this period. And if it doesn't crush it
11:37during this period, that is fine because
11:39this is strategy is. Now, in this
11:41example, I'm talking about two
11:43strategies that are covering each other.
11:45Now, imagine instead of two, you had
11:47like 10 or 20. Then, everything,
11:49including the times that the portfolio
11:52is going up or the times that the
11:54portfolio is going down or it is flat,
11:56everything is going to be combined with
11:58each other and our final equity curve is
12:01going to be much smoother than this. So,
12:04that means we're also going to need
12:05other symbols to trade. So, I'm not just
12:07going to be trading SOL, I will trade
12:10BTC, ETH, and anything else that I can
12:13find a good strategy for just so that at
12:15the end, I can be profitable every year
12:18and hopefully every quarter. I'm not
12:20even targeting to be profitable every
12:22single month. I mean, I would like that
12:24to happen, but so far that has not
12:26happened for me. So, I cannot say that
12:28I've been profitable every single month.
12:30But that to me is perfectly fine because
12:33I know if I continue doing this, over
12:35the long term, I am going to be
12:37profitable. Because otherwise, this
12:39legend for there is one single strategy
12:41that can crush, let's say, the Bitcoin
12:43market and you're going to be profitable
12:45every single month. That holy grail of
12:47trading does not exist. But
12:48unfortunately, that is exactly what most
12:51people are pursuing and they end up
12:53quitting trading altogether. And let me
12:55show you an example so you can see
12:56exactly what I mean. So, this was the
12:59result that we have for SOL, right? And
13:01I said this is good enough, but it has a
13:03huge max underwater period number, which
13:06I'm not super happy with. And I wanted
13:08to combine its results with other pairs
13:11or strategies. Now, here's the result of
13:13the same strategy, but with different
13:15parameters for trading ETH/USDT instead.
13:17Now, if you take a look at this chart,
13:19you see it is it's still good. So, most
13:21of the times it's going up and that's
13:23really great, but we had like 1 year of
13:26underwater period and I'm not happy
13:28about this at all. And in fact, if you
13:29take a look at the max underwater period
13:31here, you see it's even more than what
13:33we had for the other strategies. So, for
13:35that one, it was 354 days, but for this
13:38one, it is 478 days. So, you could kind
13:41of say it got worse, right? But, if you
13:43take a look at other metrics of the
13:44strategy, such as the sharp ratio, we
13:46can see that it's doing a good job. So,
13:47this again to me is a good enough
13:49strategy, but if you take a look at the
13:51max underwater period, I'm not super
13:53happy with it. But, if you take a look
13:54at the drawdown chart again, you can see
13:56what I just described. Now, here's the
13:58thing. If we combine these two
14:00strategies, so one of them is trading on
14:02SOL, the other is trading on ETH. If I
14:05trade them at the same time, this is
14:07going to be the result that I'm getting.
14:09Now, let's look at this one because it
14:11is based on the absolute value, so let's
14:13look at the log scaled, and you see the
14:16equity curve is much smoother now. So,
14:18this looks much better. And not only
14:20that, if I look at the max underwater
14:22period now, it is 233 days. So, instead
14:26of 478
14:28or 254,
14:29now it is 233 days. Again, it's the same
14:33strategies, but when I combine them
14:35together, I'm getting better results.
14:36Now, the max drawdown has increased a
14:38bit, so I would probably lower my
14:40position sizing, but even if I do that,
14:42the max underwater period is going to
14:43stay the same. Now, there's one more
14:45thing that is actually much more
14:47interesting. So, you see, let's look at
14:49the sharp ratio. In the first one, the
14:51sharp was 1.49. In the second, it was
14:531.64, right? But, in the third one,
14:56which is when we combine the strategies,
14:58I'm getting 1.66, which is better than
15:01all of them combined together. Now, I'm
15:03not done. I also went ahead and
15:04developed another strategy, this time
15:06for BTC. So, because I was trading SOL
15:09and ETH, so I wanted also one for BTC.
15:12And if I run the same things, this time
15:15with BTC also, so if we have three pairs
15:17now, again, ETH, SOL, and BTC, now this
15:21is the number that I'm getting. So,
15:22again, the equity curve is smoother. The
15:25max underwater period is now reduced to
15:27208 days, and the Sharpe ratio is now
15:301.99. And if you take a look at this
15:33drawdown chart, it also looks much
15:35better now. And also, another number
15:37that has increased this way, and I'm
15:39happy with it, is the average trades per
15:41month. Now, it is 23 instead of 15 when
15:44it was just two strategies, or eight or
15:47nine when it was just one strategy.
15:49Because remember, the more number of
15:50trades that you're taking, you have a
15:52higher statistical significance for the
15:54results that you're seeing. So, it means
15:56you can trust the results a little bit
15:57more the more number of trades you're
15:59taking. Now, let's also take a look at
16:01the monthly returns heat map. So, you
16:03see, now most of the months my results
16:05are positive now. So, I don't have as
16:07many negative months now like I had
16:10before. And again, this is another
16:11reason why you would want to combine
16:13multiple strategies and not just to
16:15stick to one. So, as you can see,
16:17diversification is the key here. But,
16:19there's actually something that that you
16:20need to be very careful about. So, you
16:21see, the more correlated the strategies
16:24that you're running are, the less this
16:25method is going to be effective for you.
16:27So, in this example, I was trading ES,
16:29BTC, and so on. All three of them are
16:31cryptocurrencies, and they are all
16:33correlated with each other at least to
16:34some point. When the price of Bitcoin
16:36goes up, all the others go up. When it
16:38comes down, all the others come down,
16:39right? So, this is a known fact. But,
16:41still the strategies were different
16:42enough that they did improve my results.
16:44Because maybe at one point, when one of
16:46them is trying to take a long trade, the
16:47other is going short. So, if you do
16:49that, it's still is going to help us.
16:50But, in reality, if you have access to
16:52trade other markets, so for example, if
16:54I'm trading cryptocurrencies, but at the
16:56same time I trade something like
16:58commodities, like oil, gold, or maybe
17:00even some indexes, you know, markets
17:02that aren't really correlated with each
17:04other, then we're going to get much
17:06better results. So, that's another key
17:08that if you have the option, you want to
17:10trade as many markets that aren't
17:12correlated with each other to improve
17:14your results with diversification. But,
17:16how do we know the results aren't
17:18overfit? The way I answer that question
17:20is using Monte Carlo. In just a
17:21dashboard, there are two ways to use
17:23Monte Carlo. So, one is for the trades,
17:25which is where we shuffle the order at
17:27which we took the trades, just to see
17:29what would have been the max drawdown in
17:31those cases, so that we can prepare for
17:33them. So, for example, this is for that
17:35SOL USDT strategy that I showed you
17:37earlier. And as you can see, the max
17:38drawdown, even though in the original
17:40backtest was minus 17% in the worst 5%
17:44of all the simulations that I ran, it
17:46could have been as bad as minus 42%.
17:48Now, I don't think we should take this
17:50into account all the time, because this
17:52is usually just too much of a doom
17:54scenario. And if I adjust my position
17:56sizing based on this all the time, then
17:58I'm going to leave some money on the
18:00table and lose some opportunities, which
18:02I don't want to do. So, what you could
18:03do instead is to prepare for the median.
18:05And because I personally am ready for a
18:07max drawdown as bad as minus 30%, a
18:10minus 26% is perfectly fine to me. But,
18:13this actually isn't really about
18:14overfitting as much as it is related to
18:16position sizing. However, the other tool
18:18that we have for overfitting is Monte
18:20Carlo based on the candles that we use
18:22for running the backtest. Now, in this
18:24case, the yellow line is the original
18:26backtest, and all these blue lines are
18:28the simulations. Now, these simulations
18:30are what would have happened if we
18:31executed the same backtest, but this
18:33time using simulated candle. Now, these
18:35simulations are actually based on the
18:37original data, so they are not entirely
18:39made up. There are different ways that
18:41we make them, but one way that you can
18:43think of it is what would have happened
18:45if the price was just slightly different
18:47than what it was when you took that
18:49trade. Because we always know that there
18:51are cases where, for example, the price
18:52is just about to touch your take profit
18:54order, but then it doesn't, and you
18:56might even lose that trade. If that
18:58happens just a couple of times, that's
18:59is perfectly fine. But, what if your
19:00strategy is just counting on those edge
19:03moments? In those moments, we're going
19:04to say, "Okay, the strategy is overfit."
19:06Meaning that even if your luck was a
19:08little bit off when live trading, you're
19:10not going to make the same amount of
19:11money, and maybe even you're going to
19:13lose. So, Monte Carlo can show us the
19:15other scenarios. Now, the way I read
19:17this is, first of all, I would like to
19:19see my equity curve of the original
19:20backtest to be relatively in the middle
19:23of the simulation. So, if I see it, for
19:25example, if it were here, you know,
19:27above all the other simulations, that
19:28would have definitely been an overfitted
19:30strategy for me. Now, another way that I
19:32read this is that I don't even take a
19:34look at the chart. Instead, I just look
19:35at this table here. Now, I personally
19:37care about two values, the max drawdown
19:39and Sharpe ratio. Now, you already know
19:41why I care about the max drawdown, which
19:43is the same reason that I just explained
19:44in the above chart. But, with this one,
19:47I'm going to take a look at the Sharpe
19:48ratio. So, in the original, the Sharpe
19:50ratio was 0.74,
19:52while in the best 5%, which is usually
19:54the results for an overfitted strategy,
19:56was 1.96. So, my results is definitely
19:59less than this number by a huge amount.
20:02So, that's the first thing that I would
20:03like to see, that my original backtest
20:05result is not similar to the best 5%.
20:07But, in reality, the further it is from
20:09this number, the less probability of it
20:11being overfit. Now, in this case, it is
20:130.74, right? And if you pay close
20:15attention, you can see that it is even a
20:17slightly less than the median number.
20:19Now, it would have been fine for me if
20:20it was bigger than this number, but now
20:22that it is even less than this, for me
20:24personally, this is a great indication
20:26that the strategy is not overfit. Now,
20:28one thing you need to remember is that
20:30the longer the duration of the backtest,
20:32so for example, in here is 2 years, the
20:34more statistical significance this test
20:36is going to have. So, if I were to run
20:38it for like just a few months, and saw
20:40something like this, it would not have
20:42been enough for me to make a conclusion.
20:44Now, in the past, I used other methods
20:45such as cross-validation, but these days
20:47I have a huge favor for Monte Carlo in
20:50order to prevent overfitting. Now, this
20:51was a new type of video that I just did.
20:53If you want me to make more tutorials
20:55like this one, let me know. Because if
20:57you're going to do some sort of
20:58diversification like this, you're going
21:00to have to exactly understand the amount
21:02of risk that you're taking for each
21:04trade, and how that affects the results
21:05that you get overall. And before I leave
21:07you, we're going to have a giveaway.
21:09Random subscriber who likes the video
21:11and comments is going to win 1 million
21:13Buck Token. All right, let's pick the
21:15winner from the previous video.
21:19And the winner is Emily's heart on the
21:21sand. You lost me at 15. All right,
21:23thank you so much for your comment.
21:25Please do reach out to me so I can send
21:26you your tokens. Thank you so much for
21:27watching. I'll see you in the next one.

About this transcript

This page contains the full transcript of How I Decide If a Strategy Is Ready — Not Just a Good Backtest by Algo-trading with Saleh, generated from the public captions YouTube serves with the video. The transcript has 4,512 words across 618 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.