Can You Actually Trust AI With Your Trading Strategy? — Transcript

Full transcript

1. 0:00AI can completely screw you over if you
2. 0:03don't know what to look for. It
3. 0:04hallucinates and makes mistakes just to
4. 0:07satisfy you and your question. Don't get
5. 0:10me wrong, AI has completely changed the
6. 0:12way I research trading strategies.
7. 0:15Something that would have taken me hours
8. 0:16of spreadsheets and days of data
9. 0:19crunching and a calculator can basically
10. 0:22potentially be done in minutes now. I
11. 0:24can give AI thousands of trades and ask,
12. 0:26"What's my win rate? What's my average
13. 0:28winner? What's my average loser? What
14. 0:30time of day performs best?" I can even
15. 0:32give it a strategy and have it write the
16. 0:34code necessary to backtest it. It's
17. 0:36[ __ ] incredible, but there's a
18. 0:38problem. AI can be wrong, and I'm not
19. 0:40talking about the trading strategies
20. 0:42being wrong. I'm talking about the
21. 0:43actual backtest being wrong because an
22. 0:46AI can give you beautifully formatted
23. 0:48tables and information, a win rate down
24. 0:51to two decimal places, the profit
25. 0:53factor, the drawdown, the expectancy,
26. 0:56and an entire explanation of why your
27. 0:58strategy supposedly works while the
28. 1:00underlying calculation is completely
29. 1:02[ __ ] And the dangerous part is that
30. 1:05it can sound just as confident when it's
31. 1:07wrong as when it's right. So, today I
32. 1:09want to show you the things that I check
33. 1:11whenever I use AI for trading research
34. 1:14because if you're going to start using
35. 1:15ChatGPT or Claude or Grok or any other
36. 1:19AI to backtest strategies, these are the
37. 1:22things you absolutely need to
38. 1:23understand. And here's probably the
39. 1:25easiest mistake to avoid. Let's say I
40. 1:27type, "Backtest a moving average
41. 1:29crossover on EUR/USD for the last 10
42. 1:32years and tell me the win rate." An AI
43. 1:34responds, "The strategy produced a 57.3%
44. 1:37win rate with a profit factor of 1.42."
45. 1:40That sounds very specific, but my first
46. 1:43question should be, "Where did you get
47. 1:44the data?" Because you cannot backtest
48. 1:46something without historical market
49. 1:48data. If you haven't uploaded the data
50. 1:49yourself, if the AI hasn't retrieved the
51. 1:52data from a reputable source, and we
52. 1:54haven't connected to something
53. 1:56containing the data, what exactly did it
54. 1:58backtest? This sounds ridiculously
55. 2:00obvious when I explain it like this, but
56. 2:03this is where people get caught because
57. 2:04a number like 57.3%
58. 2:07feels calculated. It's precise. Your
59. 2:10brain interprets precision as evidence,
60. 2:13but precision doesn't mean anything if
61. 2:14you can't explain where the underlying
62. 2:16information came from. So, rule number
63. 2:18one, never trust the results until you
64. 2:20understand what data produced it. The
65. 2:22next mistake is accepting the final
66. 2:24number. If I upload 10,000 trades and AI
67. 2:27tells me your win rate is 64%. Cool.
68. 2:31Don't stop there. Ask how many trades,
69. 2:34how many winners, how many losers, how
70. 2:37many break evens, what's the gross
71. 2:38profit, what's the gross loss, what's
72. 2:41the average winner, what's the average
73. 2:43loser, what's the profit factor because
74. 2:46now these numbers have to reconcile. If
75. 2:48it says 1,000 trades, 640 winners, 360
76. 2:52losers, okay. 640 divided by 1,000 is
77. 2:5664%. That's something that you can
78. 2:58independently verify. The same thing
79. 3:00applies to the profit factor. Gross
80. 3:02profit divided by the gross loss should
81. 3:05produce the profit factor. Expectancy
82. 3:07should reconcile with average winners
83. 3:09and average losers and the
84. 3:11probabilities. You're basically forcing
85. 3:13the AI to leave a trail of breadcrumbs
86. 3:15and the more breadcrumbs you have, the
87. 3:17easier it becomes to catch a mistake if
88. 3:20it does make one. Now, the next one and
89. 3:22this one is surprisingly easy. Imagine
90. 3:24you export your trades from MetaTrader.
91. 3:26You might have an entry row and an exit
92. 3:29row. AI looks at 2,000 rows and tells
93. 3:32you you made 2,000 trades, but you
94. 3:34actually made 1,000 trades or maybe you
95. 3:37have a partial close, multiple
96. 3:39positions, deposit, withdrawals,
97. 3:41commissions, swap fees, break even
98. 3:44trades. Suddenly, something that sounds
99. 3:46as simple as calculate my win rate isn't
100. 3:49necessarily simple anymore. So, whenever
101. 3:51I upload trading data, one of the first
102. 3:53things I want AI to explain is what does
103. 3:56one row represent? Then I normally want
104. 3:59to see a few examples. Show me trade
105. 4:01number one. Entry here, exit there,
106. 4:04profit here. Show me trade number two,
107. 4:06and so on. And if those examples don't
108. 4:09match what I can see manually, I stop
109. 4:11because there's absolutely no reason to
110. 4:14analyze another 10,000 rows if we're
111. 4:16already interpreting row number one and
112. 4:18two incorrectly. Now, the next thing,
113. 4:20and here's where we get into something
114. 4:21much more dangerous. Let's say you don't
115. 4:24have trade results yet. You have a
116. 4:26strategy, so you ask AI to code and
117. 4:28backtest. Now you're trusting AI to
118. 4:31translate English into mathematics.
119. 4:33And tiny differences matter. There's a
120. 4:35massive difference between enter when
121. 4:38the moving average cross and enter
122. 4:40whenever the fast-moving average is
123. 4:42above the slow-moving average. Those
124. 4:45sound similar, but they're completely
125. 4:47different trading rules. Or maybe your
126. 4:49strategy says, "Wait for the candle to
127. 4:51close." But the code enters during the
128. 4:53candle. Or your indicator repaints. Or
129. 4:56the code accidentally references
130. 4:58information that wouldn't have been
131. 5:00available at the time. Now you have what
132. 5:02is essentially a time machine. Your
133. 5:04backtest knows information from the
134. 5:06future, and surprise, surprise, you've
135. 5:08discovered the greatest trading strategy
136. 5:10ever created. This is when AI writes a
137. 5:12strategy for me. I don't immediately
138. 5:14care about the profit and loss. I want
139. 5:16to see whether the signals match
140. 5:17reality. Pull up 10 trades manually. Did
141. 5:20the backtester enter when the indicator
142. 5:22actually signaled? Did it enter on the
143. 5:24correct candle? At a realistic price?
144. 5:27With the correct stop loss and the
145. 5:29correct take profit? If 10 random
146. 5:32examples don't line up, the beautiful
147. 5:34equity curve means absolutely nothing.
148. 5:37And here's another really easy one to
149. 5:39screw up. Let's say I'm trading the
150. 5:415-minute chart, but I'm using a 1-hour
151. 5:43chart as confirmation. At 10:25 in the
152. 5:46morning, the current hourly candle
153. 5:48hasn't closed because it's still
154. 5:49forming. So, your backtest needs to
155. 5:51understand exactly what hourly
156. 5:53information was actually available at
157. 5:5510:25. If it accidentally uses
158. 5:57information from the completed hourly
159. 5:59candle, it's using information from the
160. 6:01future. And this problem gets especially
161. 6:03nasty when you're converting indicators
162. 6:05between platforms. Let's say you
163. 6:07designed it on TradingView and you move
164. 6:09it into MetaTrader. TradingView might
165. 6:11handle higher time frame information one
166. 6:13way and MetaTrader does it another way.
167. 6:16Your Python script does it another way.
168. 6:18The code can compile perfectly.
169. 6:20Everything looks great with no errors.
170. 6:22The backtest can run perfectly with no
171. 6:25errors. And the underlying strategy can
172. 6:27still be different from what you thought
173. 6:29you were testing. Now, number six. This
174. 6:31is probably the biggest problem with
175. 6:33scalping strategies. Let's say AI
176. 6:35discovers a strategy that makes 1.5 pips
177. 6:38per trade. Amazing. Except your spread
178. 6:41is one pip. You pay commissions, you get
179. 6:43occasional slippage. Congratulations,
180. 6:46your profitable trading strategy might
181. 6:48actually lose money now. And this gets
182. 6:49even worse when the stops and targets
183. 6:52are extremely small. Because now the
184. 6:54exact path price traveled inside of each
185. 6:56candle matters. Imagine one five-minute
186. 6:59candle hits both your stop loss and your
187. 7:01take profit. But, which one happened
188. 7:03first? If all I have is open, high, low,
189. 7:06and close, I don't necessarily know. The
190. 7:09backtester has to make an assumption or
191. 7:11use more granular information like
192. 7:14backtesting by ticks. MetaTrader even
193. 7:17has real tick testing mode specifically
194. 7:20because execution-sensitive testing
195. 7:22benefits from actual broker tick history
196. 7:25rather than relying entirely on
197. 7:26generated ticks. This takes much longer,
198. 7:29but it's worth it. So, the smaller your
199. 7:31trade becomes, the more suspicious you
200. 7:33should become of overly simplified
201. 7:35backtests. Now, number seven. Here's the
202. 7:38one I think traders using AI need to
203. 7:41understand more than anything else. You
204. 7:43can have a back test where the data is
205. 7:45real, the calculations are correct, the
206. 7:47code is correct, the trades are correct,
207. 7:49and the conclusion is still wrong
208. 7:51because you overfit the strategy.
209. 7:53Imagine I tell my AI, "Find me the best
210. 7:56moving average length. Now try every
211. 7:58single stop loss and every single take
212. 8:00profit, every time frame, every trading
213. 8:03session, every day of the week, every
214. 8:05indicator setting, every combination,
215. 8:07and eventually something is going to
216. 8:09look amazing. That's not necessarily a
217. 8:12new discovery. It can simply be the best
218. 8:16selection. You're effectively asking,
219. 8:18"Out of thousands of random variations,
220. 8:21show me the one that happened to predict
221. 8:22history the best." Now MetaTrader's
222. 8:24optimization feature literally ranks
223. 8:27parameter combinations according to a
224. 8:29chosen fitness criterion, and the danger
225. 8:32is that as you give the optimizer more
226. 8:34freedom, it can increasingly fit quirks
227. 8:38of the historical period rather than
228. 8:40something that's persistent. Like you're
229. 8:42more focusing on outliers at this point.
230. 8:45This is exactly why you need data that
231. 8:48the optimization has never seen before.
232. 8:50Like using a completely different pair.
233. 8:52Let's say you do EUR/USD, then you would
234. 8:55do EUR/GBP. AI is not some magical truth
235. 8:59machine. It is a large language model.
236. 9:02It's a tool. It's an incredibly powerful
237. 9:04tool, but if you feed it a flawed
238. 9:07experiment, it can help you produce a
239. 9:09flawed answer faster than you've
240. 9:12produced one before. And sometimes it
241. 9:13even puts the answer into a beautiful
242. 9:15table with two decimal places, which
243. 9:18somehow makes it more legitimate and
244. 9:19even more dangerous. So use AI,
245. 9:22absolutely. Just make it prove its own
246. 9:25work, and don't take everything that it
247. 9:27says as factual information because the
248. 9:31goal isn't to get AI to tell you that
249. 9:33your strategy works. The goal is to
250. 9:35figure out whether it actually does, in
251. 9:37real market conditions, going forward.
252. 9:40If you guys enjoyed this video, please
253. 9:42share it with your friends. Thank you so
254. 9:43much for watching, and we will see you
255. 9:45in the next video.