Using AI For Optimizing Strategies In Python — Transcript

Full transcript

1. 0:00hey guys it's Sol in the previous video
2. 0:01I created a swing Trend following
3. 0:04strategy which was performing well but
4. 0:06in this video I'm going to use the
5. 0:08optimization mode of Jesse which uses
6. 0:10the genetic algorithm to improve the
7. 0:13results that we already had also we're
8. 0:15going to have a giveaway so make sure to
9. 0:16stick until the end of the video so you
10. 0:18won't miss it all right let's get into
11. 0:19it so this was the performance that we
12. 0:21were
13. 0:22getting this was our p&l the number of
14. 0:26Trades we did and the Max rodon and the
15. 0:30sharp ratio everything looks okay but
16. 0:33let's see if we can improve it so I'm
17. 0:35going to open our strategy now we're
18. 0:37using a believe four indicators for this
19. 0:39strategy the adx has a threshold of 25
20. 0:42and I'm not passing any period number
21. 0:46for it and that's intentional and we're
22. 0:48using three EMAs and we're using the ATR
23. 0:53for setting our suplus and for setting
24. 0:56our initial take profit all right let's
25. 0:58begin so instead of 25 I'm going to
26. 1:01write self HP which is a dictionary so I
27. 1:05will give it a name adx
28. 1:09threshold and then I will come down here
29. 1:12and Define a new method called hyper
30. 1:15parameters and in it I'm going to return
31. 1:17a list of dictionary values dictionary
32. 1:20has a name it has a type it has a
33. 1:22default value and what the co-pilot is
34. 1:26missing it will also have a Min and Max
35. 1:28values
36. 1:32now for adx the type of int is fine the
37. 1:36default value of 25 is also find the
38. 1:38minimum of 10 and Max of 50 it actually
39. 1:41looks perfect okay so let's go back and
40. 1:44continue we also have EMA
41. 1:47values so instead of this one I'm going
42. 1:50to say self HP
43. 1:52E1 period I'm going to have E2 period
44. 1:57and E3 now I will go here and add okay
45. 2:01so this looks fine but it's not perfect
46. 2:05so defa the default value for the E1 was
47. 2:0821 that means the Min of it should not
48. 2:10be two it should be 10 the max should be
49. 2:1340 that's fine the default for the
50. 2:16second period was 50 so now I don't like
51. 2:22this Min so I'm going to set the Min to
52. 2:2440 and the max to 70 and for the third
53. 2:29one we had the period of 100 so the Min
54. 2:33would be around 70 and the max would be
55. 2:38130 okay this looks good next we have
56. 2:42the ATR value which we are using
57. 2:44initially for setting our suplus now we
58. 2:47were multiplying the ATR value by two so
59. 2:49instead of two I will say self HP sub
60. 2:54ATR I'm going to copy this because I'm
61. 2:57also using it elsewhere so here
62. 3:00and also here and also here and now I
63. 3:04will Define a new value called a stop
64. 3:07ATR the type of it is still int actually
65. 3:12for this one I don't want the type of it
66. 3:14to be int because we don't just want it
67. 3:16to be two or three or four we values
68. 3:19such as 2.1 2.2 these are actually
69. 3:22perfect with this use case so instead I
70. 3:24will give it the type of float the
71. 3:26default is two the minimum is one and
72. 3:29the max is five actually five is a bit
73. 3:32too much for my suplus I don't like it
74. 3:34um 3.5 I believe is like good enough for
75. 3:40me and we also need another one for the
76. 3:43take
77. 3:47profit going to copy this and paste it
78. 3:50also here and I will add another
79. 3:53one the default is two actually no the
80. 3:57default was three in our strategy and
81. 4:00the minimum of one and maximum of let's
82. 4:04say
83. 4:054.1 actually let's set it to five okay
84. 4:08so this is good so for some indicators
85. 4:11such as the EMA I am replacing the
86. 4:14period number with the hyper parameter
87. 4:16which we're going to optimize but for
88. 4:17some other such as the adx or the ATR I
89. 4:20don't do that why simply put the more
90. 4:22hyper parameters you use in your
91. 4:24strategies the more you try to optimize
92. 4:26every single part of it the more it is
93. 4:28likely that your strategy is going to
94. 4:30end up being overfit which means it
95. 4:33might perform well in back test but in
96. 4:34live touring it won't so that's always a
97. 4:37risk of optimization and we want to
98. 4:39avoid that other than that I'd like to
99. 4:42be able to understand my strategy to the
100. 4:44fullest I don't want to be a lot of
101. 4:45differences when I was trading it
102. 4:47manually or when I was trading it with
103. 4:49my butt and I almost never changed the
104. 4:51default parameters on trading view when
105. 4:53I'm trading so when I'm turning my
106. 4:56strategies into algorithms I want to
107. 4:58keep doing the same thing but I want to
108. 5:01emphasize that if you are a beginner
109. 5:03with Jesse if you're just getting
110. 5:04started don't try to optimize every
111. 5:06single part of your strategy and also
112. 5:09never use the optimization mode for
113. 5:11turning a negative strategy into a
114. 5:13profitable one only use it to improve
115. 5:15your existing strategies all right
116. 5:18especially if you're a beginner if you
117. 5:19haven't even executed let's say 100 back
118. 5:22test with Jesse yet don't start with the
119. 5:25optimization mod because you're going to
120. 5:26face lots of issues and chances are you
121. 5:29will just get exhaust it and stop
122. 5:30trading all together and I don't want
123. 5:32that happening to you okay so this looks
124. 5:34really good but I also want to do one
125. 5:36more thing because I think it will have
126. 5:37some educational value so here we were
127. 5:40using the current price for the entry of
128. 5:43our strategy we were using a market
129. 5:44order to open our positions but you
130. 5:47might wonder what we were using a limit
131. 5:49order well so what if instead of using
132. 5:52the current price I said self price plus
133. 5:55I say plus because this is a go short
134. 5:57method so instead of some kind of number
135. 6:00now to make this a dynamic number to
136. 6:03work for every other strategy I'm going
137. 6:05to use again the ATR indicator so I will
138. 6:08say t ATR and I'm going to multiply it
139. 6:12by a hyper parameter value I will call
140. 6:14it entry ATR and actually let's copy
141. 6:18this and go to go long but this time
142. 6:22because it's for long position instead
143. 6:23of adding we will be subtracting now I
144. 6:27will come down and add in another value
145. 6:30Now the default of it is set to one
146. 6:33actually for okay so that's good for the
147. 6:35default but the minimum I want it to be
148. 6:37very low such as this or even lower than
149. 6:41that and the maximum could be yeah let's
150. 6:44say two so now we are using a limit
151. 6:46order to enter our positions actually so
152. 6:49let's run another back test for this
153. 6:51strategy to First make sure that it
154. 6:53actually works and we're not making any
155. 6:55syntax error and second because I want
156. 6:58to see what this change that just made
157. 7:00had in our numbers so we were getting
158. 7:04128% with the max run of 15 and with a
159. 7:07sharp of
160. 7:101.47 okay so we made significantly less
161. 7:14than before and the Maxon is even
162. 7:17worse okay so this is the number we're
163. 7:19getting right now but it doesn't matter
164. 7:21much so let's go to the optimization
165. 7:24page now and everything is fine except
166. 7:26the strategy so I'm I'm going to choose
167. 7:30the correct one as for the duration you
168. 7:32have to notice that it's a bit different
169. 7:34than the back test mode so for instance
170. 7:36when you are back testing and your
171. 7:38duration is set to since the beginning
172. 7:40of 2022 and up until let's say 2023 it
173. 7:44will be for one year right but when you
174. 7:46set the same number for the optimization
175. 7:49it will split those candles into a
176. 7:51training and a testing set and the exact
177. 7:53number is actually it will use the 85%
178. 7:55of it for training and the other 15% for
179. 7:58testing also it's best if if you leave
180. 8:00some time out for the cross validation
181. 8:03period now if these numbers don't make
182. 8:05sense to you right now that's okay I'm
183. 8:07going to explain it later when we get
184. 8:08the results so for this one I'm going to
185. 8:10set the beginning to 2022 and the ending
186. 8:14to 20 24 now this optimal number of
187. 8:17Trades number here is actually quite
188. 8:19important now what does it mean so if I
189. 8:22run a back test with this sitation this
190. 8:25is the result that I will get now the
191. 8:27number of Trades that we're executing is
192. 8:3053 so in the optimization for the same
193. 8:33period you should set the total number
194. 8:36of Trades to approximately similar
195. 8:38number now why do we have this setting
196. 8:39in the first place well the reason is
197. 8:42imagine if your strategy during like one
198. 8:44year it executed three trades but all of
199. 8:47them were in profit which means your the
200. 8:50win rate of your strategy is going to be
201. 8:51100% is that a good strategy well on
202. 8:54paper it seems perfect right it it has
203. 8:57100% win rate but in reality it's it's
204. 8:59not great because three trades is just
205. 9:02too low it doesn't have statistical
206. 9:04significance in other words you're not
207. 9:06going to be confident trading that
208. 9:08strategy you're not going to be
209. 9:09confident with the uh with any number
210. 9:12that or chart that you're seeing because
211. 9:14there's a low chance that in the future
212. 9:16the same thing is going to happen but if
213. 9:18your strategy was executing let's say
214. 9:20100 50 200 or whatever you will be much
215. 9:23more confident in its metrics right so
216. 9:26with that in mind we want to tell justy
217. 9:28what would be an optimal number of
218. 9:30trades for this strategy now the number
219. 9:32that you pick it will have to do
220. 9:34something with the duration of the back
221. 9:35test and the time frame of your back
222. 9:37test and that's why this isn't a
223. 9:39hardcoded number and you have to give it
224. 9:41as a setting now in this example we're
225. 9:44trading the 4 hours time frame which is
226. 9:46a big one and the number of Trades that
227. 9:50we were getting in our backst was around
228. 9:5253 so in the optimization mode I will
229. 9:55set this number to something such as 50
230. 9:58which it already is so this is good for
231. 10:01me but if I wanted this strategy to
232. 10:03execute more trades just to be more
233. 10:05confident in its results yes I could
234. 10:07have increased this number to S
235. 10:08something such as 70 or something all
236. 10:11right let's bring it back to 50
237. 10:13everything else looks fine also when you
238. 10:16are running the optimization mode make
239. 10:18sure in the settings that firstly you
240. 10:20set the CPU course to a number that you
241. 10:23can afford so my machine has eight
242. 10:25chords so I'm going to set this to six
243. 10:29you can choose the fitness function the
244. 10:31default which is the sharp is fine for
245. 10:33me the warm-up candles is very important
246. 10:36so if in my back test for instance I was
247. 10:39using let's say a bigger number such as
248. 10:41400 candles then I should make sure that
249. 10:44in the optimization I also have the same
250. 10:46number so this is really
251. 10:49important and as for the exchange you
252. 10:51can set the trading fee and right now
253. 10:53this trading fee is set for spot trading
254. 10:57not for trading futures so if I go back
255. 10:59to the back test section and look at the
256. 11:01fees for binance paper future is set to
257. 11:04this
258. 11:05number but on the optimization is
259. 11:08actually higher so let's change this and
260. 11:10the assorting capital of 10,000 is fine
261. 11:12the type of it is Futures by default The
262. 11:15Leverage mode is set to cross and The
263. 11:17Leverage is set to
264. 11:19three again make sure these numbers are
265. 11:21the same that you were using for your
266. 11:23back test all right everything else
267. 11:25looks fine and we should be able to
268. 11:27start the optimization
269. 11:30oh so I forgot one thing uh let's also
270. 11:33enable the debug
271. 11:35mode and the fast mode for this
272. 11:38strategy okay let's run it
273. 11:41again notice that it's telling me that
274. 11:43in 21 minutes it will finish well here's
275. 11:46the thing the optimization mode or let's
276. 11:48call it the Genting algorithm it has two
277. 11:51phases the first phase is for generating
278. 11:54the initial population now what does it
279. 11:56mean it means it is creating some
280. 11:58numbers some hyper parameters randomly
281. 12:01and no one knows if those are going to
282. 12:03be good numbers or bad numbers and
283. 12:04that's what's happening right now so in
284. 12:0621 minutes we're going to finish doing
285. 12:08that in the second phase we're going to
286. 12:10use those population and we're going to
287. 12:13breed the numbers that are good because
288. 12:15that's how the genetic algorithm works
289. 12:17it breeds the good dnas and it tries to
290. 12:19make even better babies or in our case
291. 12:22that would be better results for our
292. 12:23strategy so that's why we have two
293. 12:26phases so in about 20 minutes I will
294. 12:28come back and we're going to have
295. 12:30another table but even in here you can
296. 12:32see we have the average strategy
297. 12:34execution time which is useful and
298. 12:36here's the information about the
299. 12:38strategy you're running this is the
300. 12:40population size the number of iterations
301. 12:43and the solution length the more hyper
302. 12:44parameters that you add to your strategy
303. 12:47these numbers are going to exponentially
304. 12:49increase so that's also another reason
305. 12:52why you shouldn't use as many IP
306. 12:53parameters as you can just try to keep
307. 12:56it as minimum as possible another reason
308. 12:58is that your optimization sessions are
309. 13:00just going to finish much faster it's
310. 13:03been around 3 hours since we started
311. 13:05this optimization session and it's been
312. 13:0731% of the second phase of the
313. 13:09optimization session however the thing
314. 13:12about optimization is that you don't
315. 13:14have to wait for it to reach 100%
316. 13:17sometimes you can start playing around
317. 13:18with the results as soon as you see some
318. 13:21numbers that you actually like and also
319. 13:23sometimes you just wait and wait and
320. 13:25wait but after a certain point you don't
321. 13:27see any improvements anymore more so
322. 13:30especially for the sake of this tutorial
323. 13:32because I want to continue my recording
324. 13:35I'm going to use the results that I have
325. 13:37so far but in reality you probably want
326. 13:40to wait for for it to reach to 60% or
327. 13:4370% uh or you want to try out all the
328. 13:46numbers after it's at 30% try the number
329. 13:49see if you like it and then at 60% and
330. 13:52then compare them and see which one
331. 13:53works better so here these are the type
332. 13:56of results that we're getting now notice
333. 13:57there's a rank which isn't really
334. 13:59important but usually the best one comes
335. 14:01first we have the DNA which is really
336. 14:03the result that uh we're going to use
337. 14:05later we have the fitness number which
338. 14:07is what it's ranking them based on and
339. 14:10then we have these numbers that are
340. 14:12really useful to us we have the training
341. 14:14versus testing win rate so this one is
342. 14:16the win rate that in theatis is getting
343. 14:19using this DNA on the training period
344. 14:22but on the testing period is getting
345. 14:24this win rate now these are the the
346. 14:26total trades that were executed so in
347. 14:28the train period is 36 on the T on the
348. 14:32testing period is six and these are the
349. 14:34p&l for those two back tests now what's
350. 14:37the difference between training and
351. 14:38testing well the training period is what
352. 14:41Jesse is using to improve itself based
353. 14:45on the metric that that it is using
354. 14:47which is this Fitness number here but
355. 14:49the testing number it has no effect in
356. 14:52in in that but for our eyes it's really
357. 14:55useful so why do we have it there
358. 14:57because if you see a DNA that's
359. 14:59performing extremely well in the
360. 15:01training period but in the testing it's
361. 15:03doing horribly then you can just ignore
362. 15:05that one and and that's also why I said
363. 15:08the ranking doesn't really matter much
364. 15:10because the ranking is only considering
365. 15:12the training period so this one looks
366. 15:14good it and it is our number one rank uh
367. 15:18but the total trades is actually lower
368. 15:21than the second option but nonetheless
369. 15:24let's begin with it so I'm going to copy
370. 15:26this DNA here and go back to my editor
371. 15:29and I will add a new function called DNA
372. 15:33and it should return simply a string
373. 15:37which is the DNA we just copied okay so
374. 15:40let's go back to the back testing page
375. 15:43now these were the results that we were
376. 15:45getting and remember that in the
377. 15:47previous video our results was actually
378. 15:49better than this and this is because of
379. 15:51that change I made last minute to open
380. 15:53positions using a limit order because I
381. 15:55also wanted to test that one but anyway
382. 15:57so this is we have 5% pnl and minus 22%
383. 16:02Max thron so let's open a new tab and
384. 16:07rerun the same back test but this time
385. 16:09using that
386. 16:11DNA okay so this is the number we're
387. 16:14getting the panl is at 97% and the max
388. 16:18thron is at minus 8% so that's a
389. 16:21significant Improvement but here's the
390. 16:22thing so far we had the training period
391. 16:25and the testing period those were the
392. 16:26two that the optimization
393. 16:29session was running and we were seeing
394. 16:31the numbers here but what I do to make
395. 16:33sure my strategy isn't overfit is to use
396. 16:37a third period which is called a
397. 16:39validation period so what I mean by that
398. 16:41is for the period of this optimization
399. 16:44session we chose since the beginning of
400. 16:472022 up until
401. 16:502024 just the first day so it hasn't
402. 16:54been back toed on since 2024 up until
403. 16:58let's say last month so let's go back to
404. 17:01backst and execute another backst since
405. 17:04the beginning of
406. 17:062024 up until the 7th month and just to
407. 17:11make sure we are running it without the
408. 17:13DNA I will comment this now let's run
409. 17:16this and this is the number we're
410. 17:19getting okay 29% profit minus 13% Max
411. 17:23thr okay so now this time let's run it
412. 17:27with the DNA
413. 17:29again 29% minus
414. 17:3413 now we're getting 21% minus almost 7%
415. 17:39so the profit number came down a bit but
416. 17:42the max Rod also came down and if you
417. 17:44remember from my previous video I
418. 17:46mentioned that when the max rodon comes
419. 17:48down it allows you to add more position
420. 17:51which will end up making us a bigger p&l
421. 17:53number so I I'm not sure if I can say
422. 17:55this is a worse strategy than the other
423. 17:57one uh but but but anyways let's just
424. 18:00continue because I want to test other
425. 18:02DNA numbers because this one was not
426. 18:04making that many trades and I wasn't
427. 18:06super happy about that now this one is
428. 18:08making more
429. 18:10Trad and it seems to have a better pnl
430. 18:13at least in here so let's copy this
431. 18:19one let's go back to back test 97 -
432. 18:258 107%
433. 18:299 and 7 okay so it does look better but
434. 18:32also Let's test it on the validation
435. 18:34period 21 -
436. 18:387 19 - 11 so you see even though it got
437. 18:43better in our training and testing
438. 18:45period it did not get better in our
439. 18:46validation period so that's why I won't
440. 18:49use this
441. 18:50DNA so let's carry
442. 18:53on so this one is executing actually the
443. 18:57most number of Trades
444. 18:59and that's something I really like
445. 19:03so let's copy this one go back to back
446. 19:06test 107% minus
447. 19:129
448. 19:1495% and minus
449. 19:1610% not bad but let's see how it does
450. 19:20the validation
451. 19:23period 24% minus 7% okay so it's doing
452. 19:27actually really well in the validation
453. 19:30period so far I think it's the best one
454. 19:33let's see if we have any one more
455. 19:38interesting okay so look at this one
456. 19:40it's actually really good the win rate
457. 19:43of it is lower than these ones but it is
458. 19:46executing more trades and the pel is
459. 19:50actually better okay so you see it
460. 19:51jumped that's because a new iteration of
461. 19:54the optimization session was just done
462. 19:56and these are the new results but
463. 19:58luckily I copied it before it happened
464. 19:59so let's move
465. 20:05on 95 -
466. 20:1110 121 Max than- 12
467. 20:1512% so it looks actually quite good 24%
468. 20:20minus
469. 20:227 oh it's actually resing the bed here
470. 20:26it's really bad
471. 20:30okay let's go back keep
472. 20:33looking let's try this
473. 20:43one 107 -
474. 20:479 19% minus this okay so I think this
475. 20:53one had the best numbers so far
476. 21:02- 7
477. 21:0424 95% minus 10 okay yeah so I think I
478. 21:08really like this one so again this is
479. 21:10what we had before the optimization
480. 21:12session and this is what we have after
481. 21:15and just to make sure we also run it on
482. 21:17the validation period and it's also
483. 21:20performing will there so before I forget
484. 21:23there's something really important that
485. 21:24when you run a back TST using a DNA
486. 21:26string you will get a new section here
487. 21:29called hyper parameters it will tell you
488. 21:31the DNA and the values that have been
489. 21:32used for this practice so remember the
490. 21:35the hyper parameters we defined here so
491. 21:37you might be asking okay so in this
492. 21:39backst what number is being used for the
493. 21:41adx threshold and if you go here it's 44
494. 21:45what number is being used for the first
495. 21:47period is 35 44 80 and the sub ATR is at
496. 21:521 and the take profit ATR is at 1.6 and
497. 21:57that inry ATR which I defined at the
498. 21:59last second is at
499. 22:020.06 so this can also be really handy
500. 22:05and teach us a lot so for example I
501. 22:07didn't even think that a very short sub
502. 22:09ATR number would be this much helpful
503. 22:12but you can clearly see that it actually
504. 22:13is okay so just as a reminder these
505. 22:16results were just after 34% of the
506. 22:20progress bar if I come back in let's say
507. 22:22three or 4 hours we're going to get
508. 22:24completely new results and hopefully
509. 22:26better ones but I think I made my point
510. 22:29in for the video and that is if you
511. 22:31believe in your strategy you know how
512. 22:33it's working and you just want to
513. 22:36improve some of the parameters of it or
514. 22:38if you have a setup that you know in
515. 22:40theory should work but in the practice
516. 22:42just isn't working so an example of that
517. 22:44is that last minute limit order that I
518. 22:47added for our entry because I think it
519. 22:50makes sense that just wait for a small
520. 22:52pullback to enter your position but the
521. 22:55numbers weren't really good and that's
522. 22:57because yes the logic it was good but
523. 23:00the exact amount that I was using just
524. 23:02wasn't good in these type of scenarios
525. 23:04the optimization session can be really
526. 23:06really helpful now before I leave we're
527. 23:07going to have a giveaway a random person
528. 23:09who likes this video posts a comment and
529. 23:11subscribes to the channel is going to
530. 23:12win 1 million bunk token all right it's
531. 23:15time to pick the winner of the last
532. 23:20video and the winner
533. 23:22is oh it CH me number nine number nine
534. 23:27shim
535. 23:30are you okay so let's pick another
536. 23:33one all right so it says really liking
537. 23:36these strategy videos thank you so much
538. 23:38for your comment please reach out to me
539. 23:39so that I can send you your bunk tokens
540. 23:41thanks for watching and happy Trading
541. 23:44[Music]