Trading Competition: GPT vs Opus vs Sonnet vs GLM vs Qwen vs Minimax — Transcript

Full transcript

1. 0:00Hey guys, this is Salah. Today, we're
2. 0:01going to have LLM models trade live
3. 0:04accounts against each other. We're going
4. 0:06to use models from OpenAI, Anthropic,
5. 0:08and even open-source models from Asia.
6. 0:10And I won't just show you the result, I
7. 0:12will actually share with you the code
8. 0:14that I wrote for this, so you can
9. 0:15[music] even customize the code and run
10. 0:17it for yourself if you think it is
11. 0:19actually useful. And before we continue,
12. 0:21I got to say I am not a financial
13. 0:22advisor, and this video [music] is just
14. 0:24for education. All right, let's get
15. 0:26right into it.
16. 0:29Now, let's go through the code because I
17. 0:31made it super simple for you guys to
18. 0:32use. The most important class you need
19. 0:34to be aware of is this one. I named it
20. 0:36LLM Benchmark Base, and technically,
21. 0:38it's a Jaseci strategy. Why? Because it
22. 0:41is inheriting from this one. Now, the
23. 0:43minimum thing that you need to edit is
24. 0:46this value, which is the API key of
25. 0:48OpenRouter, which you can create at this
26. 0:51URL. And if you don't want to use
27. 0:53OpenRouter, let's say you want to use
28. 0:55OpenAI or or Anthropic directly, then
29. 0:58you're going to create API key on their
30. 1:01platform. And in that case, you also
31. 1:03need to change this URL here based on
32. 1:06their API, okay? And these API addresses
33. 1:08are actually public and very familiar.
34. 1:10Like, if you ask an AI, it will just
35. 1:12write it for you. You don't even need to
36. 1:14do any research for that. Now, notice
37. 1:16I'm using a placeholder value here
38. 1:18because I don't want to expose my own
39. 1:20API key. So, again, don't forget to
40. 1:22replace this with your own. Now, next,
41. 1:23we have some functions that I'm not
42. 1:25going to mention yet because we're going
43. 1:27to implement it in other classes, but
44. 1:29that's for later. But for now, this one
45. 1:32is something you need to be aware of.
46. 1:33So, you see, I'm giving the AI some
47. 1:37values, such as the current price, the
48. 1:40candles, the indicator values, but
49. 1:42specifically EMA for the period of 9,
50. 1:4521, 50, and the RSI, the ATR, and the
51. 1:49ADX, and then the status of the current
52. 1:51position, and the P&L of the current
53. 1:53position. And that's it, really. But, of
54. 1:56course, this is the part that you guys
55. 1:57can customize and the sky is the limit.
56. 1:59Like, you can feed it whatever kind of
57. 2:02data that you want. You can even feed it
58. 2:04some sentiment data. You can give it
59. 2:06some alternative data. You can give it
60. 2:07like the funding fee of the market.
61. 2:09Like, anything could be helpful here.
62. 2:11But, for this demo, this is what I'm
63. 2:13doing. And then I'm creating the prompt
64. 2:15that the LLM is going to see. Now, if
65. 2:18you change these values here, then
66. 2:20you're going to also need to change
67. 2:21these. But, I suggest that if you did
68. 2:24change these manually, then maybe ask an
69. 2:26AI to also update these ones for you
70. 2:28because it's a hideous task and AI can
71. 2:31write it very easily. Next, we have this
72. 2:33function where I'm giving three choices
73. 2:35to the LLM, so it can either go long,
74. 2:38short, or hold whichever position we are
75. 2:41in right now. May that be keeping the
76. 2:43current open position or even keeping
77. 2:45the current closed position and not
78. 2:46opening any new positions because, you
79. 2:49see, sometimes overtrading is what
80. 2:51causes these models to lose more money.
81. 2:53So, maybe trading less would be better.
82. 2:55But, anyway, so we're giving it three
83. 2:56options, but again, you can customize
84. 2:59this part and give it so many more
85. 3:01options. Next, we're calling the actual
86. 3:03API of the open router, which doesn't
87. 3:05really matter for you guys. And then we
88. 3:07are parsing the decision. We're also
89. 3:09logging it here, so this is going to
90. 3:11help us figure out what exactly the LLM
91. 3:14is doing. So, if something is going
92. 3:15good, we can know why. If it's going
93. 3:18wrong, we can again know why that's
94. 3:19happening. And next, we have the typical
95. 3:21just a strategy, right? So, we have
96. 3:23these familiar functions such as should
97. 3:25long and should short methods. And in
98. 3:27it, we're simply saying, "Is it decision
99. 3:29of the current LLM to go long? Then,
100. 3:31let's go long. If it's short, let's go
101. 3:33short."
102. 3:34And then, in case of the position
103. 3:36sizing, we're simply using 95% of our
104. 3:39available margin. So, we're not doing
105. 3:40really actual risk management here. And
106. 3:42then we're saying, "Once the position
107. 3:44opens, I'm going to submit the stop loss
108. 3:46and take profit." So, if it's a long
109. 3:48position, we're doing it like this and
110. 3:50then we're doing it the opposite way.
111. 3:52But, how are we doing it? Well, we're
112. 3:54simply using the ATR indicator and we're
113. 3:56simply using two times of the current
114. 3:59ATR value in the moment the position
115. 4:01opens. So, two times for the stop loss
116. 4:03and two times for the take profit. So,
117. 4:05that means we're going to have a risk
118. 4:06reward ratio of one. Again, this is
119. 4:09something that you can customize. Or,
120. 4:11for example, you could give the LLM more
121. 4:15decisions to make. So, instead of like
122. 4:17these three, maybe it should also be
123. 4:19responsible for making a decision about
124. 4:21your risk management or closing the
125. 4:23position. Maybe you don't want to use
126. 4:24ATR. Maybe you want to use something
127. 4:26else and you want the LLM to make that
128. 4:27decision. So, if that's the case, just
129. 4:29add that option and then here you need
130. 4:31to again take that decision using this
131. 4:34LLM decision property here and parse it
132. 4:37somehow. But, again, for this demo, this
133. 4:39is what we're going to go with because I
134. 4:41want to keep it as simple as possible.
135. 4:43All right, next we need to generate one
136. 4:45strategy per each model that wants to
137. 4:47compete here. Now, because we had this
138. 4:49base model here, every new strategy
139. 4:52needs to inherit from this one. So, not
140. 4:55from J Assistant Strategy anymore, okay?
141. 4:57So, in case we do this, all of them are
142. 4:59going to have access to these methods
143. 5:01here. And here I have one example of
144. 5:02that, which I made for Claude Opus 4.6.
145. 5:05And as you can see, it's inheriting from
146. 5:07this model and this is me importing
147. 5:09that. And there's just one method in it
148. 5:12and that is returning the model name and
149. 5:14this is it. Now, where do we get this
150. 5:16from? To do that, you're going to have
151. 5:17to open OpenRouter website and search
152. 5:19for whatever model that you want and
153. 5:22this key here is the one that we need.
154. 5:23So, click here, copy it and then go back
155. 5:26to the editor and paste it here. And
156. 5:28that's it. Now, in case you're new to J
157. 5:30Assistant, well, if you want to generate
158. 5:32a new strategy from J Assistant's
159. 5:33dashboard, you just have to go here,
160. 5:35click on this button or you could also
161. 5:37click on this one and give it a name and
162. 5:40click on the create button. And as you
163. 5:41can see, I've already done this for many
164. 5:44models. So, if I look for LLM, I have
165. 5:47one for GPT 5.2 Codex. And by the way,
166. 5:51at the time of this recording, for some
167. 5:53reason, Open Router doesn't have Codex
168. 5:565.3, which is the latest model. I don't
169. 5:59know why. So, if I click here, you can
170. 6:01see it. And then, I also made different
171. 6:04ones for Cloud Sonnet 4.6, MiniMax M
172. 6:072.5, GLM 5, which is actually a great
173. 6:10open-source model, and apparently it is
174. 6:12even beating Cloud Opus in many
175. 6:15benchmarks. So, this one's really great
176. 6:17and also super affordable. And then,
177. 6:19another one for Quan 3.5 Plus, and Cloud
178. 6:23Opus 4.6. So, these are the models that
179. 6:26are going to be competing here. We have
180. 6:28everything we can begin. So, for the
181. 6:30first live session, I'm going to choose
182. 6:32the exchange Binance Perpetual Futures.
183. 6:34For the symbol, I'm going to go with BTC
184. 6:37USDT. The time frame of 15 minutes, and
185. 6:39here's the strategy. And I've also
186. 6:41enabled the paper trade mode because I
187. 6:43don't want to be losing actual money
188. 6:45with these experiments. I've also
189. 6:47selected Telegram as my notification
190. 6:49channel. Now, we can click start for
191. 6:51each and every single one of them, or
192. 6:53since the latest release of Jesse, we
193. 6:54have this new overview page, which
194. 6:56allows us to start all of them
195. 6:58simultaneously.
196. 7:02There we go. We also have some new
197. 7:03metrics. So, for example, we can see the
198. 7:05total P&L of all the accounts combined
199. 7:07together, the current balance of all of
200. 7:09them, the start balance, the open
201. 7:11positions, open orders, and the total
202. 7:13number of trades that we have executed
203. 7:15since we started all of these things
204. 7:17together. We also have this equity curve
205. 7:19chart, which we can compare all of them
206. 7:20at the same time. We also have access to
207. 7:22some more details, which is going to
208. 7:24make more sense once these sessions
209. 7:26actually start taking trades. It's been
210. 7:28around 24 hours now, and the total P&L
211. 7:30is a pretty good number, I would say. We
212. 7:32can see the equity curve of all the
213. 7:34running sessions, and most of them look
214. 7:36to be in profits, so that's good. Next,
215. 7:39we can see more details of every single
216. 7:41one, and it seems like GLM 5 is beating
217. 7:43everything else. And after that, we have
218. 7:45GPT-5.2 Codex, and then MiniMax, and
219. 7:49Cloud Opus, Sonnet, and lastly, we have
220. 7:51Coin 2.5 Plus. So, let's go and check
221. 7:55them out one by one. So, here's Cloud
222. 7:57Opus. It doesn't seem that bad. And if
223. 7:59you zoom in, we can also see the sort of
224. 8:01trades it took. So, it's bought or sold
225. 8:04here. I don't know. It did both of them
226. 8:06in the same candle. But here, it went
227. 8:07short, it closed it here, it went long,
228. 8:10it closed it here, it went long again
229. 8:12here, and closed it here. So, this is
230. 8:15also pretty helpful. And we can also
231. 8:16check out the logs, of course, to see in
232. 8:18details what it did. And here are the
233. 8:20details of the trades that it took.
234. 8:21Next, we have Cloud Sonnet, which isn't
235. 8:23doing really great, to be honest. Next,
236. 8:25we have GLM 5, which seems to have done
237. 8:28the best. So, it is in 4.45%
238. 8:31profit. So, this is pretty great. If we
239. 8:33go here, we can also see the exact
240. 8:35trades that it took. So, everyone except
241. 8:37this one was in profit. And if we click
242. 8:40here, we can also see more details. So,
243. 8:41what was the exact entry price, exit
244. 8:43price, the size of it, how much fees we
245. 8:45paid, the exact P&L, when was it opened,
246. 8:47when was it closed, and for how long did
247. 8:50we exactly hold this position. So, these
248. 8:52are also the orders. If I click on them,
249. 8:54we can also see the details of that
250. 8:56order, of course. Next, we have GPT-5.2
251. 8:59Codex, which is also doing fine. Then
252. 9:02the MiniMax model, and lastly, the Coin
253. 9:043.5, which also has a positive position
254. 9:07right now. So, if we add this one to
255. 9:09this one, it's going to be sitting at
256. 9:10flat. So, I wouldn't say it's great, but
257. 9:12it wasn't horrible, either. So, again,
258. 9:14here are the details. Overall,
259. 9:15everything looks good. So, let's go and
260. 9:17come back later. So, it's day number
261. 9:19three, and everything is improving. The
262. 9:21total capital of the account is growing,
263. 9:23the total P&L and the percentage of it,
264. 9:25of course. And we can also take a look
265. 9:26at the equity curve of all the sessions.
266. 9:28They all look good. And if you take a
267. 9:29look at the P&L of every single session
268. 9:31separately, they're all good. But number
269. 9:33one, we no longer have GLM 5. It is now
270. 9:35MiniMax M2.5. and in the second place
271. 9:38there's GLM5. They're all doing good all
272. 9:41in all except I would say Sonnet isn't
273. 9:44doing great although it is in profit
274. 9:45like today but still like it's not as
275. 9:48good as the other ones. And if we take a
276. 9:50look at this session individually we can
277. 9:52also see the trades that it took. So
278. 9:54these all look good and for example in
279. 9:56this ranging period it didn't take any
280. 9:58trades. So this is actually something
281. 10:00that I would want to see usually right?
282. 10:02So if it's in a downtrend I want it to
283. 10:04take short trades. If it's in a ranging
284. 10:06market we don't want it to take any
285. 10:07trades and then if it's going up you
286. 10:09want it to open long positions. And it
287. 10:11seems to be doing just that. And this is
288. 10:12the equity curve it seems to be very
289. 10:14steady and good. We can also change its
290. 10:16values from let's say like 50 minutes to
291. 10:201 hour which gives us a better picture
292. 10:21of what's going on here. So all in all I
293. 10:23would say this is good. Although it does
294. 10:25have a losing position but it's okay
295. 10:27like even if you considering this is
296. 10:28still going to be sitting at like more
297. 10:31than 6% in profit which is great. All
298. 10:33right that's it for now. Let's go and
299. 10:34come back later.
300. 10:36All right it's day number four and
301. 10:38everything seems to be continuing just
302. 10:40fine. So except that GLM5 is again at
303. 10:43the first spot so that's great. MiniMax
304. 10:45is still doing great. Codex is also
305. 10:47fine. Opus seems to catch up with the
306. 10:50others. So this is good. I'm curious to
307. 10:52know if it's able to even bypass these
308. 10:55ones. And then we have a Sonnet which is
309. 10:57in last and so is the coin model. Let's
310. 10:59take a look at the details of GLM5. All
311. 11:02right still I would say this is good.
312. 11:03This is a good equity curve I'm happy.
313. 11:06Especially it's taking so many trades.
314. 11:08That's good. That's good like in in an
315. 11:10uptrend it definitely did good. In the
316. 11:12downtrend it did good. And you know in
317. 11:14in these kind of situations I think it's
318. 11:16perfectly normal for the model to to
319. 11:18lose because we just had an uptrend and
320. 11:20then it went into range so it's
321. 11:23perfectly fine for it. So all right
322. 11:25let's go back and come back later.
323. 11:27All right so I skipped a couple of days.
324. 11:30It is day number seven or eight I think.
325. 11:32And as you can see, all the money that
326. 11:34was made is almost gone. Why? Well, we
327. 11:37had this war began here between Iran and
328. 11:40Israel and the US. And as a result,
329. 11:43everything just went to, uh, you know.
330. 11:46So, the only model that is still sitting
331. 11:48in profit is J Lo 5. So, I kind of
332. 11:51appreciate that, but every other model
333. 11:53has lost money so far. To be fair, this
334. 11:56was not a normal price action that
335. 11:58happens all the time, right? It was
336. 12:00literally a war happening. And if the
337. 12:02price keeps going down, of course these
338. 12:03models are going to begin taking short
339. 12:05positions and they will probably start
340. 12:07making money again. But all in all, I
341. 12:09think this is good experience, which is
342. 12:11even if the model is making money like
343. 12:13in a week, it doesn't mean it's going to
344. 12:14make money like all the time. So, we
345. 12:16should always be careful, especially
346. 12:18with the position sizing and risk
347. 12:19management. All right, look at that. So,
348. 12:21it's been a few days later and the
349. 12:23models are still not doing great. Maybe
350. 12:25after this uptrend that has started
351. 12:27here, maybe we're going to see some
352. 12:29profits soon because the current
353. 12:30position, for example, is in a profit.
354. 12:32But overall, in this range, which again,
355. 12:35it was caused by the US-Iran war, things
356. 12:38went south. So, they are not looking
357. 12:39good. So, just look at this equity
358. 12:41curve. You see, this is not good. If you
359. 12:42take a look at the overview page, you
360. 12:44can see that's happening to all the
361. 12:45models. And in fact, at number one, we
362. 12:48have now Claude Opus 4.6. Although it is
363. 12:51not sitting in a profit, it just has
364. 12:53lost less money than all the other
365. 12:55models. All right, two more days is gone
366. 12:57and things are starting to improve at
367. 12:59least a little bit, but not too much.
368. 13:00So, overall, things are not super great,
369. 13:03but at least they are about to turn
370. 13:04around. So, I think if this trend goes
371. 13:07on, these are going to be in profit one
372. 13:09more time. All right, so we just saw the
373. 13:11result of these models trading for a
374. 13:13couple of days. And to be honest, it
375. 13:15actually seems better than what I
376. 13:16expected, even though they didn't end up
377. 13:18being in a profit. And let's not forget
378. 13:20that in this experiment, I gave very
379. 13:23limited amount of information to the
380. 13:25model. And as they say in machine
381. 13:27learning, garbage in, garbage out,
382. 13:28right? So, you need to ensure to give
383. 13:30the model as high quality data as
384. 13:33possible. But even with those things, we
385. 13:35should not forget that these LLM models
386. 13:37have not been made for such usage,
387. 13:39right? They've been made to write stuff
388. 13:41for us, not to make decisions like this.
389. 13:43Although, to be fair, these reasoning
390. 13:45models are doing a pretty good job, and
391. 13:48I did not even expect that. Now, there's
392. 13:50a lot more we can do. We can change the
393. 13:52data that we are feeding the model. We
394. 13:54can also run maybe back test for these
395. 13:56models and see the results of them for
396. 13:59many years ago. Now, if there's any kind
397. 14:00of variation of this experiment and you
398. 14:02want me to do it, please let me know.
399. 14:04Now, before we go, we're going to have a
400. 14:05giveaway. A random person who likes the
401. 14:07video, posts a comment, and subscribes
402. 14:08to the channel is going to win 1 million
403. 14:11bank token. All right, let's pick the
404. 14:12winner from the previous video.
405. 14:17And the winner is Thank you so much for
406. 14:19your comment. Please reach out to me so
407. 14:20I can send you your bank tokens.
408. 14:22>> [music]
409. 14:22>> Thank you so much for watching. I'll see
410. 14:24you in the next one.