I got my code 4x faster with the Claude Fable 5 (before the US government took it down) — Transcript

Full transcript

1. 0:00In the past few weeks, we have all heard
2. 0:02about Claude 3
3. 0:03finding security vulnerabilities in
4. 0:05source code, which is the main reason
5. 0:07why Anthropic was hesitating for
6. 0:09releasing this model. So, as an aside,
7. 0:11before I move on, I got to say that
8. 0:13right before I started editing this
9. 0:14video, I found out that Anthropic has
10. 0:17suspended the Claude 3 model. And the
11. 0:20reason apparently is the US government
12. 0:22claiming that this model has some
13. 0:24national security problems. Now, if you
14. 0:26ask me, this is more hype than truth
15. 0:28because we all know that Intel couple of
16. 0:30weeks, all other companies, including
17. 0:32Chinese AI companies, are going to
18. 0:35release a similar model that can do
19. 0:37this. So, I'm not sure if suspending
20. 0:39Fable 5 could have some real impact. But
21. 0:42anyways, back to the video. So, now that
22. 0:44we have Fable 5 model, it got me
23. 0:46thinking, what if I set it loose on my
24. 0:49own code base? But not to find security
25. 0:51vulnerabilities because, you know, my
26. 0:52code is always perfect, but for
27. 0:54optimizing my code base to improve its
28. 0:56speed. Now, in case this is your first
29. 0:58time on this channel, my code base is
30. 1:01about an algorithmic trading framework
31. 1:03called Jesse. So, that's exactly what I
32. 1:05did. I let it loose overnight, so it
33. 1:07kept working for like 8 hours straight
34. 1:10and burned a lot of tokens, but at the
35. 1:12end, it was able to improve the results
36. 1:14of backtest from two to four and
37. 1:16sometimes five times faster. And
38. 1:19improved backtesting speed can lead to
39. 1:21improved optimization speed and Monte
40. 1:23Carlo simulations. It can even improve
41. 1:25the speed of our workflow for machine
42. 1:27learning stuff and maybe even for
43. 1:29reinforcement learning. So, anyways, let
44. 1:31me show you the comments of Fable 5
45. 1:33model and to show you examples of how
46. 1:35much faster things got.
47. 1:38Now, I cannot show you the exact prompt
48. 1:40that I used because there's a special
49. 1:42kind of setup and there are things that
50. 1:44I need to keep private. But I will tell
51. 1:46you that when I was giving it the
52. 1:48prompt, I asked it to work in a loop and
53. 1:51that is to keep trying to improve the
54. 1:52performance, run tests, and ensure that
55. 1:55the results of the tests are exactly the
56. 1:57same as before because I did not want to
57. 1:59break anything in the API of the app. I
58. 2:02did not want to bother my users to do
59. 2:04some sort of migration just because of
60. 2:06performance boosts. I also asked it to
61. 2:08generate a report file for me once it
62. 2:10was done and that's exactly what it did.
63. 2:12So you see it generated a table for me
64. 2:14and here I can see the test that it took
65. 2:16and then we have the baseline which is
66. 2:18the time it took for each test to finish
67. 2:21before the new implementation and then
68. 2:23after the round one, round two and round
69. 2:26three. And here we can see how many
70. 2:28times it got faster. So in this case for
71. 2:30example, by the end it was 2.46 times
72. 2:33faster. In this one it was 4.51%
73. 2:36faster and the rest of the results are
74. 2:38pretty similar and in total everything
75. 2:41got 2.88 times faster. And then it
76. 2:44explained how it ensured the correctness
77. 2:46was guaranteed and things like that.
78. 2:48Needless to say also the project has 500
79. 2:51tests. So the fact that we are now
80. 2:53getting a green pass on all of those
81. 2:55tests is reassuring and if it wasn't
82. 2:58because of those things, I probably
83. 2:59would not have done such a huge
84. 3:01refactor. So in front of me I have this
85. 3:03typical uh strategy of Jesse. I have
86. 3:06chosen Binance Perpetual Futures as the
87. 3:08exchange. The route is USDT. The time
88. 3:11frame is 30 minutes and this is the
89. 3:14strategy. And for the duration I have
90. 3:15chosen approximately 4 and 1/2 years.
91. 3:18And to keep everything fair, I have kept
92. 3:21everything the same. The only thing that
93. 3:23is different is that the first two have
94. 3:26the fast mode disabled and the second
95. 3:28two have the fast mode enabled. So
96. 3:31before I show you the new implementation
97. 3:33or the comments of fable five, let's run
98. 3:36this strategy and also let's run it
99. 3:39using the fast mode being enabled so
100. 3:41that I can show you guys how fast things
101. 3:43are. So you see this is already too
102. 3:45fast. Like for most of you guys, you
103. 3:47probably don't even care about this and
104. 3:50while that was going the fast mode was
105. 3:52already done. So, the fast mode took
106. 3:543.19 seconds
107. 3:56and the one without the fast mode took
108. 3:5812.51 seconds. And the result of them
109. 4:01you see is exactly the same thing. So,
110. 4:04they are identical. Now, let's switch
111. 4:06from the master branch into the
112. 4:08performance back test branch. And as you
113. 4:10can see, we have the comments of the
114. 4:12Cloud Fable 5 model here. So, if I
115. 4:15scroll down, it's made approximately 17
116. 4:18or 22 commits, I didn't really count.
117. 4:20And all of them are a small things, so
118. 4:23they're not really big, but they kept
119. 4:25accumulating together. And the one thing
120. 4:27that is really outstanding out is the
121. 4:30usage of the Jesse Rust library. Now, in
122. 4:33case you're not familiar, Jesse Rust is
123. 4:35our very own built-in library and we use
124. 4:38it in order to move some of Jesse's
125. 4:40built-in codes into Rust, which we all
126. 4:43know is like the fastest option out
127. 4:45there. And then we just import it as a
128. 4:46simple function inside the code base.
129. 4:48And one of the main things that Fable 5
130. 4:50did is that it took more advantage of
131. 4:53Rust and it integrated those code inside
132. 4:56the code base. Now, the framework is
133. 4:57open source, so if you want to check out
134. 4:58the comments in more details, feel free
135. 5:00to do so on our GitHub. All right,
136. 5:02moving back, now let's run the same
137. 5:04simulations, but this time using this
138. 5:07new optimized code. So, let's run this
139. 5:09one and this one which also has the fast
140. 5:11mode on.
141. 5:12Now, I'm not even going to fast forward
142. 5:14the video just so you guys can see how
143. 5:16fast it is. And if I scroll down, you
144. 5:18see it took 1.98 seconds instead of 3.19
145. 5:23seconds. So, this is almost the double.
146. 5:25And for the one without the fast mode,
147. 5:27so it used to take 12.51 seconds, but
148. 5:31now it only took 4.24 seconds. So, this
149. 5:34is almost three times faster. Now, in my
150. 5:37other tests, it was sometimes up to four
151. 5:39times faster. So, this isn't even the
152. 5:41maximum gain that I saw.
153. 5:43And by the way, I have not made an
154. 5:44official release about this yet. So, if
155. 5:46you want to use the code that I just
156. 5:48showed you guys, you need to install
157. 5:50just it from the source code, and you
158. 5:52need to check out to this branch. Or,
159. 5:54you could just wait for me to make an
160. 5:55official release about it soon. Now,
161. 5:57this was a short and different kind of
162. 5:59video. I just wanted to share my
163. 6:01progress with you guys because I was
164. 6:02really excited about this performance
165. 6:04boost that we just got. Now, if you
166. 6:05enjoyed the video, or if you want to
167. 6:07watch my strategy tutorials, which I
168. 6:09keep releasing on this channel, make
169. 6:11sure that you're subscribed and hit the
170. 6:13bell button so you can be notified as
171. 6:15soon as possible. Thank you so much for
172. 6:16watching. I'll see you in the next one.