How to deploy your Jesse project into the production server for live trading — Transcript

Full transcript

1. 0:00usually people think it's okay to run
2. 0:02your algo trading sessions locally or
3. 0:04that it's even safer this way but over
4. 0:06the years i faced a lot of problems
5. 0:08which i could have avoided if instead i
6. 0:10used a remote server so in this video i
7. 0:12will show you why do you need the setup
8. 0:14where to get high quality and affordable
9. 0:16servers how to set it up for jesse and
10. 0:19how to make it secure so that only you
11. 0:21can access it let's get started
12. 0:23[Music]
13. 0:27there are many reasons but i'll give it
14. 0:29three number one servers have fast and
15. 0:31reliable internet connections for
16. 0:34example my headset servers usually have
17. 0:36a gigabit connection which is way faster
18. 0:38than my local one number two you don't
19. 0:40wanna worry about how to use your
20. 0:42computer while your algo is running
21. 0:44maybe you want to restart or shut down
22. 0:46your computer and if you're running a
23. 0:48live sessions then you can't do that
24. 0:50number three linux is the best os for
25. 0:52production but most of the development
26. 0:54machines are either windows or mac os so
27. 0:57by using a server you can still develop
28. 0:59on your operating system of choice but
29. 1:02run your production code on linux
30. 1:04[Music]
31. 1:08when it comes to choosing a cloud server
32. 1:10provider what we're looking for is a
33. 1:12good balance of the quality and the
34. 1:14price if you are living in the us i
35. 1:16highly recommend using digitalocean
36. 1:18there is starting server which is more
37. 1:20than enough for our use case it starts
38. 1:22from 5 bucks a month but if you use my
39. 1:24link you can get started for free the
40. 1:27other cloud provider that i personally
41. 1:28recommend and use is hessner their
42. 1:31starting plan offers more resources for
43. 1:33the same price and if you use my link
44. 1:35you will get 20 euros free credits
45. 1:38[Music]
46. 1:42i already have a history account so i'm
47. 1:44going to log in
48. 1:51projects on your hesnar account are like
49. 1:53folders on your machine they're used for
50. 1:55organizing your servers you can create a
51. 1:57new one or choose an existing one i'm
52. 2:00going to choose the tutorial project and
53. 2:03click on add server
54. 2:05i will change the location to germany
55. 2:08ubon 2204 is exactly the image that i
56. 2:11want
57. 2:13the first server plan is more than
58. 2:14enough for my use case so i will leave
59. 2:16that b
60. 2:18next i have to choose the ssh keys of my
61. 2:20computer so that the server can
62. 2:22recognize me in your case if your
63. 2:24account is new you probably have to
64. 2:27generate ssh keys and add it here
65. 2:30and at the end i will change the name of
66. 2:32this server to something that i can
67. 2:34remember such as tutorial 1.
68. 2:37the server is going to be ready in just
69. 2:39a few seconds all right now let's copy
70. 2:41the ip address of my server
71. 2:43open the terminal app and enter
72. 2:46ssh
73. 2:48root add sign and then paste the ip
74. 2:51address
75. 2:52because this is the first time we are
76. 2:53connecting to this server it's going to
77. 2:55ask us about whether or not we trust
78. 2:58this fingerprint just go ahead and enter
79. 3:00yes
80. 3:04[Music]
81. 3:07there are a couple of ways for setting
82. 3:09up jesse on a server i use docker
83. 3:11because it's easier to set up run and
84. 3:14also install future updates of jesse so
85. 3:17i'll start by installing docker itself
86. 3:19on my server this is a fresh vps so if i
87. 3:22run the docker compose command i should
88. 3:24see the command not found error which is
89. 3:26expected because we haven't installed
90. 3:28docker yet once again i will open the
91. 3:30documentation website and head over to
92. 3:32the docker section here there are two
93. 3:34commands for installing docker and
94. 3:36docker compose so let's copy the first
95. 3:39one
96. 3:41and paste it here
97. 3:43this should take a few seconds
98. 3:45now let's copy the second command which
99. 3:47is for installing docker compose
100. 3:53now if i run the docker compose command
101. 3:55one more time it should work
102. 3:59[Music]
103. 4:02if you're watching this i'm guessing you
104. 4:04already have a jesse project but for the
105. 4:06sake of this tutorial i will be creating
106. 4:08a new one to create a new jesse project
107. 4:10inside my local machine first i have to
108. 4:12make sure that i am inside the correct
109. 4:14directory
110. 4:16next from the documentation website i
111. 4:18will copy the command for creating a new
112. 4:20jesse project what this command is doing
113. 4:22is cloning a git repository from github
114. 4:25and specifying a custom name so i'll go
115. 4:28ahead and change the name to
116. 4:30tutorial
117. 4:31deploy
118. 4:33this command creates a new project so i
119. 4:35will see the into tutorial deploy
120. 4:38next as it is mentioned by the
121. 4:40documentation i will create my dot and v
122. 4:42file notice that the dot and v file is
123. 4:45ignored by git which means it doesn't
124. 4:47get pushed into github and that's
125. 4:49because the whole point of the data mv
126. 4:51file is to be unique to the environment
127. 4:53you're running it in in this case being
128. 4:55our production server because this
129. 4:57project was pulled from another github
130. 4:59repository by default it assumes that we
131. 5:01are the owners of the repository which
132. 5:03we are not so i'm just going to remove
133. 5:05git entirely from this code base so that
134. 5:08i can later push it into my own github
135. 5:11repository i can do that by typing rm
136. 5:14python rf.get
137. 5:18[Music]
138. 5:22it's always a good practice to keep your
139. 5:23code in a repository because it's easier
140. 5:26to track changes keep your backup and
141. 5:28maybe even share your code with others
142. 5:30if you need to so instead of uploading
143. 5:32my code directly into the server i'm
144. 5:34going to push my code into a github
145. 5:35repository and then pull it into the
146. 5:38server before i move on i'm going to
147. 5:39quickly create a new private depository
148. 5:42on my github account i'm already logged
149. 5:44in so i'm just going to click on the
150. 5:46plus button here and choose new
151. 5:48repository my repository name is going
152. 5:51to be tutorial
153. 5:52deploy and next make sure to set the
154. 5:55privacy settings into private otherwise
155. 5:57everyone can see your code and finally
156. 6:00click on create repository
157. 6:02all right now let's move back to the
158. 6:04terminal app to push our code repository
159. 6:07here are the commands for initiating a
160. 6:09git repository and pushing it into
161. 6:11github git init
162. 6:13which is for initiating git inside our
163. 6:15source code next i will run git add dot
164. 6:19what this will do is prepare all the
165. 6:21files for being pushed into github now
166. 6:24that all the files are added i'm going
167. 6:25to create my first commit
168. 6:27get commit python m and then i will pass
169. 6:31a message typically the first message is
170. 6:34always initial commit recently github
171. 6:36changed their primary branch name from
172. 6:38master into main so we need to tell our
173. 6:41local git to use main as the primary
174. 6:44branch instead of master we can do that
175. 6:46by typing git
176. 6:48branch
177. 6:50python
178. 6:51capital m main
179. 6:53next i will set my remote origin address
180. 6:56into what github just gave me by typing
181. 6:58git remote
182. 7:01add origin
183. 7:02let's quickly copy the origin from
184. 7:04github
185. 7:05and paste it here
186. 7:08now we can push by typing git push
187. 7:11hyphen u
188. 7:13origin
189. 7:14maine
190. 7:16if i move back to github and give the
191. 7:18page a refresh i should see my source
192. 7:20code being uploaded here because i set
193. 7:22the privacy settings of my repository to
194. 7:24private my server needs to first
195. 7:26authenticate to gain access to the
196. 7:28source code so if i try to run
197. 7:31git clone
198. 7:32and then paste in the address of my
199. 7:34repository here
200. 7:37it tells me that i do not have the
201. 7:39correct access rights
202. 7:41we can fix this by authenticating our
203. 7:43server
204. 7:44into our github account there are a
205. 7:46couple of ways for doing this my
206. 7:48recommended way is using ssh keys so i'm
207. 7:51gonna create a new ssh key for my server
208. 7:54so that i can add it into my github
209. 7:56account i can do that by typing ssh
210. 7:59python
211. 8:00keygen
212. 8:01press enter to skip the other steps now
213. 8:04i'm going to copy my ssh keys from this
214. 8:07location so i'm going to copy it and
215. 8:09write cat and paste it here next i will
216. 8:12select the whole thing copy it
217. 8:15open github again
218. 8:16go into my settings and then choose ssh
219. 8:20and gpg keys
220. 8:23new ssh key
221. 8:25paste it here the title doesn't really
222. 8:27matter but i will type in tutorial
223. 8:31so if i try to pull the repository one
224. 8:33more time it should work now by pressing
225. 8:35the up arrow button on my keyboard i can
226. 8:37use the previous commands again
227. 8:42let's check the existing files by typing
228. 8:44ls actually there are a couple of extra
229. 8:46files here because of the whole
230. 8:48installation that i just did so let's
231. 8:50remove those
232. 8:51i can do that by typing rm and then the
233. 8:54name of the file
234. 8:57the snap directory is also extras so
235. 9:00let's remove that by typing rm hyphen rf
236. 9:04snap
237. 9:05one more time and we're good
238. 9:10i will cd into my project
239. 9:13let's write the ls command one more time
240. 9:15to see if the source code is here
241. 9:17everything looks good now let's create
242. 9:19the dmv file by typing cp dot
243. 9:25now using the nano editor i will open my
244. 9:27data mv file
245. 9:29and the first thing that i'm going to do
246. 9:31is to change the default password from
247. 9:33test into something that no one can
248. 9:35guess
249. 9:36and the second line that i need to
250. 9:37change is to enter my license api token
251. 9:42i need to create a new one so i'm going
252. 9:44to open jesse website i'm already inside
253. 9:47the api token page but if you are not
254. 9:49you can find it here
255. 9:51from here i can create an api token
256. 9:54the name of it doesn't really matter i'm
257. 9:56just going to use tutorial
258. 9:58and then copy this api token and paste
259. 10:01it inside your dmv file press ctrl x and
260. 10:04then hit y to save the file we should be
261. 10:07good to go now so let's cd into the
262. 10:08docker directory and all that i have to
263. 10:11do for installing jse and inserting the
264. 10:14containers is to type
265. 10:16docker compose
266. 10:18up
267. 10:19if i were using docker inside my local
268. 10:21machine this should have been enough but
269. 10:23because it is on a remote server it is
270. 10:25important that i run the process in the
271. 10:27background so that after i close my
272. 10:29terminal app the session will continue
273. 10:31to work to execute docker in the
274. 10:33background all i need to do is to pass
275. 10:35heighten d at the end of it
276. 10:37this command is going to pull all the
277. 10:39required images from docker hub install
278. 10:42the live thread plugin and run the
279. 10:44containers for us this should take a few
280. 10:46seconds and maybe one minute so i'll
281. 10:48just skip the video
282. 10:51all right everything looks fine now
283. 10:53so now i should be able to open the
284. 10:55dashboard from my browser
285. 10:57the url of the dashboard is going to be
286. 11:00the ip address of my remote server
287. 11:02followed by the port which by default is
288. 11:059000
289. 11:06now i'm going to copy this wrong
290. 11:08password that i set earlier and paste it
291. 11:10here
292. 11:12everything looks fine and if i can start
293. 11:14a paper trading session that means
294. 11:16everything including the live thread
295. 11:18plugin is working fine
296. 11:22and yes everything is indeed working
297. 11:24properly
298. 11:26[Music]
299. 11:30on your hester account go into your
300. 11:32servers page
301. 11:35open the firewall section
302. 11:37click on create firewall
303. 11:40in order to make our server secure we
304. 11:42have to restrict the inbound traffic the
305. 11:45outbound traffic however should be open
306. 11:47because we want our application to be
307. 11:49able to communicate with the exchanges
308. 11:51that it wants and there is really
309. 11:53nothing to worry about there so we don't
310. 11:55have to add any rules to the outbound
311. 11:57section of the firewall
312. 11:59on the inbound section however we're
313. 12:01going to change things according to the
314. 12:03default configurations all ip addresses
315. 12:06whether version 4 or version 6 have
316. 12:08access to the 22 port which is used for
317. 12:10the ssh protocol
318. 12:12first find the ip address of your local
319. 12:14machine for that you can use a couple of
320. 12:16tools i'm using a website called
321. 12:19icanhasip.com but feel free to use any
322. 12:21other method that you like
323. 12:23after copying my ip address i will head
324. 12:25back to the head center page
325. 12:27and remove the default inputs by
326. 12:29clicking on them and pressing the delete
327. 12:31button on my keyboard
328. 12:33then i will paste in my ip address
329. 12:36the tcp protocol is fine the 22 port is
330. 12:40also fine
331. 12:41so
332. 12:42the first rule is done
333. 12:44for the second rule i will again paste
334. 12:46in my ip address
335. 12:48and change the protocol into tcp
336. 12:51and for the port i will enter 9000
337. 12:54everything looks fine now so i'll just
338. 12:56click on the create firewall button
339. 13:00so if i change my ip address and try to
340. 13:02enter the dashboard again
341. 13:05it shouldn't work
342. 13:07now if i change my ap address back to
343. 13:09what it was inside the firewall
344. 13:11and try again it should work just fine
345. 13:24[Music]
346. 13:28let's say that i want to make a few
347. 13:29changes to my strategy since i'm using
348. 13:31git as my version manager it's important
349. 13:33to never push changes directly to the
350. 13:35server so instead i'll push them to my
351. 13:38github repository first and then i'll
352. 13:40pull them into the server so i will open
353. 13:42the project inside my code editor
354. 13:46then i will look at my strategy from the
355. 13:48strategies directory example strategy
356. 13:51and init file
357. 13:53let's change a few things for example
358. 13:55return 2 here
359. 13:58and inside my go long method i will say
360. 14:02just submit one by order for the
361. 14:04quantity of one and the order price
362. 14:07should be the current price
363. 14:10to commit my changes and push it into
364. 14:12github i could either use the terminal
365. 14:15app or a git management app with a gui
366. 14:19a good example of that is called github
367. 14:22desktop which is what i usually use
368. 14:24myself it's free to use and also
369. 14:26supports all major operating systems but
370. 14:28in this video i'll be using the built-in
371. 14:30git management in whis code
372. 14:33as it shows us there has been one
373. 14:34modified file that hasn't been committed
374. 14:37yet to commit the file click on the plus
375. 14:39button here
376. 14:41and type a message for our commit i will
377. 14:44just say
378. 14:45update
379. 14:47strategy and press command or on windows
380. 14:50it will be control and enter to save it
381. 14:53to push this command into github i will
382. 14:55click on the three dots button
383. 14:58and choose
384. 14:59push
385. 15:02now if i open my github repository one
386. 15:04more time i should see the changes here
387. 15:07as you can see the last commit message
388. 15:09is called update strategy which is
389. 15:11exactly what i sent and it's been pushed
390. 15:1427 seconds ago
391. 15:16now let's move back into the server to
392. 15:18pull the changes
393. 15:19first i will make sure that i am inside
394. 15:21the correct directory
395. 15:26then inside the root of my jesse project
396. 15:29i will run git
397. 15:31pull
398. 15:33right it looks good but let me show you
399. 15:35how you can make sure that the last
400. 15:37commit is indeed the one you just pushed
401. 15:40to see the latest commits enter git
402. 15:43log
403. 15:44and the last one is indeed the message
404. 15:46that i gave it
405. 15:48whenever you start a new session with
406. 15:50jesse it will use the most recent
407. 15:52strategy code and there is no need to
408. 15:53restart jesse itself
409. 15:59[Music]
410. 16:03i'm constantly releasing new versions of
411. 16:05jesse containing improvements bug fixes
412. 16:08new features and so on so it's really
413. 16:10important to update your json
414. 16:11installation regularly don't worry
415. 16:14though it's really easy to do let me
416. 16:15show you first head over to the top
417. 16:17right corner of the dashboard and click
418. 16:20on the about button to open the about
419. 16:22section here you can see the jesse
420. 16:24version that you have installed and
421. 16:26below it you will find the version for
422. 16:27the live thread plugin if at the time
423. 16:29that you are watching this or in the
424. 16:31future a new version is out and you want
425. 16:34to update that one first make sure to
426. 16:36terminate your running sessions
427. 16:39then head back into the server and
428. 16:41because i'm using the docker setup all i
429. 16:43have to do is to enter the docker
430. 16:45directory once again
431. 16:49and first to stop the running containers
432. 16:51i will run docker compose stop
433. 16:55now if i head back to the dashboard and
434. 16:57give the page to refresh it shouldn't
435. 16:59work
436. 17:01now to pull the latest version of the jc
437. 17:03image which has everything installed on
438. 17:05it all i have to do is to run docker
439. 17:08compose
440. 17:09pull
441. 17:10since no newer version of the docker
442. 17:12image has been released since the last
443. 17:14time i pulled it it didn't download
444. 17:16anything but in your case if there has
445. 17:18been indeed a newer version you should
446. 17:20see a short progress bar
447. 17:23and lastly to start the containers again
448. 17:25i will run docker compose up
449. 17:28hyphen d
450. 17:29i'll give it a few seconds for the
451. 17:31containers to start again in the
452. 17:33background
453. 17:34and if i refresh the page one more time
454. 17:36it should work
455. 17:38if i open the about section i should see
456. 17:40the updated versions now
457. 17:44if you want to use the remote server not
458. 17:46just for live trading but also for a
459. 17:48strategy development then make sure to
460. 17:49watch my previous video if you're
461. 17:51interested in tutorials about algo
462. 17:52trading please subscribe to the channel
463. 17:55thanks for watching