How I solved my biggest pixel art problem | Pixel Perfect — Transcript

Full transcript

1. 0:04This is Callous.
2. 0:07And now Callous has a big issue.
3. 0:10They're trapped in a static world. One
4. 0:13where the bounds of their existence is
5. 0:15limited by what we can see.
6. 0:20In the series so far, all of the
7. 0:22examples we've looked at involved scenes
8. 0:24with a motionless camera.
9. 0:26>> [music]
10. 0:27>> If we want to build worlds that people
11. 0:29can explore, we need some way of moving
12. 0:32our viewport in a pixel-conscious way.
13. 0:34>> [music]
14. 0:36>> Simply moving the camera as you would in
15. 0:39a traditional game has issues. The
16. 0:41low-resolution grid that our engine runs
17. 0:44off cannot handle the high-resolution
18. 0:46movement of our camera.
19. 0:50In today's episode of Pixel Perfect, I
20. 0:52want to build a camera system that can
21. 0:54handle the responsibilities of moving
22. 0:56through our worlds, dealing with
23. 0:58sub-pixel movement, camera rotation, and
24. 1:01interactive gameplay systems.
25. 1:06If you enjoy the episode, a like and
26. 1:08subscribe always goes a long way in
27. 1:10helping the channel. It's free, and you
28. 1:12[music] can always change your mind at
29. 1:14any time. Thank you, and let's get into
30. 1:17it.
31. 1:28To understand what we're trying to
32. 1:30solve, let's make the problem clear.
33. 1:32>> [music]
34. 1:32>> We have a high-resolution output that
35. 1:34the user sees, and a low-resolution
36. 1:36pixel scene [music] that needs to be
37. 1:38rendered. The camera and pipeline is
38. 1:40sort of the arbiter between these two
39. 1:42endpoints,
40. 1:43>> [music]
41. 1:43>> and needs to decide how to translate the
42. 1:45source image.
43. 1:47Now, we covered this partially in the
44. 1:49very first episode of the series when
45. 1:51[music] looking at point samplers, but
46. 1:53in those examples, it was clear how the
47. 1:55source can be scaled up to full
48. 1:57resolution.
49. 1:58But now [music] we want to look at
50. 2:00moving the camera.
51. 2:01And this causes pixels to travel across
52. 2:03boundaries, [music]
53. 2:04which introduces new issues.
54. 2:09To take a quick detour to look [music]
55. 2:11at some existing examples, we can sort
56. 2:13of categorize pixel art games into two
57. 2:15distinct buckets. That is pixel perfect
58. 2:18[music] games. These are which the
59. 2:21underlying source image is rendered to a
60. 2:23low resolution grid, and pixels can only
61. 2:26move between cells on that grid.
62. 2:28>> [music]
63. 2:28>> And the second bucket is sub-texel.
64. 2:31These are games which the assets are
65. 2:33drawn as pixel art, but they exist
66. 2:35[music] in the world using typical
67. 2:37floating point transforms and can move
68. 2:39smoothly anywhere on the screen. [music]
69. 2:42In our project, we're firmly in bucket
70. 2:44one. We have a low resolution grid,
71. 2:47>> [music]
72. 2:47>> and objects can only communicate their
73. 2:49pixels by writing their colors to these
74. 2:52limited cells.
75. 2:54This is an important distinction because
76. 2:56sub-texel games
77. 2:57>> [music]
78. 2:57>> do not have to worry about the issues
79. 2:59mentioned previously.
80. 3:01Let's have a look at the magnitude
81. 3:02[music] of the problem actually in our
82. 3:04engine.
83. 3:06We can see here a typical example of the
84. 3:08type of sprite we might see in a pixel
85. 3:10art game.
86. 3:12>> [music]
87. 3:12>> When we start to move this sprite across
88. 3:14our screen, the issue is immediately
89. 3:16obvious. We've lost the coherency of the
90. 3:19original sprite, and we see this shimmer
91. 3:21that leads to an unstable image.
92. 3:24And this is due to pixels [music] not
93. 3:26uniformly crossing the grid boundaries,
94. 3:28which results in the squash and stretch
95. 3:30[music] effect we see in this demo. To
96. 3:32mitigate these floating point issues,
97. 3:34>> [music]
98. 3:35>> we can look at locking the camera to the
99. 3:36pixel grid.
100. 3:42Implementing this is a relatively
101. 3:44straightforward task. [music] If we can
102. 3:46calculate the size of a single pixel in
103. 3:48the high resolution output, we can then
104. 3:51use that information to ensure the
105. 3:53camera is always positioned at a
106. 3:55multiple of that value using a rounding
107. 3:58function. [music]
108. 4:01Now, this is good news for our horse
109. 4:03callus. [music] With the grid snapping
110. 4:05implemented, we're able to move around
111. 4:07the world and explore [music] the space
112. 4:09without the extreme shimmer we saw in
113. 4:10the demo.
114. 4:13>> [music]
115. 4:14>> When we enable the same grid snapping as
116. 4:16we saw in our main scene, we can witness
117. 4:18the issue more clearly. We see that the
118. 4:21shimmer no longer exists, [music] but
119. 4:23now the camera is restricted to move in
120. 4:25discrete blocky steps. And the lower the
121. 4:28resolution, the worse this becomes.
122. 4:32Our camera no longer has the ability to
123. 4:34move smoothly. [music]
124. 4:35This blocky motion really doesn't feel
125. 4:37great to play, and we'll need to come up
126. 4:40with a way to have a grid aligned camera
127. 4:42whilst maintaining [music] sub-texel
128. 4:44motion.
129. 4:47The first part of solving any problem is
130. 4:50identifying the source of the issue.
131. 4:52>> [music]
132. 4:52>> When we rounded the camera position to
133. 4:54the grid, we introduced a discrepancy
134. 4:56between where the camera is and where
135. 4:58the camera renders. We can capture this
136. 5:00loss by storing the difference between
137. 5:02the position and its rounded location
138. 5:05into a new variable.
139. 5:11Originally, we identified that we
140. 5:12couldn't move the camera smoothly
141. 5:14because of this high-to-low resolution
142. 5:16mismatch.
143. 5:17>> [music]
144. 5:18>> But surely, if the movement was computed
145. 5:20after upscaling, high-to-high resolution
146. 5:23should not display any of these issues.
147. 5:26>> [music]
148. 5:29>> So, let's create a new render texture
149. 5:30and copy over the result of our scene
150. 5:33camera into this dedicated RT.
151. 5:37We'll establish now a secondary game
152. 5:39camera alongside the one currently
153. 5:41rendering our scene. And this new camera
154. 5:44has one job, and that's to render out
155. 5:46this new texture created from our first
156. 5:48pipeline.
157. 5:50The scope of this new camera is just a
158. 5:52raw image, which sits perpendicular to
159. 5:54the view direction. [music]
160. 5:56We disable any additional render passes
161. 5:58or post-processing, as this is handled
162. 6:01when the scene is rendered by the main
163. 6:03camera.
164. 6:05Now, reintroducing the stored loss
165. 6:07variable from before,
166. 6:09>> [music]
167. 6:09>> we can offset the transform of this RT
168. 6:12plane by the vector calculated in the
169. 6:14rounding step. And so, every frame our
170. 6:17texture represents [music] the position
171. 6:18of our render camera minus the
172. 6:20discrepancy lost when rounding.
173. 6:23And this might be a little difficult to
174. 6:25conceptualize, but essentially our view
175. 6:27of the scene is now able to move
176. 6:29smoothly anywhere in the world,
177. 6:31>> [music]
178. 6:31>> and at the same time the render camera
179. 6:33can only be positioned on discrete grid
180. 6:36boundaries.
181. 6:42Now, Callus is completely free. We can
182. 6:45move around the world unimpeded by
183. 6:47jitter or snapping.
184. 6:48>> [music]
185. 6:50>> Whilst the image isn't entirely
186. 6:51shimmer-free, as not all surfaces are
187. 6:53perpendicular with the camera, compared
188. 6:55[music] to the old method, the game feel
189. 6:58is dramatically improved.
190. 7:00>> [music]
191. 7:02>> Our camera system is now able to
192. 7:04smoothly handle translations and
193. 7:06movement. But potentially, we don't want
194. 7:08to be limited to just a single camera
195. 7:10angle when making a game. So, the last
196. 7:13thing I want to look at today is
197. 7:14handling rotation
198. 7:16>> [music]
199. 7:16>> and how we might orbit the camera around
200. 7:18a central object.
201. 7:23From my current understanding, rotation
202. 7:25is somewhat of an unsolvable problem.
203. 7:27The issues we mentioned earlier of pixel
204. 7:30crawl across grid boundaries cannot be
205. 7:32avoided when dealing with rotation.
206. 7:37A fairly intuitive way to see this is to
207. 7:39imagine the camera from the pixel's
208. 7:41point of view.
209. 7:42>> [music]
210. 7:43>> When the camera rotates, we can imagine
211. 7:45the pixel as a projection onto the
212. 7:47surface of a sphere.
213. 7:52The position of that pixel on the sphere
214. 7:55is some cosine of the view vector.
215. 7:58And from there, you can see that for a
216. 8:00screen of pixels, [music] not all of
217. 8:02these values can live at fixed grid
218. 8:04intervals.
219. 8:07If we try to deploy our same tricks from
220. 8:09earlier, translating high to high
221. 8:11resolution, we would first need a
222. 8:13spherical or cube-based map of the scene
223. 8:16from multiple angles.
224. 8:17>> [music]
225. 8:18>> And my intuition feels we couldn't get
226. 8:19away with potentially six additional
227. 8:22cameras without compromising on the
228. 8:24vision of a game.
229. 8:27So, there's no way around it. We have to
230. 8:29bite the bullet and implement a standard
231. 8:31camera rotation, knowing that jitter
232. 8:33will occur.
233. 8:34>> [music]
234. 8:35>> As we've seen previously, a rotation
235. 8:37matrix or comparatively an inbuilt
236. 8:40[music] function call can implement this
237. 8:42trivially.
238. 8:44Depending on the compression of this
239. 8:46video, you'll see the results are
240. 8:48somewhat polarizing.
241. 8:50I'm keen to hear what people think about
242. 8:52rotation in general.
243. 8:53On one side, you might feel it's part of
244. 8:55the [music] charm, and once you play for
245. 8:57a while, you don't notice it.
246. 8:59And others may feel that the jitter is
247. 9:01too distracting to the presentation.
248. 9:03And this potentially throws into
249. 9:05question the type of game you might want
250. 9:07to make with this tech.
251. 9:10Similar to our approach in the God Ray
252. 9:12episode, we have a few non-pixel
253. 9:15techniques that we can look at to remedy
254. 9:17this, but at the expense of breaking the
255. 9:19faithfulness of the aesthetic.
256. 9:22Blur is perhaps a controversial topic,
257. 9:25but by applying a small amount to
258. 9:27average out neighboring pixels, we can
259. 9:29mask some of the artifacts. But the
260. 9:31smudgy look that this adds is quite the
261. 9:34departure from our original crisp image.
262. 9:37A different attempt at solving this is
263. 9:39to apply a global dither across the
264. 9:41entire screen. The idea being that our
265. 9:44brains have a harder time noticing pixel
266. 9:46crawl in a sea of different values.
267. 9:49This is a bit more radical, but it seems
268. 9:51to be effective if [music] we can
269. 9:52tolerate the slight washed out look it
270. 9:54gives when rotating.
271. 9:57And the last technique is to steer into
272. 10:00the jitter and make it a part of the
273. 10:01style. Since the resolution in our new
274. 10:04camera system is just a parameter, we
275. 10:06can increase the pixelation amount when
276. 10:08rotating the screen to alias areas of
277. 10:11jitter.
278. 10:12This, in addition to other effects, can
279. 10:14really create a stylized look where it
280. 10:17may be easier to get away with
281. 10:18instability if the tone of the game
282. 10:21matches the virtual look.
283. 10:24These three techniques don't at all feel
284. 10:26like solutions to our issue, but in
285. 10:27general, it does highlight that we can
286. 10:29sometimes improve the results of a
287. 10:31technique by finding ways to remedy
288. 10:33their weaknesses.
289. 10:38And that concludes the content for this
290. 10:40episode. We've looked at dealing with
291. 10:43our original low-to-high resolution
292. 10:45problem, how we can move the camera
293. 10:47smoothly aligned with a grid, and ways
294. 10:50we might try to deal with rotational
295. 10:51[music] artifacts.
296. 10:54Let me know what you think about the
297. 10:55results in the comments, and I'm curious
298. 10:57[music] to hear ideas on how we might
299. 10:59improve the result.
300. 11:01Thank you so much for all the support
301. 11:03over on Patreon. [music]
302. 11:04And if you made it this far in the
303. 11:05video, a like and subscribe goes a long
304. 11:08way to help out the channel.
305. 11:10>> [music]
306. 11:12>> As always, thanks for watching.
307. 11:39>> Mhm.