Color Science Explained — Transcript

Full transcript

1. 0:00Hi!
2. 0:00When we talk about image quality,  most people focus on resolution—the
3. 0:04number of pixels in an image,  which is important for sure.
4. 0:08But what about the qualities  of each one of those pixels,
5. 0:11such as bit depth, dynamic range, and color space?
6. 0:15These are some of the key elements  when it comes to color science.
7. 0:19Understanding color science and management  can fundamentally change how you capture,
8. 0:23generate, or process images.
9. 0:25That’s what we are going to be  talking about in this video.
10. 0:27Let’s take a look at each one of these one by  one: bit depth, dynamic range, and color spaces.
11. 0:33Bit depth refers to the number of bits used  to define each color component of pixels.
12. 0:38The more bits per pixel, the  more colors you can represent.
13. 0:41Let's take an 8-bit RGB image as an example.
14. 0:44It uses 8 bits for each of the  red, green, and blue channels.
15. 0:49This means we can have 256 values per  channel. When we combine these channels,
16. 0:54we get about 16.8 million  possible colors per pixel.
17. 0:58Sounds like a lot of colors, but it actually  provides very little room for editing.
18. 1:02When you try to color grade, adjust colors,
19. 1:05contrast, or exposure of your footage,  you can easily get banding artifacts.
20. 1:10That's why images used in professional  workflows typically involve at least 10 bits.
21. 1:15The bit depth determines how many  colors can be represented but it
22. 1:18doesn’t tell what colors and in what  range of brightness can be represented.
23. 1:22The range of brightness is  determined by the dynamic range,
24. 1:25which determines how dark and  how bright a pixel can get.
25. 1:28A high dynamic range is great, even  if you're not using an HDR display.
26. 1:32Because it lets you adjust the exposure after
27. 1:35the fact without losing detail  in the shadows or highlights.
28. 1:38When the lightness of a pixel is out  of boundaries of the dynamic range,
29. 1:41it gets clipped and loses detail as a result.
30. 1:44For example, I have a standard dynamic range image
31. 1:47on the left and a high dynamic  range (HDR) image on the right.
32. 1:51At first glance, they might look the same, because  this video itself is not high dynamic range.
33. 1:56But if we lower the exposure in  post, we'd see a big difference.
34. 2:00The HDR image still retains the  highlights. It had more dynamic
35. 2:03range than a display can show.  It had more room for editing.
36. 2:07This was not the case for the SDR image.  Look at all those highlights that were lost.
37. 2:12HDR images use more bits to  represent the higher dynamic range,
38. 2:15but even with the same number of bits  the dynamic range can be different.
39. 2:19A logarithmic curve, for example, is often used
40. 2:22to compress a wide range of luminance  values into a smaller bit-depth range.
41. 2:26This allows for more efficient  storage while preserving detail
42. 2:30in both the shadows and highlights. Which gives  us flexibility in color grading and correction.
43. 2:35This is also known as log  profile on digital video cameras.
44. 2:39Videos shot in log profile look washed out before  any color grading is applied. This is because
45. 2:44displays don't naturally interpret log profiles  correctly. Why? We’ll get back to that later.
46. 2:50To put this into context, let’s  first talk about color spaces.
47. 2:54The dynamic range determines the range of  brightness. The range of colors is determined by
48. 2:59the color gamut. Color gamut refers to the range  of colors which a device can capture or display.
49. 3:04And color spaces define how  those colors are represented.
50. 3:08In 1931, the International Commission on  Illumination introduced a three-dimensional
51. 3:12space representing all perceivable  colors, known as the CIE XYZ color space.
52. 3:19This color space provides a standard reference  against which color spaces can be defined,
53. 3:24measured, and converted to each other.
54. 3:27This CIE 1931 chromaticity  diagram is a two-dimensional
55. 3:30projection of that. It visualizes only the  chromaticity, independent of luminance.
56. 3:35It shows all colors visible to the human visual
57. 3:38system in this horseshoe shape  known as the spectral locus.
58. 3:42Along this curved edge, you see the wavelengths  of light. The straight line in the bottom, known
59. 3:47as the line of purples, on the other hand has no  wavelength. Because purple is not a “real” color!
60. 3:53Purple is not a real color in the sense that it  doesn't have a single wavelength of light. It’s
61. 3:58the way our brain perceives a mixture of red  and blue to make sense of what's being seen.
62. 4:03Obviously, the colors on this chart are  not the actual colors but are just an
63. 4:07approximation. Because our displays can't show  the full range of colors our eyes can see.
64. 4:11A monitor using the standard RGB  (sRGB) color space, for example,
65. 4:15can show the colors within  this triangle on the chart.
66. 4:18An RGB colorspace is defined by 3 components:  primaries, whitepoint, and transfer functions.
67. 4:24The primaries are the corners of the  triangle in this diagram. They set
68. 4:28the boundaries for the colors that can be  represented using a specific color space.
69. 4:32The whitepoint is a reference point that  defines what is considered "white" within
70. 4:37that color space. Typically, it’s based on  a standard illuminant, such as D65, which
71. 4:42approximates average daylight. The whitepoint  provides a common baseline for color balance.
72. 4:48Finally, transfer functions define  the relationship between the actual
73. 4:51light intensity and its digital representation.
74. 4:54Here’s what I mean by that.
75. 4:56Camera sensors perceive light linearly. Twice the
76. 4:59amount of light results in  twice the sensor response.
77. 5:03To a camera sensor, the difference  between no light and one light bulb is
78. 5:07perceived as more or less the same as the  difference between 49 and 50 light bulbs.
79. 5:12That's obviously not the case for human vision.
80. 5:15Switching the first bulb clearly makes a bigger  difference. Going from no light to some light is
81. 5:21a lot more noticeable than switching the 50th  light on, which barely makes any difference.
82. 5:26The just noticeable difference in  brightness follows a logarithmic scale,
83. 5:30meaning that our perception of changes is based  on relative rather than absolute differences.
84. 5:36This phenomenon is described by Weber's  Law and applies to many other senses.
85. 5:40A transfer function takes that into account.
86. 5:43It takes linear sensor data and compresses it  in a way that more bits are used to represent
87. 5:48lower light levels, where humans  are more sensitive to differences,
88. 5:52and fewer bits are used for higher light  levels, where the eye is less sensitive.
89. 5:56This type of transfer function is called an  Opto-Electronic Transfer Function (OETF).
90. 6:02Displays do the opposite of that. They take  in that compressed signal and map that back
91. 6:07to linear light output. This is called the  Electro-Optical Transfer Function (EOTF),
92. 6:12because we go from electrical input  signals to optical output light.
93. 6:17Transfer functions are typically expressed by a
94. 6:19power function with a gamma value  of around 2, commonly 2.2 or 2.4.
95. 6:25Gamma curves are similar to, but not the same as,
96. 6:28log curves. Log curves encode  a higher dynamic range.
97. 6:31You might wonder why gamma encoding  was adopted instead of simply using
98. 6:35monitors that invert a log profile.  It’s because of historical reasons!
99. 6:40CRT displays naturally follow  a nonlinear power-law curve,
100. 6:44which made gamma a more practical choice  for display technology at the time.
101. 6:48Unlike gamma encoding, logarithmic  encoding wasn’t designed to be tied
102. 6:51to a display technology. Instead, it was designed  to *represent* a high dynamic range efficiently.
103. 6:57Before we move on I think it's worth mentioning  some color spaces that are commonly used.
104. 7:01sRGB and Rec. 709 both share the same color  primaries, with different transfer functions.
105. 7:08sRGB is commonly used for still images, Rec.  709 is used for videos. Another common color
106. 7:14space is Rec. 2020, which has a wider  color gamut and is used for HDR content.
107. 7:19The color spaces we talked about so far are  considered RGB color spaces because they
108. 7:24define specific sets of RGB primaries.  There are other color spaces as well.
109. 7:29For example, CIELAB represents color  as three values: one for lightness,
110. 7:34one for green and red, and  one for blue and yellow.
111. 7:37Those color pairs are opponent colors.
112. 7:40Have you ever tried to imagine a  greenish red, or bluish yellow?
113. 7:44I don’t mean colors you get when you  mix them. I mean a single color that
114. 7:48looks red and green or blue  and yellow at the same time.
115. 7:52Quite hard, isn't it? That's because our visual
116. 7:55system perceives those colors as  directly opposite to each other.
117. 7:59CIELAB uses those color dimensions to  represent color rather than a mixture
118. 8:03of RGB to have a more perceptually  uniform space. Meaning that a change
119. 8:08in the values would correspond to a  similar perceived change in color.
120. 8:12There are also other color spaces that separate
121. 8:14brightness and color for different  purposes, such as video encoding.
122. 8:18YCbCr, for example, consists of a luma  component Y and chroma components Cb and Cr.
123. 8:25Since human vision is more sensitive to  luminance than color, chroma channels can be
124. 8:29subsampled and given less bandwidth than the luma  component without much loss in perceived quality.
125. 8:35This process is called chroma subsampling. That's
126. 8:38what the ratios you see in digital  media formats like ProRes 422 mean.
127. 8:42A 4:2:2 ratio indicates  horizontal chroma subsampling:
128. 8:46half the resolution in the horizontal axis.
129. 8:494:2:0 means half the resolution in both  axes, and 4:4:4 means no chroma subsampling.
130. 8:55Anyway, I digress, this falls more in the domain
131. 8:58of video processing and compression than  color science but it’s good to know.
132. 9:02Cameras typically capture more  data than displays can reproduce.
133. 9:06This is especially true for high-end cameras used
134. 9:08in production. Their color gamuts  are wider than any display can show.
135. 9:13So, the colors they capture  are adjusted, compressed,
136. 9:16or clipped to fit within  what a target display allows.
137. 9:19This brings us to the concepts of  scene-referred and display-referred data.
138. 9:24Scene referred data represents the real world,  rather than the light leaving the screen.
139. 9:28It is not meant to be viewed directly. Instead,  it aims to represent the physical properties of
140. 9:34the scene, such as the intensity and  color of the light in the real world.
141. 9:37Display-referred data, on the other hand,  represents what will be shown on a display.
142. 9:42So, how do we go from scene-referred  data to display-referred data?
143. 9:46That’s where color management comes into play.
144. 9:48Color management involves a series of steps,  including color space transformations,
145. 9:52tone mapping, gamut mapping, and Look-Up Tables  (LUTs), which map input colors to output colors.
146. 9:58The goal of this whole process is to ensure  that the final image is visually pleasing,
147. 10:03true to the creative intent, and respects  the constraints of the target display.
148. 10:07There is an industry-standard framework  that facilitates this process.
149. 10:11It is called the Academy Color  Encoding System, or ACES for short.
150. 10:15Having standard formats makes it easier  to work with different brand cameras and
151. 10:19even virtual cameras and CGI elements. It  allows them all to exist in the same space.
152. 10:25ACES has a scene-referred  color space, called ACES2065-1.
153. 10:30It has a linear representation, meaning  that it represents light just like the
154. 10:35camera sensors capture it. If you double  the amount of light, the values also double.
155. 10:39So, multiplying all the values by two  increases the exposure by one stop.
156. 10:44It has no log curves, but it still captures a wide
157. 10:46dynamic range because it has a high  bit depth, typically 16 or 32 bits.
158. 10:51ACES2065-1 uses primaries that contain every  visible color! They are called AP0 primaries.
159. 10:58On this diagram, AP0 primaries form  the smallest triangle that contains
160. 11:03the entire spectral locus. It even  covers some colors that we can’t see.
161. 11:07Camera manufacturers have  their own secret sauce to
162. 11:10whatever color science they use  to capture and process images.
163. 11:13ACES expects them to provide Input Device  Transforms (IDTs) to convert their camera's
164. 11:19raw data into this standardized,  scene-referred, linear color space.
165. 11:24Regardless of the input source—be it different  cameras, CG renders, or anything else—IDTs
166. 11:30transform it into ACES.
167. 11:32Once in this color space,  you can do all your work,
168. 11:34including color grading,  compositing, VFX, and CGI.
169. 11:39Finally, you use an Output Device  Transform (ODT) to convert your
170. 11:42work into the target format,  such as Rec 709 or Rec 2020.
171. 11:47Lastly, it’s worth mentioning some derivative  working spaces. ACEScc, ACEScct, and ACEScg, for
172. 11:54example, use AP1 primaries, which have a smaller  gamut than AP0, but avoid most non-visible colors.
173. 12:02Unlike ACES2065-1, ACEScc and  ACEScct are logarithmic color spaces,
174. 12:08which many professionals in  color grading are familiar with.
175. 12:12ACEScc provides a pure log encoding,
176. 12:15whereas ACEScct has a "toe" in the shadows,  which emulates the behavior of film.
177. 12:20ACEScg, on the other hand, is a linear  color space similar to ACES2065-1
178. 12:26but uses AP1 primaries. It is mainly  designed for CGI and compositing work.
179. 12:32Alright, that was pretty much it! I hope you found
180. 12:34it interesting and useful. Thanks  for watching, and see you next time.