Effects of ipRGC on Color Perception based on Display-Based Color-Matching Task Kyosuke Ota 1, Kota Akiba 1, Midori Tanaka 2 and Takahiko Horiuchi 1 1 Chiba University, Graduate School of Science and Engineering, 1-33, Yayoicho, Inage-ku, Chiba-shi, Chiba, 263-8522, Japan 2 Chiba University, Graduate School of Global and Transdisciplinary Studies, 1-33, Yayoicho, Inage-ku, Chiba- shi, Chiba, 263-8522, Japan Abstract Intrinsically photosensitive retinal ganglion cells (ipRGCs) affect pupillary light reflex and circadian rhythm regulation. Recent studies have reported that they affect visual perception, particularly brightness perception, and their effect on color perception has also been gradually reported. In this study, we performed a color-matching task on a display device to verify the effect of ipRGC on color perception. Over a year, one participant performed 310 color matching sessions, day and night, by central and peripheral vision. Three blue colors with high ipRGC absorption and three red colors with low ipRGC absorption were used in one session. The color matching results suggest that non-image-forming and image-forming functions interact with col-or perception. We built a regression model in which ipRGC acts on the LMS. We found that the models constructed for each hue explained the experimental results well. Keywords 1 ipRGC, color perception, modeling 1. Introduction At the beginning of the 21st century, a third photoreceptor, intrinsically photosensitive retinal ganglion cells (ipRGCs), which are distinct from cone and rod cells, were discovered. ipRGC is a specialized cell that contains the photoreceptor melanopsin. Since melanopsin is structurally similar to invertebrate opsin, and since opsin can signal the presence or absence of light in invertebrates, melanopsin is thought to per-form similar functions in vertebrates [1]. Previous studies have shown that ipRGC affects the pupillary light reflex and circadian rhythm regulation [2]. Circadian rhythms are biological rhythms that have a daily cycle and influence physiological phenomena such as hormone secretion, sleep, and thermoregulation. Morphologically and physiologically, ipRGCs can be classified into six subtypes, M1 to M6. M1 ipRGC act on non-image-forming functions, while M2 to M6 ipRGCs (non-M1 ipRGC) act on image- forming functions. M2 ipRGCs are larger than M1 ipRGCs. M3 ipRGC is morphologically similar to M2 ipRGC. M4 ipRGC has the largest and most complex dendrites of the subtypes. M3 ipRGC and M5 ipRGC are not as well-known as the other subtypes, but the M3 ipRGC size and complexity of the dendrites of ipRGC are similar to those of M2 ipRGC, and the characteristics of M5 ipRGC are similar to those of M4 ipRGC. M6 ipRGC is considered to have dendrites that may match both M2 ipRGC and M5 ipRGC [3, 4]. Figure 1 shows the spectral sensitivity of human photoreceptors [5]. The red, green, blue, yellow, and black lines represent L-cones, M-cones, S-cones, rods, and ipRGCs, respectively. The spectral sensitivities of each cross each other. Although it has been assumed that cones and rods influence visual perception, ipRGCs have also influenced visual perception, especially brightness perception [6]. Recent reports have cited that ipRGCs influence brightness and color perception. A previous study showed that ipRGC might affect The 11th Colour and Visual Computing Symposium, September 8–9, 2022, Gjøvik, Norway EMAIL: kyosuke1021@chiba-u.jp (K. Ota); midori@chiba-u.jp (M. Tanaka); horiuchi@faculty.chiba-u.jp (T. Horiuchi) ORCID: 0000-0002-4651-4942 (M. Tanaka); 0000-0002-8197-6499 (T. Horiuchi) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr ks hop oceedi ngs ht I tp: // ceur - SSN1613- ws .or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) color perception by color matching, which was performed by juxtaposing a color patch and its reproduced color on a display [7]. To verify the effects of ipRGCs on color perception, this study per- formed a color-matching task on a display device during different time periods in the central and peripheral vision, where ipRGCs are relatively scarce and abundant, respectively. This is because M1 ipRGCs affect non-image-forming functions, and non-M1 ipRGCs affect image-forming functions. By repeating the experiment, we aim to build a color reproduction model that considers the influence of ipRGC on color perception. Figure 1: Spectral sensitivity functions of the cone, rod, and ipRGC. 2. Experiments We performed color matching experiments on a display device to examine the effect of ipRGC on color perception. To examine the effects of M1 ipRGC and non-M1 ipRGC on color perception, we performed color matching at different times of the day for peripheral vision, which is considered to have relatively high ipRGC, and central vision, which has relatively low ipRGC. 2.1. Experimental Method The color matching task consisted of repeated color matching of a large circle and a small circle with the same center displayed on the display, where the color of the large circle is targeted and the color of the small circle is manipulated. The color matching operations were performed by manipulating H (Hue), S (Saturation), and V (Value Brightness) using a keyboard. Three blue colors with high ipRGC absorption, [190, 200, 220], [210, 200, 220], [230, 200, 220], and three red colors with low ipRGC absorption, [5, 200, 220], [325, 200, 220], [345, 200, 220] were used in one session. Figure 2 shows each color stim-ulus and its spectral distribution. ipRGCs are abundant at viewing angles of 7°–10°. To verify the effect of non-M1 ipRGCs on color perception, the experiment was divided into two viewing conditions, central and peripheral. As shown in Fig. 3, the small and large circles were designed to have viewing angles of 0°–1° and 1°–3°, respectively, for the central viewing experiment, and 0°–7° and 7°–10°, respectively, for the peripheral viewing experiment. Tkinter, a Python GUI tool, was used to dis-play the experimental stimuli, and colorsys [8] was used for the conversion between RGB and HSV. The initial values of the small circles in the color matching were randomly set to H: ±15, S: ±35, and V: ±35 from the target large circle value. The HSV controls were assigned to the keyboard so that they could be manipulated arbitrarily while checking the experimental stimuli on the display. H (Hue) was set to range from 0 to 359, with counterclockwise changes when advanced in the + direction and clockwise changes when advanced in the − direction. For S (saturation) and V (value brightness), the range 0–255 can be manipulated with “S”:−10, “D”:−1, “F”:+1, “G”:+10 for S (saturation) and “X”: −10, “C”: −1, “V”: +1, “B”: +10 for V (value brightness). If the color matching process was interrupted and the participants wanted to start over from the initial value, they could return to the initial value by pressing “0”. At the end of color matching, they could enter “P” to move on to the following color matching process. The experiment is completed when all six colors are color matched. The experimental environment was a dark room, and the display was a Microsoft Surface Pro4 with a screen size of 12.4 inches. The color profile was "surface-srgb-enhanced.icm". To confirm the effects of both M1 ipRGC and non-M1 ipRGC on color perception, we performed color matching at 13 h and 21 h in the noon and night, respectively, in central and peripheral vision, with a constant waking time in consideration of the circadian rhythm. Dark acclimatization was required before the experiment began because the experiment was conducted in a darkroom environment. One of the authors with normal color vision participated in the experiment. (a) [H, S, V]=[5, 200, 220] (b) [H, S, V]=[190, 200, 220] (c) [H, S, V]=[210, 200, 220] (d) [H, S, V]=[230, 200, 220] (e) [H, S, V]=[325, 200, 220] (f) [H, S, V]=[345, 200, 220] Figure 2: Experimental stimuli and their spectral distributions. (a) Central (b) Peripheral Figure 3: Central and peripheral visual stimuli used in the experiment. 2.2. Experimental Method To obtain a large amount of data, 310 color matching runs were conducted from April 2021 to March 2022. Table 1 shows the results of the day and night color matching for central and peripheral vision. Table 1 shows that the color matching results for the central vision are worse than those for the peripheral vision, regard-less of the day, night, hue, or any other comparison between them. This result indicates that the central vision has a lower color discrimination ability than the peripheral vision in this experimental design. Figure 4 is a boxplot of the color difference between large and small circles, showing that the color difference between large and small circles at H=230 and 345 is distributed over a wider range than in other hues. Significant difference tests were performed on the results obtained. Since the significant difference test is for the color difference, which takes only positive values, only the Wilcoxon rank sum test was performed without a normality test. Table 2 shows the combinations and hues for which significant differences were confirmed due to the significant difference test with ΔE00. Table 2 shows a significant difference between central and peripheral vision in daytime for all blue colors, but not for all blue colors between central and peripheral vision in nighttime. This suggests that the characteristics of color perception may differ between day and night and that both M1-ipRGC and non-M1 ipRGC may influence color perception. Table 1 Average color difference ΔE00 of color matching between large and small circles. Central Peripheral Hue(H) Noon Night Noon Night 5 0.281 0.310 0.118 0.224 190 1.103 1.209 0.789 1.001 210 1.865 1.888 1.414 1.673 230 1.437 1.350 0.841 0.844 325 1.072 0.997 1.056 0.916 345 1.196 1.269 1.100 0.847 average 1.159 1.171 0.887 0.918 (a) Central, Noon (b) Central, Night (c) Peripheral, Noon (d) Peripheral, Night Figure 4: Boxplot of color difference between large and small circles. Table 2 Significant difference test combinations and the hues for which significant differences were confirmed for those combinations. Combinations Hue(H) Central and peripheral vision in the noon 5, 190, 210, 230 Central and peripheral vision in the night 5, 190, 230, 345 Noon and night in central vision Non Noon and night in peripheral vision 5, 345 3. Modeling Sec. 2.2 showed that the color perception might be affected for both M1 ipRGC and non-M1 ipRGC. Previous opinions for a ganglion cell reported that ipRGC acted after absorbing the tristimulus values. Therefore, the modification formula of LMS obtained by correcting each value of LMS with ipRGC absorption rate was derived by regression. 3.1. ipRGC absorptivity Since a measure of the effect of ipRGC is needed, the ipRGC absorptivity is incorporated into the model as a measure of the effect of ipRGC. The ipRGC absorptivity was determined based on the spectral sensitivity of ipRGC and the spectral distribution of the target color on the display device as follows: 𝑖𝑝𝑅𝐺𝐶 𝑠𝑡𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑑𝑜𝑠𝑒 (1) = 2(𝑆𝑝𝑒𝑐𝑡𝑟𝑎𝑙 𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 𝑜𝑓 𝑖𝑝𝑅𝐺𝐶 × 𝑆𝑝𝑒𝑐𝑡𝑟𝑎𝑙 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 𝑠𝑡𝑖𝑚𝑢𝑙𝑖), 𝑖𝑝𝑅𝐺𝐶 𝑠𝑡𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑑𝑜𝑠𝑒 (2) 𝑖𝑝𝑅𝐺𝐶 𝑎𝑏𝑠𝑜𝑟𝑝𝑡𝑖𝑣𝑖𝑡𝑦 = , ∑(𝑆𝑝𝑒𝑐𝑡𝑟𝑎𝑙 𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 𝑜𝑓 𝑖𝑝𝑅𝐺𝐶 ) where the spectral distribution of the target is normalized by the maximum spectral distribution within the six hues. The ipRGC absorptivities for each hue determined by Eqs. (1) and (2) are shown in Table 3. As shown in Table 3, the blue colors, which have their spectral peaks on the low wavelength side and spectral sensitivity of ipRGC, have a higher ipRGC absorptivity rate than the red colors, and the highest ipRGC absorptivity rate was at H = 190. Conversely, the lowest ipRGC absorptivity was observed in the red at H = 5. Table 3 ipRGC absorptivity for experimental stimuli. Hue(H) ipRGC absorptivity (× 10!" ) 5 8.08 190 260 210 204 235 172 320 70.5 345 21.3 3.2. Modelling The obtained ipRGC absorptivity was used to create a model that considers the effects of ipRGC in the LMS under the following three conditions. i. Linear model ii. Nonlinear model iii. Linear model by each color system Modified equations were developed using the models shown in Eq. (3) for the linear model and Eq. (4) for the nonlinear model. 𝑓(𝑥, 𝑦) = 𝑎𝑥 + 𝑏𝑦, (3) # # (4) 𝑓(𝑥, 𝑦) = 𝑎𝑥 + 𝑏𝑥 + 𝑐𝑦 + 𝑑𝑦 , where, variables a–d are coefficients, x is the value after color matching, and y is the ipRGC absorptivity. Multiple regression analysis was performed using "ideal LMS" as the dependent variable and "LMS after color matching" and "ipRGC absorptivity" as the independent variables for the model (i) building using a linear model. Equations (5) and (6) show the modified LMSs for central vision at noon and night, respectively, and Eqs. (7) and (8) show the modified LMSs for peripheral vision at noon and night, respectively. The coefficients of determination for each equation are shown in Table 4. Central Noon 𝐿$%&'( = 1.017 × 𝐿) − 1.982 × 𝑖𝑝𝑅𝐺𝐶, 𝑀$%&'( = 1.026 × 𝑀) − 10.681 × 𝑖𝑝𝑅𝐺𝐶, (5) 𝑆$%&'( = 1.005 × 𝑆) − 11.473 × 𝑖𝑝𝑅𝐺𝐶. Night 𝐿$%&'( = 1.017 × 𝐿) − 8.578 × 𝑖𝑝𝑅𝐺𝐶, 𝑀$%&'( = 1.025 × 𝑀) − 14.612 × 𝑖𝑝𝑅𝐺𝐶, (6) 𝑆$%&'( = 1.004 × 𝑆) − 8.381 × 𝑖𝑝𝑅𝐺𝐶. Peripheral Noon 𝐿$%&'( = 1.021 × 𝐿) − 2.710 × 𝑖𝑝𝑅𝐺𝐶, 𝑀$%&'( = 1.032 × 𝑀) − 13.446 × 𝑖𝑝𝑅𝐺𝐶, (7) 𝑆$%&'( = 1.015 × 𝑆) − 16.094 × 𝑖𝑝𝑅𝐺𝐶. Night 𝐿$%&'( = 1.011 × 𝐿) − 0.898 × 𝑖𝑝𝑅𝐺𝐶, 𝑀$%&'( = 1.019 × 𝑀) − 7.786 × 𝑖𝑝𝑅𝐺𝐶, (8) 𝑆$%&'( = 1.007 × 𝑆) − 8.874 × 𝑖𝑝𝑅𝐺𝐶. Here, 𝐿) , 𝑀) and 𝑆) represent the LMS values of the reproduced color on the display device after the color matching, including the impact of ipRGC. The variable ipRGC indicates the ipRGC absorptivity of the reproduced color. Table 4 Coefficient of determination for linear regression in LMS. L M S Central Noon 0.996 0.995 0.998 Night 0.994 0.994 0.995 Peripheral Noon 0.996 0.995 0.998 Night 0.994 0.994 0.996 The modified LMSs show that the absolute value of the coefficient on ipRGC absorptivity increases from L (the long-wavelength component) to S (the short wave-length component) regardless of day or night in peripheral vision, where ipRGCs are abundant, indicating that ipRGCs influence the color perception. In central vision, the absolute value of the coefficient on the ipRGC absorptivity of the modified MS is generally larger than that of L, although not as large as in peripheral vision. In the peripheral vision, the absolute value of the coefficient of ipRGC absorptivity is more significant at noon, suggesting that the effect of ipRGC on color perception may be larger at noon than at night. The coefficient of determination values for each equation shown in Table 4 indicate that the equations fit very well. Table 5 shows a heat map of the results of a model building under the three conditions. The areas highlighted in red are the points where the color difference could not be improved before and after modification. In the case of model (i), it was possible to consider the involvement of ipRGC in the modified equation, but the final modification result showed that the color difference was not improved in most cases. Similarly, model (ii) failed to improve color differences in most hues. In addition, model (iii) also improved the color differences more than models (i) and (ii), but it cannot be said that the modification was performed at a sufficient level. Comparing the results of modification for blue- and red-toned colors, the modification for mod-el (i) is equally successful in both cases, but for models (ii) and (iii), the blue-toned colors were better than the red-toned colors. This confirms the possibility to im-prove the color difference for blue colors with high ipRGC absorption by including a term that considers the ipRGC effect in the modified equation. However, none of the models could correct at a sufficient level. Even if model (iii) can improve better than the others, it is difficult to believe that they are doing so physiologically to process the color signals by entering the eye separately for each color system, as in the present model. Since it was confirmed that it is possible to improve the color difference using the modified formula by dividing the color system, we would like to work in the future to create a model with more intensity by considering the ipRGC absorptivity. Table 5 Heatmap of modification results. (i) (ii) (iii) H Central Peripheral Central Peripheral Central Peripheral Noon Night Noon Night Noon Night Noon Night Noon Night Noon Night 5 190 210 230 325 345 Avg. 4. Conclusions In this study, we examined the effects of M1 ipRGC and non-M1 ipRGC on color perception by repeating the color matching over a long period of time and discussed the results and model in the LMS. The color matching results showed that the central vision had worse color matching results than the peripheral, indicating that the central vision has a lower color discrimination ability. The results of the significant difference test showed that there was a significant difference between the central and peripheral vision at noon for all blue colors, but not for all blue colors between central and peripheral vision in night, indicating that the characteristics of color perception may differ between noon and night. This suggests that both M1 ipRGC and non-M1 ipRGC may influence color perception. In the field of circadian typology [9], the differences of the circadian typology affect the color confusion axis, suggesting an influence on color perception [10]. The involvement of M1 ipRGC in color perception needs to be further investigated. Since the experimental results indicated that ipRGC might affect color perception, we built a modified formula in LMS using the ipRGC absorptivity, an index that indicates the effect of ipRGC. As a result, although neither the linear nor the nonlinear model was able to improve the color difference well, it was able to im-prove the color difference better than the linear or the nonlinear model by modeling it separately for each color system. This suggests that the magnitude of the effect of ipRGC on color perception may vary depending on the spectral distribution of the stimuli and the absorptivity of the LMS. In future, we plan to create a more complex and accurate model than the one created in this study and conduct verification experiments using other colors with similar ipRGC absorptivity to verify the effect of the amount of ipRGC stimuli. 5. Acknowledgments This work was supported by JSPS KAKENHI (Grant Numbers 19K12038). 6. References [1] D.M. Berson, F.A. Dunn, and M. Takao, Phototransduction by Retinal Ganglion Cells That Set the Circadian Clock, Science 295 (2002) 1070-1073. doi:10.1126/science.1067262 [2] R.J. Lucas, S. Hattar, M. Takao, D.M. Berson, R.G. Foster, and K.-W. Yau, Diminished Pupillary Light Reflex at High Irradiances in Melanopsin-Knockout Mice. SIENCE 299(2003) 245-247. doi:10.1126/science.1077293 [3] T.M. Schmidt, S.-K. Chen, S. Hattar, Intrinsically Photosensitive Retinal Ganglion Cells: Many Subtypes, Diverse Functions, Trends in Neurosciences 34.11 (2011) 572-580. doi:10.1016/j.tins.2011.07.001 [4] L.E. Quattrochi, M.E. Stabio, I. Kim, M.C. Ilardi, P.M. Fogerson, M.L. Leyrer, D.M. Berson, The M6 cell: A Small-Field Bistratified Photosensitive Retinal Ganglion Cell, J Comp Neurol 527.1 (2019) 297-311. doi:10.1002/cne.24556 [5] CIE, CIE System for Metrology of Optical Radiation for ipRGC-Influenced Responses to Light, CIE, 2018. doi:10.25039/S026.2018 [6] T.M. Brown, S. Tsujimura, A.E. Allen, J. Wynne, R. Bedford, G. Vickery, A. Vugler, A. Lucas, R.: Melanopsin-Based Brightness Discrimination in Mice and Humans, Current Biology 22.12 (2012) 1134-1141. doi:10.1016/j.cub.2012.04.039 [7] K. Akiba, M. Tanaka, T. Horiuchi, Color Matching Consideration on the Effect of ipRGC for Color Reproduction on Display Device, Proc. 5th CIE Symposium on Colour and Visual Appearance (2020). doi:10.25039/x47.2020.PO39 [8] S. Storchaka, colorsys, 2013. URL: https://github.com/python/cpython/blob/3.7/Lib/colorsys.py. [9] A. Adan, S.N. Archer, M.P. Hidalgo, L.D. Milia, V. Natale, C. Randler, Circadian Typology: A Comprehensive Review, Chronobiology International 29.9 (2012) 1153-1175. doi:10.3109/07420528.2012.719971 [10] M.D. de Andrade, E.D. Cristino, L.B. Santos, A.R. de Oliveira, N.A. dos Santos, Daily variation of visual perception of colors: Preliminary studies, Psychology & Neuroscience 11.3 (2018) 238– 251. doi:10.1037/pne0000132