Display resolution is an important indicator of image quality and is generally quantified as a pixel count with one pixel as the minimum element. Conventional displays consist of colored red-green-blue (RGB) subpixels. Recently, various subpixel layouts such as RGB and white (RGBW) and RGB and yellow (RGBY) have been used. Be-cause such subpixel layouts employ various subpixel rendering technique, a new method for evaluating resolution is required.
The guidelines in Sections 7.2 [1] and 7.8 [2] of the International Display Measurement Standard (IDMS) by the International Committee for Display Metrology (ICDM) proposed a method to evaluate the resolution capability of a display, with respect to its addressability, based on threshold contrast modulation (Michelson contrast) associated with grille patterns. Several studies on subjective spatial resolution have also been reported [3,4]. However, these conventional studies have not investigated the effect of different subpixel layouts on subjective spatial resolution.
In our previous work, we conducted evaluation experiments to examine the subjective spatial resolution for RGB, RGBW, PenTile RGBG (hereafter abbreviated as PenTile) subpixel layouts and confirmed that differences in subjective spatial resolution occur even at a viewing distance equivalent to the ITU-R [5] recommended angular resolution of 30 cycles per degree (cpd) [6,7]. Analysis of the modulation transfer function (MTF) of the display [8] suggested that the MTF could be an indicator of subjective spatial resolution [7]. However, these studies only consider three types of subpixel layouts, and further validation is needed for many more layouts. In addition, there was no significant difference in subjective spatial resolution between RGB layout with the highest MTF and PenTile layout with the second highest MTF. Therefore, it is necessary to further investigate the relationship between the difference in magnitude of the MTF and subjective spatial resolution using different subpixel layouts.
In this study, to further examine the relationship between display MTF and subjective spatial resolution, we design two new subpixel layouts with slightly higher and much higher MTF than RGB, and conduct visual evaluation experiments with RGB, PenTile and the new subpixel layouts.
In the experiment of Ref. [7], the RGB subpixel layout had the highest MTF. However, because there was no significant difference in subjective spatial resolution between the RGB and PenTile subpixel layouts which had the next highest MTF, we designed new subpixel layouts with a higher MTF than the RGB. Several new candidate subpixel layouts were created in reference to conventional RGB and PenTile subpixel layouts. By manually calculating the MTF of these structures and selecting subpixel layouts with higher MTF than RGB, two new types of subpixel layouts were designed-BRGRB and BRGBR525-as shown in Fig. 1. BRGRB was named after the order of subpixels, and 525 was added to BRGRB525 because the area ratio of R, G, and B subpixels is 5: 2: 5.
In this study, we computed the MTF for each subpixel layout based on the method proposed by Masaoka [8]. The MTF was calculated by computing the vertically averaged luminance line spread function (LSF) for the 2 Γ 2 black and white line patterns of each subpixel layout, performing Fourier transform and normalizing it with π = 0. For example, the LSF of the RGB subpixel layout is as follows:
where ππππ‘ is a rectangular function. π !"# , πΊ !"# , and π΅ !"# represent the luminance of the RGB subpixels, where π πΊπ΅ !"# = π !"# + πΊ !"# + π΅ !"# . Then, the πππΉ(π) is obtained as
where π is the spatial frequency in cycles per display pixel. Using this method, the MTF for each of the RGB, PenTile, BRGRB, and BRGRB525 subpixel layout were calculated respectively, and the results are shown in Fig. 2. The MTF values are in the order BRGRB525 β« BRGRB > RGB β« PenTile.
In addition to the conventional RGB and PenTile, and the newly designed BRGRB and BRGRB525, four types of vertical black and white grille patterns were used as experimental stimuli. Figure 3 shows the black and white grille patterns of each subpixel layout. In this experiment, the grille patterns were presented on a liquid crystal display (ColorEdge CG248-4K, Eizo Corp., Japan). Table 1 shows the display specification. Because this display has the RGB subpixel layout, it is not possible to display the grille patterns with other subpixel layouts. Therefore, as in Ref. [7], we regarded 12 Γ 12 pixels of the actual display as one virtual pixel and devised a method to virtually perceive different subpixel layouts by viewing one virtual pixel from a viewing distance of 12D-12 times the viewing distance D for one pixel-as shown in Fig. 4. To unify the one virtual pixel pitch, the virtual subpixel pitch of the RGB layout was set to 4 Γ 12 pixels of the display, 4 Γ 12 and 8 Γ 12 for PenTile, 2 Γ 12 and 4 Γ 12 for BRGRB, and 2 Γ 12 and 3 Γ 12 for BRGRB525. Therefore, the number of stripes in the experimental stimuli would be the same for different subpixel layouts, and the subpixel layouts would not be perceived when evaluating the stimuli.
Circular vertical grille patterns of 150 virtual pixels in diameter (1,800 real display pixels) were created as the experimental stimuli. The area ratio of the R, G, and B subpixels were different and the difference in brightness and chromaticity of the experimental stimuli affected the evaluation only for BRGRB525. Therefore, we adjusted the subpixel area Γ luminance values to be equal among the subpixel layouts using a spectroradiometer (CS-2000, KONICA MINOLTA, INC., Japan). For instance, the G subpixel in BRGRB525 is half the area of the G subpixel in the other subpixel layouts; hence, adjustments were made to double the luminance value. The luminance values of the black and white lines in the grille pattern were set to be 0.4 and 49.5 ππ/π * . The background color of the experimental stimuli patterns was supposed as the average of the black and white luminance in the grille pattern.
When the experimental stimuli were displayed, Moire fringes appeared at the boundary between the left and right edges of the stimuli and background. To prevent the stripes at the edge of the stimulus from becoming a response cue, gradient processing was applied to the boundary between the stimulus and background using the following procedure [7]. First, the center of the stimulus (720 virtual pixels in radius from the center) was considered as the observation area, and no gradient processing was applied. The radius was normalized within the target area, with the radius closer to the center of the target area being 0 and the radius closer to the background being 1. Then, the following equation was applied to the experimental stimuli:
where πππ₯ππ π£πππ’π represents the pixel value of the experimental stimuli. ππππ¦ represents the pixel value of the gray background, π is the radius (0 to 1) of the processed area after normalization, and πΎ refers to the gamma value of the display.
The participants employed for this study were four students with normal color vision of 20/20 or better (three with naked eyes and one with glasses). A total of 16 stimuli pairs were used-4 (RGB, PenTile, BRGRB, and BRGRB525 for left stripe) Γ 4 (RGB, PenTile, BRGRB, and BRGRB525 for right stripe). The experiments were conducted in a dark room at 20 cpd (3.77 m) and 30 cpd (5.66 m), based on the ITU-R [5] recommended angular resolution of 30 cpd (cycles per degree). Here, cpd was calculated based on one virtual pixel, so the viewing distance was 12 times the viewing distance for one pixel.
To ensure that the observers had binocular vision of at least 20/20, their visual acuity was tested each time before the experiment. After the visual acuity test, a gray image consisting of only the background color was presented on the display for 3 min to adjust to the brightness of the display. Then, the stimuli were shown on the left and right sides of the display, and the participants were asked to respond by the two-alternative forced-choice task method-which stimulus they perceived more clearly-thereby confirming subjective spatial resolution. To account for the non-uniformity of luminance of the monitor, the experiment was also conducted with the display rotated by 180 degrees. To ensure stability of the responses, one stimulus pattern was repeated 16 times at the same display position for each set of stimuli. That is, participants conducted 1024 judgments (16 stimuli patterns Γ 16 times Γ 2 angles Γ 2 viewing distance) in total. To consider observer fatigue, the experiment was divided into two days, with a viewing distance of 20 cpd on the first day and 30 cpd on the second day.
The results of the experiment at 20 and 30 cpd are shown as percentages of responses in Figs. 5(a), and (b), respectively. The numbers in the pie charts in Fig. 5 represent the percentages of stimuli on the left and right side that were clearly perceived. For example, the 2-line, 1-column pie chart in Fig. 5(a) shows that 68% of the participants perceived RGB stripe clearly and 32% perceived BRGRB clearly when stimuli with RGB and BRGRB subpixel layouts were presented on the left and right, respectively. A response rate (in favor of one pattern) exceeding 75% for each experiment was defined as a significant difference. This is because the chance level of the two-alternative forced-choice is 50% and its psychological discrimination level is between 50-100%. The experimental results showed no significant differences at all viewing distances and for all stimulus pairs. The inter-observer standard deviation was 18.21% at 20 cpd and 11.53% at 30 cpd. The average of intra-observer standard deviation was 22.78% at 20 cpd and 21.00% at 30 cpd. Because no significant differences were observed in the stimulus pairs with the same subpixel layouts on the left and right sides, it appeared that the left and right responses are stable.
From Figs. 2 and 5, there is no significant difference between BRGRB525 and PenTile despite the large difference in their MTF. Therefore, we created a new subpixel structure called GBWR with MTF value between that of PenTile and RGBW, for which a clear perceptual difference was confirmed in a previous study [7], and con-ducted additional experiments. GBWR is a rearrangement of the subpixels of RGBW, named after the order in which the subpixels are arranged. Figure 6 represents the GBWR subpixel layout and Fig. 7 shows the MTF with GBWR added.
Two participants with normal color vision and visual acuity of 20/20 or better participated in the additional experiment. The results of the experiment at 20 and 30 cpd are shown in Figs. 8(a) and (b), respectively, as percentage of responses. Pairs for which significant differences were confirmed are circled in gray. The experimental results showed that subjective spatial resolution generally depended on the display MTF because significant differences were observed for the PenTile-RGBW pair, which has the largest MTF difference, at all viewing distances. At 20 cpd, however, only the GBWR-RGBW pair showed significant differences among the PenTile-GBWR and GBWR-RGBW pairs. The interobserver standard deviation was 9.375% at 20 cpd and 3.819% at 30 cpd. The average of intra-observer standard deviation was 22.64% at 20 cpd and 23.84% at 30 cpd. To discuss the experimental results, the MTF values at the Nyquist frequency (0.5 cpd) for each subpixel layout are shown in Table 2. The difference in modulation at the Nyquist frequency between PenTile and GBWR and between GBWR and RGBW are shown in Eqs. ( 4) and ( 5), respectively.
PenTile-GBWR 0.704 β 0.654 = 0.050
GBWR-RGBW (significant difference) 0.654 β 0.612 = 0.042
The MTF values at the Nyquist frequency for each subpixel layout PenTile 0.704 GBWR 0.654 RGBW 0.612 Equations ( 4) and ( 5) indicate that the difference in the magnitude of subjective spatial resolution cannot be explained by the difference in modulation at Nyquist frequency as the difference is significant only for the GBWR-RGBW pair, for which the difference in modulation at the Nyquist frequency is smaller than that of PenTile-GBWR.
Next, we examined the ratio of the modulations at the Nyquist frequency. The Nyquist MTF ratios between each subpixel layout are shown in Equations ( 6)- (9). PenTile-RGBW (significant difference) 0.612 0.704 β 0.869
PenTile-GBWR 0.654 0.704 β 0.929
GBWR-RGBW (significant difference) 0.612 0.654 β 0.936
The closer the Nyquist MTF ratio is to 1, the smaller is the difference in the MTF. The ratio between GBWR and RGBW, for which a significant difference was con-firmed, is closer to 1 than that between PenTile and GBWR. Therefore, the Nyquist MTF ratio cannot explain the difference in the magnitude of subjective spatial resolution.
To consider human visual characteristics, we used the contrast sensitivity function (CSF) obtained by Mannos and Sakrison [9] through psychophysical experiments. We took the product of the CSF at 20 cpd and MTF of the subpixel layouts and calculated the integral values (Table 3). There are two MTF values for PenTile in Table 3-for the first and the additional experiment, respectively. The ratios of these integrals are shown in Eqs. ( 9)-(12). Equations ( 9)-(11) show that for the ratios of the integrals, PenTile-RGBW < GBWR-RGBW < PenTile-GBWR; hence, the results of the additional experiment can be explained by the ratio of the integrated values of CSF Γ MTF at 20 cpd. However, the ratio between BRGRB525 and PenTile, which did not differ significantly, was the smallest, indicating that not all results can be explained by the ratio of CSF Γ MTF integrals.
To further analyze the effect of display MTF on subjective spatial resolution, we designed two new types of subpixel layouts and experimentally verified the effects of these subpixel layouts on subjective spatial resolution. Our results confirmed that the subjective resolution generally follows the magnitude relationship of display MTF. Additional experiments showed that the magnitude of the difference of subjective spatial resolution cannot be explained by the difference and ratio in modulation at Nyquist frequency. Furthermore, we found that the integral of the product of the contrast sensitivity function and MTF was highly correlated with subjective spatial resolution; however, they did not explain all the results. In the future, we will conduct more experiments with more participants and analyze metrics that can explain subjective spatial resolution.