=Paper=
{{Paper
|id=Vol-3271/Paper10_CVCS2022
|storemode=property
|title=Effects of Display Modulation Transfer Function with Different Subpixel Layouts on Subjective Spatial Resolution
|pdfUrl=https://ceur-ws.org/Vol-3271/Paper10_CVCS2022.pdf
|volume=Vol-3271
|authors=Tsubasa Ando,Midori Tanaka,Takahiko Horiuchi,Kenichiro Masaoka
|dblpUrl=https://dblp.org/rec/conf/cvcs/AndoTHM22
}}
==Effects of Display Modulation Transfer Function with Different Subpixel Layouts on Subjective Spatial Resolution==
https://ceur-ws.org/Vol-3271/Paper10_CVCS2022.pdf
Effects of Display Modulation Transfer Function with Different
Subpixel Layouts on Subjective Spatial Resolution
Tsubasa Ando 1, Midori Tanaka 2, Takahiko Horiuchi 1 and Kenichiro Masaoka 3
1
Chiba University, Graduate School of Science and Engineering; Chiba, Japan
2
Chiba University, Graduate School of Global and Transdisciplinary; Chiba, Japan
3
NHK Science and Technology Research Laboratories; Tokyo, Japan
Abstract
Although a typical display consists of red-green-blue (RGB) subpixels, displays with various
subpixel layouts have been used owing to their various advantages such as luminous efficiency.
The purpose of this study is to verify the effects of display MTF on subjective spatial resolution
by creating new subpixel layouts. We designed BRGRB and BRGRB525 subpixel layouts with
slightly higher and much higher MTF than RGB subpixel layouts, and conducted visual
evaluation experiments with RGB, PenTile RGBG and the two new subpixel layouts at 20 and
30 cycles per degree. It was verified that subjective spatial resolution generally follows the
large and small relationship of display MTF. Additional experiments showed that the integral
of the product of the contrast sensitivity function and MTF was highly correlated with the
subjective spatial resolution.
Keywords 1
Display MTF, Subjective spatial resolution, Subpixel.
1. Introduction
Display resolution is an important indicator of image quality and is generally quantified as a pixel
count with one pixel as the minimum element. Conventional displays consist of colored red-green-blue
(RGB) subpixels. Recently, various subpixel layouts such as RGB and white (RGBW) and RGB and
yellow (RGBY) have been used. Be-cause such subpixel layouts employ various subpixel rendering
technique, a new method for evaluating resolution is required.
The guidelines in Sections 7.2 [1] and 7.8 [2] of the International Display Measurement Standard
(IDMS) by the International Committee for Display Metrology (ICDM) proposed a method to evaluate
the resolution capability of a display, with respect to its addressability, based on threshold contrast
modulation (Michelson contrast) associated with grille patterns. Several studies on subjective spatial
resolution have also been reported [3, 4]. However, these conventional studies have not investigated the
effect of different subpixel layouts on subjective spatial resolution.
In our previous work, we conducted evaluation experiments to examine the subjective spatial
resolution for RGB, RGBW, PenTile RGBG (hereafter abbreviated as PenTile) subpixel layouts and
confirmed that differences in subjective spatial resolution occur even at a viewing distance equivalent
to the ITU-R [5] recommended angular resolution of 30 cycles per degree (cpd) [6, 7]. Analysis of the
modulation transfer function (MTF) of the display [8] suggested that the MTF could be an indicator of
subjective spatial resolution [7]. However, these studies only consider three types of subpixel layouts,
and further validation is needed for many more layouts. In addition, there was no significant difference
in subjective spatial resolution between RGB layout with the highest MTF and PenTile layout with the
second highest MTF. Therefore, it is necessary to further investigate the relationship between the
difference in magnitude of the MTF and subjective spatial resolution using different subpixel layouts.
The 11th Colour and Visual Computing Symposium, September 8–9, 2022, Gjøvik, Norway
EMAIL: tsubasaando@chiba-u.jp (T. Ando); midori@chiab-u.jp (M. Tanaka); horiuchi@faculty.chiba-u.jp (T. Horiuchi); masaoka.k-
gm@nhk.or.jp (K. Masaoka)
ORCID: 0000-0002-4651-4942 (M. Tanaka); 0000-0002-8197-6499 (T. Horiuchi)
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
In this study, to further examine the relationship between display MTF and subjective spatial
resolution, we design two new subpixel layouts with slightly higher and much higher MTF than RGB,
and conduct visual evaluation experiments with RGB, PenTile and the new subpixel layouts.
2. Experiment 1
2.1. Subpixel Layouts
In the experiment of Ref. [7], the RGB subpixel layout had the highest MTF. However, because
there was no significant difference in subjective spatial resolution between the RGB and PenTile
subpixel layouts which had the next highest MTF, we designed new subpixel layouts with a higher MTF
than the RGB. Several new candidate subpixel layouts were created in reference to conventional RGB
and PenTile subpixel layouts. By manually calculating the MTF of these structures and selecting
subpixel layouts with higher MTF than RGB, two new types of subpixel layouts were designed—
BRGRB and BRGBR525—as shown in Fig.1. BRGRB was named after the order of subpixels, and 525
was added to BRGRB525 because the area ratio of R, G, and B subpixels is 5: 2: 5.
In this study, we computed the MTF for each subpixel layout based on the method proposed by
Masaoka [8]. The MTF was calculated by computing the vertically averaged luminance line spread
function (LSF) for the 2 × 2 black and white line patterns of each subpixel layout, performing Fourier
transform and normalizing it with 𝜁 = 0. For example, the LSF of the RGB subpixel layout is as
follows:
5 3 (1)
𝑅!"# 𝑥+6 𝐺!"# 𝑥+6
𝐿𝑆𝐹(𝑥) = 2 , 𝑟𝑒𝑐𝑡 4 :+ 𝑟𝑒𝑐𝑡 4 :
𝑅𝐺𝐵!"# 1 𝑅𝐺𝐵!"# 1
3 3
5
𝐵!"# 𝑥+6
+ 𝑟𝑒𝑐𝑡 4 :;
𝑅𝐺𝐵!"# 1
3
where 𝑟𝑒𝑐𝑡 is a rectangular function. 𝑅!"# , 𝐺!"# , and 𝐵!"# represent the luminance of the RGB
subpixels, where 𝑅𝐺𝐵!"# = 𝑅!"# + 𝐺!"# + 𝐵!"# . Then, the 𝑀𝑇𝐹(𝜁) is obtained as
1 𝑅!"# $%'( 𝐺!"# $'( 𝐵!"# $)'( (2)
𝑀𝑇𝐹(𝜁) = >sinc C 𝜁D C 𝑒 & + 𝑒 + 𝑒 & D>
3 𝑅𝐺𝐵!"# 𝑅𝐺𝐵!"# 𝑅𝐺𝐵!"#
where 𝜁 is the spatial frequency in cycles per display pixel. Using this method, the MTF for each of the
RGB, PenTile, BRGRB, and BRGRB525 subpixel layout were calculated respectively, and the results
are shown in Fig. 2. The MTF values are in the order BRGRB525 ≫ BRGRB > RGB ≫ PenTile.
(a)BRGRB (b)BRGRB525
Figure 1: Designed subpixel layouts
Figure 2: The MTF of each subpixel layout
2.2. Stimuli
In addition to the conventional RGB and PenTile, and the newly designed BRGRB and BRGRB525,
four types of vertical black and white grille patterns were used as experimental stimuli. Figure 3 shows
the black and white grille patterns of each subpixel layout. In this experiment, the grille patterns were
presented on a liquid crystal display (ColorEdge CG248-4K, Eizo Corp., Japan). Table 1 shows the
display specification. Because this display has the RGB subpixel layout, it is not possible to display the
grille patterns with other subpixel layouts. Therefore, as in Ref. [7], we regarded 12 × 12 pixels of the
actual display as one virtual pixel and devised a method to virtually perceive different subpixel layouts
by viewing one virtual pixel from a viewing distance of 12D—12 times the viewing distance D for one
pixel—as shown in Fig. 4. To unify the one virtual pixel pitch, the virtual subpixel pitch of the RGB
layout was set to 4 × 12 pixels of the display, 4 × 12 and 8 × 12 for PenTile, 2 × 12 and 4 × 12 for
BRGRB, and 2 × 12 and 3 × 12 for BRGRB525. Therefore, the number of stripes in the experimental
stimuli would be the same for different subpixel layouts, and the subpixel layouts would not be
perceived when evaluating the stimuli.
Circular vertical grille patterns of 150 virtual pixels in diameter (1,800 real display pixels) were
created as the experimental stimuli. The area ratio of the R, G, and B subpixels were different and the
difference in brightness and chromaticity of the experimental stimuli affected the evaluation only for
BRGRB525. Therefore, we adjusted the subpixel area × luminance values to be equal among the
subpixel layouts using a spectroradiometer (CS-2000, KONICA MINOLTA, INC., Japan). For instance,
the G subpixel in BRGRB525 is half the area of the G subpixel in the other subpixel layouts; hence,
adjustments were made to double the luminance value. The luminance values of the black and white
lines in the grille pattern were set to be 0.4 and 49.5 𝑐𝑑/𝑚* . The background color of the experimental
stimuli patterns was supposed as the average of the black and white luminance in the grille pattern.
When the experimental stimuli were displayed, Moire fringes appeared at the boundary
between the left and right edges of the stimuli and background. To prevent the stripes at the edge
of the stimulus from becoming a response cue, gradient processing was applied to the boundary
between the stimulus and background using the following procedure [7]. First, the center of the
stimulus (720 virtual pixels in radius from the center) was considered as the observation area,
and no gradient processing was applied. The radius was normalized within the target area, with
the radius closer to the center of the target area being 0 and the radius closer to the background
being 1. Then, the following equation was applied to the experimental stimuli:
) (3)
𝑛𝑒𝑤 𝑝𝑖𝑥𝑒𝑙 𝑣𝑎𝑙𝑢𝑒 = (𝑝𝑖𝑥𝑒𝑙 𝑣𝑎𝑙𝑢𝑒 + × (1 − 𝑟) × 𝑔𝑟𝑎𝑦 + × 𝑟)+
where 𝑝𝑖𝑥𝑒𝑙 𝑣𝑎𝑙𝑢𝑒 represents the pixel value of the experimental stimuli. 𝑔𝑟𝑎𝑦 represents the pixel
value of the gray background, 𝑟 is the radius (0 to 1) of the processed area after normalization,
and 𝛾 refers to the gamma value of the display.
Table 1
Display settings
Resolution 3800 × 2160
Luminance 300 𝑐𝑑/𝑚*
Color temperature 6500 𝐾
Gamma value 2.2
Color gamut Native
(a) RGB (b)PenTile RGBG (c) BRGRB (d) BRGRB525
Figure 3: The black and white grille patterns of each subpixel layout
Figure 4: Virtual pixel representation
2.3. Procedure
The participants employed for this study were four students with normal color vision of 20/20 or
better (three with naked eyes and one with glasses). A total of 16 stimuli pairs were used—4 (RGB,
PenTile, BRGRB, and BRGRB525 for left stripe) × 4 (RGB, PenTile, BRGRB, and BRGRB525 for
right stripe). The experiments were conducted in a dark room at 20 cpd (3.77 m) and 30 cpd (5.66 m),
based on the ITU-R [5] recommended angular resolution of 30 cpd (cycles per degree). Here, cpd was
calculated based on one virtual pixel, so the viewing distance was 12 times the viewing distance for one
pixel.
To ensure that the observers had binocular vision of at least 20/20, their visual acuity was tested
each time before the experiment. After the visual acuity test, a gray image consisting of only the
background color was presented on the display for 3 min to adjust to the brightness of the display. Then,
the stimuli were shown on the left and right sides of the display, and the participants were asked to
respond by the two-alternative forced-choice task method—which stimulus they perceived more
clearly—thereby confirming subjective spatial resolution. To account for the non-uniformity of
luminance of the monitor, the experiment was also conducted with the display rotated by 180 degrees.
To ensure stability of the responses, one stimulus pattern was repeated 16 times at the same display
position for each set of stimuli. That is, participants conducted 1024 judgments (16 stimuli patterns ×
16 times × 2 angles × 2 viewing distance) in total. To consider observer fatigue, the experiment was
divided into two days, with a viewing distance of 20 cpd on the first day and 30 cpd on the second day.
2.4. Results
The results of the experiment at 20 and 30 cpd are shown as percentages of responses in Figs. 5(a),
and (b), respectively. The numbers in the pie charts in Fig. 5 represent the percentages of stimuli on the
left and right side that were clearly perceived. For example, the 2-line, 1-column pie chart in Fig. 5(a)
shows that 68% of the participants perceived RGB stripe clearly and 32% perceived BRGRB clearly
when stimuli with RGB and BRGRB subpixel layouts were presented on the left and right, respectively.
A response rate (in favor of one pattern) exceeding 75% for each experiment was defined as a significant
difference. This is because the chance level of the two-alternative forced-choice is 50% and its
psychological discrimination level is between 50–100%. The experimental results showed no
significant differences at all viewing distances and for all stimulus pairs. The inter-observer standard
deviation was 18.21% at 20 cpd and 11.53% at 30 cpd. The average of intra-observer standard deviation
was 22.78% at 20 cpd and 21.00% at 30 cpd. Because no significant differences were observed in the
stimulus pairs with the same subpixel layouts on the left and right sides, it appeared that the left and
right responses are stable.
(a)20 cpd
(b)30 cpd
Figure 5: Results of the experiment at (a) 20 and (b) 30 cpd as percentages
3. Experiment 2
3.1. Subpixel Layouts
From Figs. 2 and 5, there is no significant difference between BRGRB525 and PenTile despite the
large difference in their MTF. Therefore, we created a new subpixel structure called GBWR with MTF
value between that of PenTile and RGBW, for which a clear perceptual difference was confirmed in a
previous study [7], and con-ducted additional experiments. GBWR is a rearrangement of the subpixels
of RGBW, named after the order in which the subpixels are arranged. Figure 6 represents the GBWR
subpixel layout and Fig. 7 shows the MTF with GBWR added.
Figure 6: GBWR
Figure 7: The MTF of each subpixel layout (additional experiment)
3.2. Results
Two participants with normal color vision and visual acuity of 20/20 or better participated in the
additional experiment. The results of the experiment at 20 and 30 cpd are shown in Figs. 8(a) and (b),
respectively, as percentage of responses. Pairs for which significant differences were confirmed are
circled in gray. The experimental results showed that subjective spatial resolution generally depended
on the display MTF because significant differences were observed for the PenTile-RGBW pair, which
has the largest MTF difference, at all viewing distances. At 20 cpd, however, only the GBWR-RGBW
pair showed significant differences among the PenTile-GBWR and GBWR-RGBW pairs. The inter-
observer standard deviation was 9.375% at 20 cpd and 3.819% at 30 cpd. The average of intra-observer
standard deviation was 22.64% at 20 cpd and 23.84% at 30 cpd. To discuss the experimental results,
the MTF values at the Nyquist frequency (0.5 cpd) for each subpixel layout are shown in Table 2. The
difference in modulation at the Nyquist frequency between PenTile and GBWR and between GBWR
and RGBW are shown in Eqs. (4) and (5), respectively.
PenTile-GBWR
0.704 − 0.654 = 0.050 (4)
GBWR-RGBW (significant difference)
0.654 − 0.612 = 0.042 (5)
Table 2
The MTF values at the Nyquist frequency for each subpixel layout
PenTile 0.704
GBWR 0.654
RGBW 0.612
Equations (4) and (5) indicate that the difference in the magnitude of subjective spatial resolution
cannot be explained by the difference in modulation at Nyquist frequency as the difference is significant
only for the GBWR-RGBW pair, for which the difference in modulation at the Nyquist frequency is
smaller than that of PenTile-GBWR.
Next, we examined the ratio of the modulations at the Nyquist frequency. The Nyquist MTF ratios
between each subpixel layout are shown in Equations (6)–(9).
PenTile―RGBW (significant difference)
0.612 (6)
≃ 0.869
0.704
PenTile―GBWR
0.654 (7)
≃ 0.929
0.704
GBWR―RGBW (significant difference)
0.612 (8)
≃ 0.936
0.654
The closer the Nyquist MTF ratio is to 1, the smaller is the difference in the MTF. The ratio between
GBWR and RGBW, for which a significant difference was con-firmed, is closer to 1 than that between
PenTile and GBWR. Therefore, the Nyquist MTF ratio cannot explain the difference in the magnitude
of subjective spatial resolution.
To consider human visual characteristics, we used the contrast sensitivity function (CSF) obtained
by Mannos and Sakrison [9] through psychophysical experiments. We took the product of the CSF at
20 cpd and MTF of the subpixel layouts and calculated the integral values (Table 3). There are two
MTF values for PenTile in Table 3—for the first and the additional experiment, respectively. The ratios
of these integrals are shown in Eqs. (9)–(12).
Table 3
CSF × MTF integral values at 20 cpd for each subpixel layout
PenTile 0.332
(Additional experiment)
GBWR 0.327
RGBW 0.321
BRGRB525 0.357
PenTile 0.331
PenTile―RGBW (significant difference)
0.321 (9)
≃ 0.969
0.332
PenTile―GBWR
0.327 (10)
≃ 0.985
0.332
PenTile―RGBW (significant difference)
0.321 (11)
≃ 0.984
0.327
BRGRB525―PenTile
0.331 (12)
≃ 0.972
0.357
Equations (9)–(11) show that for the ratios of the integrals, PenTile-RGBW < GBWR-RGBW <
PenTile-GBWR; hence, the results of the additional experiment can be explained by the ratio of the
integrated values of CSF × MTF at 20 cpd. However, the ratio between BRGRB525 and PenTile, which
did not differ significantly, was the smallest, indicating that not all results can be explained by the ratio
of CSF × MTF integrals.
(a)20 cpd
(b)30 cpd
Figure 8: The results of the additional experiment
4. Conclusions
To further analyze the effect of display MTF on subjective spatial resolution, we designed two new
types of subpixel layouts and experimentally verified the effects of these subpixel layouts on subjective
spatial resolution. Our results confirmed that the subjective spatial resolution generally follows the
magnitude relationship of display MTF. Additional experiments showed that the magnitude of the
difference of subjective spatial resolution cannot be explained by the difference and ratio in modulation
at Nyquist frequency. Furthermore, we found that the integral of the product of the contrast sensitivity
function and MTF was highly correlated with subjective spatial resolution; however, they did not
explain all the results. In the future, we will conduct more experiments with more participants and
analyze metrics that can explain subjective spatial resolution.
5. References
[1] ICDM, Section 7.2. Grille Luminance & Contrast Modulation, in: IDMS V1.03, 2012, pp. 123-
130.
[2] ICDM, Section 7.8. Resolution From Contrast Modulation, in: IDMS V1.03, 2012, pp. 147-148.
[3] Y. S. Baek, Y. Kwak, S. Park, Visual Resolution Measurement of Display using the Modified
Landolt C, IDW Symp. Dig. Tech. Pap. 25 (2018) 976-978.
[4] M. E. Becker, Measurement of visual resolution of display screens, SID Symp. Dig. Tech. Pap.
48.1 (2017) 915-918. doi: 10.1002/sdtp.11784.
[5] Recommendation ITU-R BT.2035, A reference viewing environment for evaluation of HDTV
program material or completed programmes, ITU Radiocommunication Sector (2013).
[6] M. Tanaka, D. Nakayama, T. Horiuchi, K. Masaoka, An Experimental Study of the Effect of
Subpixel Arrangements on Subjective Spatial Resolution, Proc. SID Display Week 2020 (2020).
doi: 10.1002/sdtp.14095
[7] K. Nakamura, M. Tanaka, T. Horiuchi, K. Masaoka, Experimental Study of the Effect of Subpixel
Layouts on Subjective Spatial Resolution, Report on ITE winter annual convention(2021) (in
Japanese).
[8] K. Masaoka, Line-based modulation transfer function measurement of pixelated displays, IEEE
Access 8 (2020) 196351-196362. doi: 10.1109/access.2020.3033756
[9] J. L. Mannos, D. J. Sakrison, The Effects of a Visual Fidelity Criterion on the Encoding of Images,
IEEE Transactions on Information Theory 20.4 (1974) 525-535.