-
Visual Computing (VC) denotes the methods for acquisition, processing, and
elaboration of visual data [
7
]. In particular, it concerns the techniques of
pattern extraction and analysis whose aim is capturing visual characteristics of data.
The approaches of VC include: (i) methods of image and video analysis, (ii)
computer vision, (iii) visualisation and visual analytics, (iv) augmented reality, (v)
human computer interaction, and (vi) computer graphics. The methods of image
and video analysis aim to capture patterns and content information from images
and video frames. A further step is computer vision, which is oriented to
recognition and interpretation of visual structures. Visualisation and visual analytics
is referred to production of images and interactive interfaces which communicate
using messages. By contrast, augmented reality includes methods for
augmenting real-world objects with computer-generated features. It can generate virtual
elements which are visualised by the users as embedded inside the real-world
environment. Human computer interaction concerns the design, processing and
evaluation of interfaces between users and machines. Finally, computer graphics
is the set of methods aiming to produce images or 3D objects.
One of the most relevant application areas of VC is cultural heritage
preservation and understanding, where VC plays a key role in capturing, modelling
and exploring visual features and representations of findings of historical interest
worldwide. Especially in art and painting, the di↵erent methods of VC are
invaluable in extracting meaningful patterns and models for advancing important
hypotheses and revealing useful information. The motivations for exploring VC
methods in cultural heritage understanding are manifold. Cultural heritage is
included in multiple aspects of everyday life. Also, it is everywhere, spread in
little towns and big cities, in natural scenes and archeological sites. Cultural
heritage involves the literature, art, knowledge inherited from ancestors, culinary
traditions, films and cinema. Nowadays, it is considered as a world shared wealth
which is composed of traditions and history of di↵erent countries, that should
be preserved, understood and celebrated. Also, the cultural heritage is essential
for tracking the horizon and planning the future [
6
].
In the context of cultural heritage, an invaluable piece of art and masterpiece
of the art history worldwide is the Mona Lisa, a well-known portrait painted by
Leonardo Da Vinci, an Italian Renaissance artist. However, the identity of the
portrait’s subject, its painting date, who commissioned the portrait, how long
Leonardo Da Vinci worked on it, and how long he kept the portrait, still remain
a mystery and di↵erent hypotheses were advanced on this [
12
]. In particular, the
portrait could be painted between 1503 and 1506, and continued till late 1517.
Also, some recent works advanced the hypothesis that the portrait could not be
started before 1513 [
11
]. The portrait could be of Lisa Gherardini, who was the
wife of Francesco del Giocondo, a Florentine cloth merchant, from which the
portrait was named as La Gioconda (in Italian language). The portrait could be
painted for celebrating the new house of Francesco del Giocondo and his wife in
1503, or the born of their second son Andrea in 1502, after the death of their
daughter in 1499 [
12
]. However, it is likely that Leonardo Da Vinci brought the
portrait with him in France instead of leaving it to the person who commissioned
it. Currently, the Mona Lisa is hosted in Louvre Museum in Paris, France. Figure
1 shows the portrait of Mona Lisa in all its sheen.
In this paper, di↵erent relevant works of VC are reported and described for
studying, exploring and modelling the portrait of Mona Lisa. In particular, the
first part of the paper analyses di↵erent hypotheses about the identity of Mona
Lisa’s subject, ambiguity of the portrait and other hidden features related to
the painting using image analysis and human computer interaction. In order to
advance new hypotheses, the second part of the paper aims to present di↵erent
graphics models in 2D or methods for 3D rendering of Mona Lisa’s portrait.
Finally, a discussion about the di↵erent methods is performed and a suggestion
for future work directions is presented. The proposed analysis is useful for
understanding the current literature techniques, their limitations and explore new
directions which shed light on a mystery still partially unsolved in the art
history. To the very best of knowledge, this is the first paper surveying about the
topic of Mona Lisa’s portrait in the state-of-the-art.
The paper is organised as follows. Section 2 describes image analysis and
human computer interaction works about identity of Mona Lisa’s subject,
ambiguity and hidden features of the portrait. Section 3 presents computer graphics
works of the portrait’s 2D and 3D modelling. Section 4 makes a discussion about
the described works. Finally, Section 5 draws conclusions about the proposed
analysis.
2
Identity, Perception and Hidden Characteristics
In the following, relevant visual computing works describing hypotheses about
the identity of Mona Lisa’s subject, ambiguity and hidden features are presented.
The di↵erent works are categorised as: (i) approaches based on image analysis,
and (ii) approaches based on human computer interaction.
2.1
Image Analysis Approaches
Multiple theories have been advanced about the identity of Mona Lisa’s subject.
In [
14
], Schwartz proved that Leonardo Da Vinci used himself as a model for
realising the Mona Lisa. This was validated by both historical as well as visual
characteristics. First, there was no clear information about Leonardo’s
commission of the Mona Lisa or the identity of the model. Second, at the time when
Mona Lisa was realised, Leonardo moved in di↵erent places and lived in
families with no women. Also, the painting did not have female features, and the
di↵erence in the left and right landscape suggested an ambiguity in the subject.
The last feature was a supraorbital ridge which could be observed on the Mona
Lisa’s face and that was also present on Leonardo’s face, which is typical in male
subjects. From a visual perspective, an experiment was performed which
juxtaposed the Mona Lisa’s and Leonardo’s images in order to show their common
characteristics. In particular, the two images were obtained by scanning and
digitising the Mona Lisa and Leonardo’s Self-Portrait. The grey levels of the
images were enhanced. The image of Leonardo’s Self Portrait was flipped along
the vertical axis. Both images were scaled and aligned, vertically bisected and
the two halves juxtaposed (see Fig. 2 (a)). The author observed as the position
of nose, mouth, eyes, chin and forehead matched in the two images. About the
landscape of Mona Lisa’s portrait, it was observed that it has di erences in the
left and right part. In particular, the left part is lower, less logical and di erent
in time and place than the right part. It could indicate a sort of dichotomy in
Leonardo, where features of two sexes are mixed together. Another analysis was
related to an x-ray of the Mona Lisa which revealed a second portrait below the
surface portrait (see Fig. 2 (b)). In the past, the common opinion was that the
two portraits represented the same subject. In this work, the author juxtaposed
the two portraits for alignment and demonstrated, from the analysis of the
xray’s features, that they do not represent the same subject. In particular, the
face’s features, i.e. eyes, mouth, or chins, did not match in the two portraits.
By contrast, the x-ray appeared as very similar to a Cartoon depicting Isabelle,
Duchess of Aragon, that Leonardo painted with the same pictorial technique
before the Mona Lisa. The final hypothesis was that the real subject of the portrait
was Isabelle, that was overpainted with Mona Lisa, using Leonardo Da Vinci as
the model.
The results of this analysis were contradicted by Lin et al. [
10
] who compared
the subject of the portrait with Leonardo Da Vinci’s subject. It was accomplished
by extracting shape features using active shape models. The experiment was
performed using a database of 488 frontal faces of various ethnicities, of which 151
were female faces, and 337 were male faces. Each face was manually labeled with
87 landmarks. Also, the face from Mona Lisa’s portrait and a renowned portrait
of Leonardo were labeled with the landmarks. They corresponded to significant
face points, such as eye and lip corners, points along the bottom of nose and
face edge. A preprocessing step to the landmarks was performed for increasing
symmetry and equal spacing among the points. After that, Principal
Component Analysis (PCA) was applied on the landmark representation of the faces,
which determined a 12-dimensional feature vector for each face. It represented
the way the face could di↵er from the average face along the k most relevant
variation modes in the data. The Mahalanobis distance was used for comparison
between the feature representations. Finally, a K-nearest neighbour classifier for
categorisation of gender was employed on the obtained feature vectors. Results
from classification showed that Mona Lisa’s face is classified as female, while
Leonardo’s face is classified as male. Also, the computed Mahalanobis distance
between the two faces is over 3.6 standard deviations from the average,
demonstrating that Leonardo and Mona Lisa are two di↵erent subjects.
Finally, analysis of Mona Lisa’s ambiguity was performed by Asmus [
1
],
after the damages occurred on the portrait over time, e.g. discoloured, crackled
and soiled varnish and cleavage inside the paint layer. First, a high-quality
picture of the painting was acquired from the Louvre Museum at a resolution of
6-million pixels. Then, digital data was collected from the central part of the
picture in order to create three files, one for each primary colour, i.e. green, blue
and red (RGB). Second, a gain-bias modulation was applied on the RGB image
files, which compensated for the filtering of the discoloured varnish, in order to
recover the original and natural colours which are varnish-free. Also, removal
of the craquelure was performed using sequential application of bi-dimensional
Fast Fourier Transform (FFT) filtering approaches and blue/green bi-scatter
filters. Since e↵ects of craquelure-induced glints are naturally periodical, a
twodimensional matrix with phase and amplitude of di↵erent spatial waves related
to the picture was created. Then, filtering was applied on that matrix in order to
reduce the waves which caused those e↵ects. Finally, the Inverse Fourier
Transform, applied on the product of the filtered image with the filter, determined a
picture with reduced glints. After that, selected pixels of unwanted colour were
repainted to a wanted colour by modification of their three-channel values. It
was accomplished by generating a bi-scatter plot counting the pixels at di↵erent
combinations of blue and green with the highest alteration. A mask was
generated from information extracted from the plot, which was applied on the picture
for further reduction of the craquelure e↵ects. Figure 3 shows the output of the
described procedure.
Regional contrast stretch was then performed on the restored picture. In
particular, a region of interest was selected, on which statistics on the three channels
were computed in order to perform a histogram equalisation on the image. This
operation obtained a higher level of detail on that region for its analysis. Also,
local intensity enhancement was performed, which computed a new value for
pixels according to the statistics computed in a square of their neighbourhood.
Finally, a pseudo-colour mapping was performed, where small ranges of pixel
levels were clustered and arbitrarily re-coloured. It revealed pentimenti and
portions where the painting was changed, specifically the presence of necklace and
additional mountains. Figure 4 shows the result of this procedure in the neck
area of Mona Lisa.
The author observed that these features revealed the ambiguous aspect of
Mona Lisa’s painting, with the left part representing order and conformity of
the appearance of the subject, and the right part depicting disorder and chaos,
which represents the interiority of Mona Lisa’s subject.
2.2
Human Computer Interaction Approaches
Though analysis of the painting composition e↵ect on youthfulness, facial
femininity, and attractiveness, Pausch and Kuhnt [
13
] provided an estimate of Mona
Lisa’s age. The experiment involved a population of 107 subjects (76 females and
31 males). They were a random sample of fifth-year German dental students from
University of Leipzig. Each subject was equipped with a questionnaire and was
asked to observe five portraits. Each image was separately shown to each subject
on a monitor screen inside a dark silent room. In the first portrait, the Mona
Lisa’s face was substituted with the face of a male, which was Christian IV, Duke
of Zweibru¨cken. In the second portrait, the Duke of Zweibru¨cken was correctly
reported with his original face. In the third portrait, the Mona Lisa’s face was
substituted with the face of Marie-Suzanne Giroust-Roslin. In the fourth
portrait, Marie-Suzanne Giroust-Roslin was painted with her original face. Finally,
in the fifth portrait, the original version of Mona Lisa was depicted. Figure 5
shows the five described portraits.
The two alternative paintings were randomly selected, were painted after the
Renaissance and had di↵erent background. All collected data was statistically
processed with a significance threshold of 0.05. The null hypothesis was that
the painting composition had no influence on the Mona Lisa’s youthfulness,
facial femininity, and attractiveness. Another hypothesis was that in the original
painting composition Mona Lisa’s face appeared younger, more attractive and
more feminine than the same face in a male painting composition. The last
hypothesis was that Mona Lisa’s face was more attractive in the original painting
composition than in the female composition. The second aim of the analysis was
to investigate about the perception of Mona Lisa’s age. Hence, the hypothesis
was that age was in the third decade of life. The independent variable was
the portrait composition. The dependent variables were estimated age (years),
facial femininity, youthfulness, and attractiveness of the subject in each portrait.
Results from the analysis showed that the portrait composition has an influence
on facial femininity, youthfulness, and attractiveness. Also, the estimated age of
the Mona Lisa’s face was 32.3 ± 5.6 years. In particular, the male portrait with
Mona Lisa’s face was ranked as younger and more feminine but less attractive
than the original painting. By contrast, the female portrait with Mona Lisa’s
face was ranked as older but more attractive than the original painting.
Also, the emotional ambiguity of Mona Lisa was analysed in [
9
] using a
variant of the constant stimuli psychophysical approach, well-known for detecting
perceptual thresholds. In this way, the e↵ective degree of ambiguity was
measured through the happy-sad axis of emotional expressions. It was experimented
on a population of twelve subjects, of which 5 were male, and 7 were female,
of age between 20 and 33 years. A grey-scale version of Mona Lisa was used
for generating 12 variants each corresponding to a di↵erent curvature of the
mouth, which is the most relevant aspect of ambiguity. Each variant represented
an emotional state from sad to happy. The experiment was characterised by
two conditions. In the first one, each of nine variants equally spaced in
mouthmanipulation degree (from the happiest to the saddest) was presented in random
order to each subject for a maximum of 6 seconds. The perceived emotional
expression and a rating of the given response were asked to each subject within
the established time range. The variants’ block was presented 30 times to the
subjects in random order of the variants. In the second condition, the resolution
of ambiguity was increased by decreasing the range of variants in the emotional
scale. Also, nine variants were presented to each subject. In both conditions, the
perceived happiness was modelled as sigmoid functions of Mona Lisa variants.
The sigmoid functions, confidence rating per subject and variant, and reaction
times to the variant presented meaningful di↵erences between the two conditions.
From their analysis, it was observed that Mona Lisa was almost always
considered as unambiguously happy. However, the emotional perception and reaction
to the variants was strongly dependent on the adopted emotional range.
In addition to the painting’s ambiguity, an enigma is represented by the
multiple copies of Mona Lisa painted over the years, specifically a restored one
presented in 2012 in the Museo del Prado in Madrid. In order to compare the
original Mona Lisa with the Prado copy, Carbon and Hesslinger [
4
] conducted an
experiment involving a population of thirty-two participants, of which twenty-six
were female, of mean age 21.3. Each participant was required to carefully observe
the two paintings and estimate the position of the painter for both paintings
(original Mona Lisa and Prado copy) in terms of distance and direction. The
average perception of the original Mona Lisa was compared with the Prado
version. Such comparison revealed a significant di↵erence of the painter position
in the two paintings. The di↵erence in perspective was analysed using landmarks
which were set in the original and Prado version of Mona Lisa. Landmarks were
categorised in nine di↵erent types: (i) face, (ii) hair, (iii) body left, (iv) body
right, (v) left arm, (vi) right arm, (vii) left hand, (viii) right hand, and (ix)
chair. Analysis discovered that this change in perspective is not random but
systematic. In particular, the lower part of the paintings shows a visual pattern
of horizontal o↵set, which corresponds to a stereoscopic image composed of two
parts with the same scene horizontally shifted. Consequently, the original and
Prado version of Mona Lisa might be composed simultaneously with the aim of
simulating the depth perception like in a stereoscopic image. It was accomplished
by creating perspective di↵erence between the two paintings for simulating the
human binocular vision. If it is so, the Mona Lisa would be the first stereoscopic
image in the history.
2.3
Other Approaches
Another fascinating characteristic of Mona Lisa’s painting is the geometry and
mathematics-based visual aspect underlying the portrait. One direction in this
context is represented by the Traveling Salesman Problem (TSP) Mona Lisa.
This is a 100,000-city instance of the Symmetric TSP whose solution provided a
mathematical representation of Mona Lisa as a continuous-line drawing. Cities
are random points found on a grayscale version of Mona Lisa’s image. The
solution is a (near) optimal path of the points (see Fig. 6).
Accordingly, Carbajal et al. [
3
] proposed a solution based on Genetic
Algorithms and Ant Colony Optimisation for the solution of TSP Mona Lisa problem.
The project included di↵erent data structures which supported in finding the
solution. In particular, the matrix of paths had one row for each node of the TSP
graph in the TSP problem. The i-th row included the best possible successors of
node i in the path. The current solution was a vector which contained the set of
nodes with order corresponding to the current state of the path. Solutions were
generated using a greedy strategy. It started from some node of the graph and
selected its best successor from the matrix of paths until an Hamiltonian Path
of the TSP graph was obtained. A selection mechanism was provided for having
more attractive nodes at the end of each row in the matrix of paths. Finally,
an update mechanism of matrix of paths was performed using an index derived
from theoretical and current solution best length and number of candidates.
3
Computer Graphics
In order to capture the visual details of Mona Lisa and advance new hypotheses
about the painting, di↵erent works of 2D and 3D modelling have been introduced
in recent times. They can be classified as: (i) 2D models, and (ii) 3D models.
3.1
2D Models
From October 2004, Cotte and Dupraz [
5
] used a high-definition multispectral
system developed by the Lumiere Technology inside the scope of the European
Crisatel project. The research team took two photos of the Mona Lisa using
the system from ultraviolet to infrared, which contributed to collect data and
visual details about the painting. The high-definition spectral imaging system
was characterised by 13 channels with an optimal definition of 360 Mega-pixels
for each channel. Mona Lisa’s painting was installed on a power-driven easel. It
was parallel to the multispectral camera, while the painting was subjected to less
direct lighting on the left and right with an angle varying between 57.5 and 66.5
degrees. First, the control software was used for setting the camera, in order to
obtain the maximum quality of acquisition in a limited time. Then, a calibration
step was performed for storing the levels of dark noise, compensating inter-pixel
di↵erences and lack of homogeneity of the lighting and lens for obtaining a
calibrated image. Finally, the Mona Lisa was positioned on the easel for starting
the process of image acquisition. It was performed in two phases: global frames
of the all painting, then macro frames of the face. For each channel, the camera
acquired a digital value, for a total of 13 values per pixel. The missing reflectance
according to the sampling interval was interpolated in order to reconstruct the
Mona Lisa’s image. The final result was a high-definition colour image of the
painting, by which a simulation of the lighting under di↵erent illuminants could
be provided. It allowed to examine each part of the painting, study the
pictorial approach, make suggestions about virtual restorations and make analysis of
cracks of the painting.
3.2
3D Models
The exploration of the portrait was extended with 3D modelling by Gril et al. [
8
]
through the study of an international research group of wood technologists. The
study started in 2004 and was supervised by the Louvre Museum. The Mona
Lisa is painted on a poplar panel that is surrounded with an oak frame named
chaˆssis-cadre. The panel is fixed to the frame using four crossbars. Also, the
oak frame is put inside a wooden gilded frame. The aim of the study was to
explore the e↵ects of environmental fluctuations when the painting is located
in the showcase or is moved outside and subjected to occasional checks. It was
first accomplished by measuring di↵erent parameters associated to the painting.
They included measurement of the panel shape, relief and out of plane
deformation field on both front and rear face, 3D surface displacements, forces between
panel and crossbars, deflection variations along time (transversal deflection at
the center of the panel versus lateral borders, longitudinal deflection versus the
chaˆssis-cadre), and contact forces between panel and chaˆssis-cadre. Then, all
measurement results were used as input to a 3D numerical finite elements model
which was able to simulate the e↵ects of environmental fluctuations on the panel,
and consequently, predict and validate in which scenarios the Mona Lisa’s
painting would be safer.
Also, Borgeat et al. [
2
] proposed a graphical 3D rendering framework with
the aim to better characterise the Mona Lisa’s painting methodology and
details. The proposed framework was able to create an interactive environment
where data was analysed under di↵erent modes and scales on a commodity
hardware. First, a NRC’s high-resolution polychromatic laser scanner was adopted
for obtaining a digitisation of Mona Lisa. The goal was the acquisition of a
highresolution 3D version of Mona Lisa under di↵erent poses and sides for detecting a
model of the portrait. Such color-per-vertex model, characterised by 333-million
polygons representing the complete poplar panel of the painting, brought higher
resolution in correspondence of the frontal side of the painting. The adopted
method for interactive visualisation of large amount of retrieved surface data
extended the technique of view-dependent hierarchical Levels of Detail (LODs),
where temporal and spatial continuity between the various LODs and across
the model was accomplished by using geomorphing on GPU. The full-resolution
model was able to capture the finest level of detail. A preprocessing phase was
performed for constructing the multiresolution data structure. The model was
also equipped with di↵erent analytical tools to be directly executed on GPU,
such as representing transformations, rendering depth information, multi-step
filtering and image-composition methods. The framework also showed other
interesting aspects related to Mona Lisa, i.e. measuring the shape of the poplar
panel on which it was painted, and analysing some patterns of the paint layer.
Finally, restorations of the painting were analysed using di↵erent lighting, since
they could be associated to di↵erent pigments than the original paint with
consequent di↵erent spectral response. Also, presence of the paint accumulation
forming a crest along the original support frame and visible from 3D data
disproved the hypothesis that thieves cut the wooden panel of Mona Lisa when
they stole it.
4
Discussion
Di↵erent visual computing approaches have been explored for the analysis of
Leonardo Da Vinci’s Mona Lisa portrait, one masterpiece of the art history
worldwide. They are characterised by image analysis and human computer
interaction methods which are very useful for advancing hypotheses about identity,
perception, ambiguity and hidden features of Mona Lisa.
From the literature review, it is worth noting that there is still contradiction
about the identity of the painted subject, if Leonardo Da Vinci used himself or
not as the model. Also, the ambiguity of Mona Lisa is revealed by di↵erences
between the left part, more ordered, and right part, more chaotic, of the portrait,
analysis of the emotions when the subject is observed, and the multiple copies
of Mona Lisa painted over the years. It is also unclear the age of Mona Lisa
which was estimated between 32 and 37 years old. Finally, the mathematical and
geometrical aspects underlying the portrait are still subjected to analysis and
study. In order to advance new hypotheses about Mona Lisa, di↵erent computer
graphics works of 2D and 3D modelling of the painting show their importance
for capturing details of the portrait.
From this study, it is clear that still e↵ort is needed for solving the open
problems and ambiguities of Mona Lisa’s portrait. Since it revealed interesting
patterns of rigorous geometry, proportions and outlook, future research could be
focused on exploring the painting details under a mathematical perspective.
5