- In a previous paper [13] we discussed ISO/IEC 25000 application when new quality measures are defined. In the present paper: - some quality issues in A.I. are identified, - then known solutions are recalled and - new quality measures for A.I. are proposed data Quality issues in A.I.
In this paper new ISO/IEC 25000 quality measures for
dataset used in some A.I. applications are proposed based
on [
II. when referring to Machine Learning.
In this paper, the term A.I. is used for simplicity even
Figure 1 Definitions [
A.I.
application
face
recognition, well-known both for the solutions and for the
open issues point of view. Among the open issues there is
how
understand
whether the
training
dataset is
“optimal”. To this end, we will explore the measure of
completeness characteristic [
The
basis of our analysis is the calculation of
“eigenfaces”
[
Karhunen-Loève transformation (PCA) with the following steps: 1. Collect M images with nxn grayscale pixels of faces with similar dimension, light condition, shot, etc. 2. Transform image i (i=1,..M) in a (n2x1) column vector Γi {Γ1, Γ2,… ΓM}; 3. Compute the “average face” Ψ = 1 ∑1 Γ and subtract Ψ to each image and obtain new vectors {Φ1, Φ2,… ΦM} 1
AAT
4. Build the matrix (n2xM) A = [Φ1, Φ2,… ΦM] and compute the covariance matrix (n2xn2) C = 1 Φ Φ = 5. Compute M eigenvalues λi (i=1,..M) of matrix ATA and then eigenvectors of AAT 6. Sort the eigenvalues of C in descending order 7. Choose a number N of eigenvalues, starting from the biggest, in order to represent 95% of their sum η and keep them; the other n2-N eigenvalues will not be considered 8. Calculate the images dataset as a linear combination of the N eigenvectors defined at step 7
Intuitively, if we want to measure the completeness1 of an image dataset, we try to answer questions like:
how many similar images are there how strong is the similarity of some images Here the proposed measures for (a) and (b):
A. as some dimensions are eliminated in step 7 above 2 , we can measure N\M “PCA space dimension against dataset space dimension” B. the more a dataset is orthogonal, the less its images are similar to each other; as a measure of it, we can consider the product of N eigenvalues λ1*λ2*...*λN, that is also the “Determinant of reduced eigenvalues matrix”, that in turn is the volume of the hyper-parallelepiped that this matrix represents.
To sum up, with this proposal we reframe the issue of finding an effective data quality (for completeness) measurement function into a well-known geometrical calculation.
The steps 1-8 above were proven to be effective in face recognition and are potentially applicable to any dataset. To do this, the vectorization step 2 above shall be applied to any attribute(s) of the dataset: as images were vectorized pixel by pixel3, similar operation could be performed e.g. for char strings, taking into account possible different lengths that require a further step of normalization. Further studies are needed to apply the method also to images rotated and translated, that is the most frequent case in A.I. applications (fig.3).
Figura 3 Dataset trial MPEG-CDVA (Compact Descriptor for
Video Analysis)– [
As bias is critical for the learning dataset quality, the measures (A) and (B) are suggested also for bias 5 measurement, when defining bias as the modification of an ideal fully orthogonal and normalized dataset6.
The measures (A) and (B) appear belonging to data
quality completeness characteristic [