In modern astronomy, increasing number of data is becoming an ever-growing problem and opportunity. Formulation and validation of many theories require scientists to go through huge databases, which have become impossible to do by hand. At the same time, increasing capabilities of earth-based observatories and space telescopes are providing us with many sky surveys containing petabytes of quality data [1,2]. This data-intensive situation encourages the investigation of new methodologies, big data tools and techniques, therefore providing a great environment for astroinformatics development [3].
Machine learning has a significant impact on this new reality [4,5,6]. It provides many tools that can be used to swiftly classify huge amounts of data, which we will try to explore in this paper. We will go through algorithms such as Decision Tree [7], Naive Bayes [8] and K-Nearest Neighbors [9] and analyse their accuracy to distinguish between different objects.
In the beginning, we will need to transform our data into a convenient form. In the case of the non-numerical data, we will simply map it to one by associating separate numbers for each value. On the other hand, numerical data will be rescaled using min-max normalization.
We will compare performance of different algorithms, given the task of classification of stellar objects. For the comparison, we have chosen:
• KNN (K-Nearest Neighbors) classification.
• Decision tree model.
• Naive Bayes.
If we assume we have a training dataset consisting of 𝑁 data points:
where 𝑥 𝑖 is the feature vector for the 𝑖-th point, and 𝑦 𝑖 is the class label (for classification) or value (for regression).
Then we can calculate a distance metric, typically using the Euclidean distance 𝑑 between two points 𝑥 and 𝑧 defined as:
where 𝑥 and 𝑧 are feature vectors of dimension 𝑚. To classify a new point 𝑥, we compute the distances between 𝑥 and all points in the training set, then select 𝐾 nearest neighbours and assign a class label based on the majority.
The parameter 𝐾 is a crucial hyperparameter in the KNN algorithm. A small 𝐾 can lead to overfitting, while a large 𝐾 can lead to underfitting. The optimal value of 𝐾 is often selected using cross-validation methods.
If we assume we have a training dataset consisting of 𝑁 data points:
where 𝑥 𝑖 is the feature vector for the 𝑖-th point, and 𝑦 𝑖 is the class label (for classification) or value (for regression). Then a decision tree is a tree-like model where internal nodes represents a test on a feature, branches represents outcomes of those tests and leaf node represents a class label.
To build a decision tree, we recursively split the data at each node. The choice of split is based on a criterion that maximizes the separation of the classes or reduces the prediction error.
Gini Index:
where 𝑝 𝑘 is the proportion of instances of class 𝑘 in the dataset 𝐷.
where 𝐸𝑛𝑡𝑟𝑜𝑝𝑦(𝐷) is given by: and 𝐷 𝑣 is the subset of 𝐷 where attribute 𝐴 has value 𝑣.
where 𝑦¯ is the mean of the values in the dataset 𝐷.
Assume we have a training dataset consisting of 𝑁 data points:
where 𝑥 𝑖 = (𝑥 𝑖1 , 𝑥 𝑖2 , . . . , 𝑥 𝑖𝑚 ) is the feature vector for the 𝑖-th point, and 𝑦 𝑖 is the class label from a set of classes {𝐶 1 , 𝐶 2 , . . . , 𝐶 𝐾 }.
The Naive Bayes algorithm is based on Bayes' Theorem: Additionally, we will look for the best number of neighbours for KNN classifier. We will use a few libraries to handle our operations: Sklearn [10]-will provide us with algorithm implementations, saving us a lot of time and ensuring we will be able to go through relatively big databases in reasonable time. Pandas [11] -will provide us with data structure (DataFrame). Seaborn [12] and Matplotlib [13]-will be used for visualizations, graphs, etc.
In order to find the best constant for KNN, we will launch classification in a simple loop, looking for the best solution. Generally speaking, when this number will increase our accuracy should decrease, therefore this approach is reasonable and should not take too much time. In the end, we present the confusion matrix for each of our solutions, and we will consider only two metrics:
• Accuracy (Equation 15) -to measure how many correct classifications we get.
• False categorization -in order to check if any of the classes are more often confused with others.
For our dataset, we have chosen data from Sloan Digital Sky Survey DR17 [14] (it was accessed from [15]). Which was the fourth phase of the Sloan Digital Sky Survey (we will call it SDSS-IV from now on). It contains 100000 observations, each containing (qouting [16]):
• obj_ID = Object Identifier, the unique value that identifies the object in the image catalogue used by the CAS • alpha = Right Ascension angle (at J2000 epoch) Some of that information will not be used for our classification, as they are contained in SDSS-IV for cataloguing purposes (such as object identifiers). We will focus on: coordinates alpha and delta; data from filtered channels u, g, r, i and z; class, which is the aim of our classification efforts.
After mapping and normalising our data in ultraviolet, green and infrared presented strange pattern, where basically all data is accumulated near value 1.0. Upon further inspection it turns out that one of the observed objects have some abnormal values (equals to -9999), we will remove it from our dataset and then proceed. Now we will have a look at the correlation matrix (figure 1a) and address some of the relations:
• Coordinates have neutral relations with all the other data.
• Ultraviolet and green relation-green light is a part of spectrum of many stars similar to the Sun (G-type main-sequence stars). Those stars also happens to emit significant part of their radiation as ultraviolet. An additional effect, that can also explain moderate relation with infrared and near infrared light is absorption and re-emission of different by interstellar gas, which then re-emits in those wavelengths (heat radiation) [17]. • Infrared, near-infrared and red data have strong relation -red stars are typically colder, but they still emit a lot of infrared radiation. Additional factor -absorption and re-emission of light was mentioned above. • Moderate relation of red, near infrared and infrared light with redshift can by explained by many objects detected as red having their colour shifted due to phenomenons as Doppler effect. This relation might be absent from other detectors, as light of stars different from infrared might have been cut off by stardust or shifted strong enough to not be detected at all [17].
In general, it is easy to notice strong relations with red and infrared light. This phenomenon might be related to extinction of light in the space, which is more explicit for shorter wavelengths. The coordinates of our objects are mostly related to each other (but it is still very weak relation). It also has a pure neutral relation with most of the data from detectors, therefore we are going to drop this one. Our final correlation matrix is shown for the sake of clarity in figure 1b.
Additionally, we will provide histograms for SDSS-IV data, we will plot them on to one histogram, excluding redshift, which will be shown separately for clarity (figures 2a and 2b). We will also have a look at a number of each of the individual objects in our data (figure 2c)we can notice significant dominance of galaxies. Galaxies and quasars are similar in number, with a small margin for stars.
We will split our data with at train and test set with ratio of 0.2. After running the calculation mentioned in chapter before, we get: • For KNN we get 96.465% accuracy, which was best for numbers of neighbours equal to 3 as shown in figure 3 with confusion matrix as in figure 4a. • Decision tree have achieved 96.78% accuracy (confusion matrix in figure 4b).
• Naive Bayes have achieved the lowest accuracy of 92.11% (confusion matrix in figure 4c)
Behaviour of KNN accuracy was, as expected, decreasing with relation to its constant. On the other hand, all the analysed algorithms achieved good accuracy (above 90%). Bayes algorithms turned out to have had some problems distinguishing between galaxies and quasars (almost 1000 wrongly classified galaxies), although two others algorithms also struggled there. KNN seems to deal the best with this problem, recognising even so slightly more QSO objects than others but have more mismatches, recognising some of the galaxies as stars. None of the algorithms have any problems recognising stars and rarely ever mismatches them.