Since the date of establishment of the SETI Institute, its scientists have used various approaches in their search for extra-terrestrial intelligence (SETI). A novel idea involved image categorisation techniques in classifying radio signals represented by 2D spectrograms. The dataset of simulated radio signals, created for classi cation purposes have been used in this work to train models based on neural network architectures. It is shown in this paper that combining three di erent models, trained on features obtained by various techniques, has a positive impact on model accuracy and performance. Features learned by a convolutional neural network (CNN), bottleneck features from existing models and manually extracted features from the spectrograms comprised the three feature sets used as training data for the combined model. It was also shown that combining di erent methods of spectrogram generation resulted in improving the accuracy of the nal model.
Scientists from the SETI Institute (Search for Extra-terrestrial Intelligence)
monitor radio signals coming from multiple directions from space, searching for signs
of extra-terrestrial intelligence [
The ATA radio telescopes record analogue voltage signals representing
electromagnetic eld oscillations reaching individual antennas. Before the signals are
stored, they are demodulated down from GHz range to lower frequencies and
digitised, which results in a stream of complex-valued time-series data. The
digitisation process produces 104 million complex samples per second, corresponding
to about 70MHz bandwidth window around central frequency from the range
110 GHz, which is the observation range for ATA [
Usage of frequency domain for signals oscillating in time, in many use cases gives a more meaningful illustration of signal's characteristics than a time domain representation. A transition from time to frequency domain can be achieved by Fourier Transform (FT), based on the assumption that each periodic signal can be decomposed to a sum of sines and cosines:
1 S(t) = 2 a0 + 1 X an cos n!t + n=1 1 X bn sin n!t n=1 or, if represented in a complex space, it can be expressed as
S(t) =
1
X cnejn!t
where S(t) is the approximation of signal s(t) in the time domain and ! is signal
frequency. The coe cients an and bn or cn describe a signal in the frequency
domain and they are expressed as:
and
where T is an oscillation period. The link between sine, cosine and
exponential functions is provided by the Euler formulas [
(a) narrowband
(b) squiggle
and parameters of individual categories have been presented in detail by [
The following section describes the methods utilised in this work and the data pre-processing steps, which are as follows: { obtaining spectrograms from time-series data { spectrograms enhancement { exploration of existing Convolutional Neural Networks (CNN) { creation of manual features { building the combined architecture 3.1
The simulated data consisted of a series of complex numbers in the time domain. Complex numbers are often chosen for mathematical convenience, when it is required to perform operations on amplitude and frequency.
For example, a signal from a real space written as x(t) = A1 cos !t in a complex space can be expressed as x(t) = A2ej!t and then by using the Euler formula (x2), it can be written as a sum of Real and Imaginary components: x(t) = A2 cos !t + A2j sin !t = a(t) + jb(t) The raw data used in this work consisted of numbers of the above pattern.
The following steps were performed to obtain spectrograms from the
simulated data:
1. Time-series data has been split into time intervals. The length of the interval
would later re ect a time unit on the spectrogram's vertical axis.
2. The time segments have been stacked horizontally to create a 2D array.
3. The array has been processed with Hann window [
This step was performed only in some cases, as described later. 4. Fourier Transform (FT) has been performed on each time segment and a square of its modulus has been taken. 5. A logarithm has been applied on the FT results. This operation brings higher granularity into the lower ranges of a colour histogram. This step was not done in all of the cases, which will be discussed later. 6. The resulting values have been normalised to the range 0-255, for the array to be plotted as an image, where brighter colours correspond to higher signal power. 3.2
The maximum image size attainable from the simulated data was 196,608 pixels, re ecting a resolution of 384x512. However, for the majority of use cases lower resolution images (128x256) have been created.
Monochromatic pictures have only one colour channel, whereas colour
pictures can have 2-4 channels; e.g. the RGB colour space uses 3 channels where
each colour { red, green and blue are described separately on a scale from 0 to
255. Convolutional networks are therefore well suited to operate on colour images
by convolving 3-dimensional lters simultaneously through all colour layers and
forming independent weight tensors. Even though spectrograms are
monochromatic, in this research separate channels have been used to carry the results of
di erent techniques of image generation. The following methods were explored
for this purpose:
{ a logarithm has been applied over the FT (Figure 2a),
{ the logarithm has been omitted (Figure 2b),
{ Hann window has been applied before the FT and the logarithm was taken
from the FT result (Figure 2c),
{ Radon transform [
(b) Spectrogram obtained without applying the logarithm; no Hann function used.
(c) Spectrogram obtained using the Hann window before Fourier Transform and the logarithm over the FT output.
While the Radon transform can be useful in detecting straight lines within
a noisy background [
Prototyping with CNN's has been performed on 4 categories, with total size of the data set being 4000. Further method exploration was performed on a set of 7000 signals, including all 7 categories, whereas the full data set size used here was 35,000 signals. The CNN architectures, which have been tested in this research include: { 3 and 4-convolutional-block networks of Convolutional + ReLU4 + Max
Pooling5 layers with variable lter sizes: 3 { 7,
{ VGG16 and VGG19 [
Most of the above networks were used through transfer learning, rather than training from random weights. The CNNs of 3 and 4 convolutional blocks, as well as VGG16 have been fully trained from randomly distributed weights, yet the accuracy from the VGG did not outperform the former one.
In addition to the above architectures a few variants of combined models were tested, such as DenseNet or ResNeXt accompanied with a neural network of fully connected layers trained on manually crafted features. 3.4
Apart from variants of CNN's, a more traditional machine learning method was
also part of this research. A regular neural network was trained on a set of
features, which were manually crafted to re ect the main characteristics of all
classes. The following features were generated:
{ the number of pixels in each of the 4 brightness ranges, such as: 0 { 62.5,
62.5 { 125, etc., marked as Q1, { Q4 in Figure 3a,
{ characteristics of pixel brightness distribution, such as mean, variance and
skewness,
{ estimated width of the bounding box around the signal, as shown in
Figure 3b; this was obtained using a custom function,
{ estimated total height of individual bounding boxes, as in Figure 3b; this
was generated using a custom method,
{ estimated length of the signal's line using Canny edge detection algorithm
[
The width and height of a signal line were calculated by counting total number of columns or rows containing pixels of a brightness over a prede ned threshold. (a) Colour (brightness) histogram of a sample spectrogram with annotated four quarters.
(b) Bounding boxes around signal labelled with width and height.
Fig. 3: Illustration of extraction of some of the features. 3.5
Residual Networks (ResNet) [
performance out of all the networks tested [
Final model implementation. The model developed in this work is shown in Figure 4. Pretrained on the ImageNet dataset, the DenseNet201 model has been used for extracting features out of the training dataset (bottleneck features), by trimming the top part of its architecture and using only the convolutional part. A separate 4-convolutional-block CNN was trained on the spectrogram data from random weights, which produced similarly structured bottleneck features. Two fully connected layers were then added on top of each model and the output was passed onto the common layer. A third element of the whole architecture was a regular network with one hidden and one output layer trained on manually extracted features. The output produced was passed again to the common layer, which was comprised of the outputs of all three individual models. The common layer combined the outputs together, giving each of them the same number of features. Two fully connected layers were added on top of the common layer, with the softmax activation, producing the nal output of probabilities for seven categories. In order to achieve the optimal batch size for training, the split of 35,000 pictures into train, validation and test datasets was performed with the ratio of 76:12:12 respectively. The data split have been performed once and then the same subsets were fed into three models. The test data was used at the evaluation stage only, after the training process had completed. During training, images were loaded in batches in order to accommodate available memory resources. In order to overcome over tting, dropout was applied onto three branches of the combined model.
The gradient descent process was accomplished with the Adam optimiser [
From among the single models, the best accuracy has been achieved by the
CNN with 4 convolutional blocks, though the combined model outperformed all
single model architectures. While other combined architectures were explored,
the one combining DenseNet with manual features (MF) and the CNN-trained
features yielded the best results. The Table 3 shows the accuracy for all
categories. The category with the highest F1 score was squiggle, which by visual
inspection is also the most obvious to recognise. The next best recognised signal
is brightpixel. While the precision of brightpixel detection is quite high, the
recall is not as good, as many actual brightpixel signals were not
distinguishable from noise signals. The noise classi cation has relatively low precision and
in addition to brightpixel, many other signals were misclassi ed as noise. The
lowest confusion with noise appears for squiggle and narrowband. As the
correct classi cation of the narrowband category is of the greatest interest to SETI
researchers [
In this work a means of improving classi cation accuracy (in relation to other examined methods) by both enhancing the source data and consolidating NN architectures was presented. Three methods were integrated by creating spectrograms from raw time-series data into 3 image channels and the inclusion of manually extracted features from the spectrograms. In addition, the consolidation of three neural network architectures into a single model was implemented. By using these methods an accuracy of 87.86% with F1 scores ranging from 0:83 to 0:93 was obtained. This research shows that including manually extracted features helps in classifying speci c categories, such as the SETI category noise. Finally, this work showed favourable results in comparison to competitor methods implemented on more advanced architecture environments. Further work in this area could be a promising direction for additional model enhancement. Future work seeks to explore the addition of a wider range of manual features, as well as other denoising techniques.