Approaches for Classifying Rare Mobile-App Encrypted Trafic Samples

Abstract

Deep Learning (DL) is efective for classifying encrypted network trafic. However, it requires large amounts of labeled data to feed typical data-hungry training processes. Unfortunately, collecting and labeling rich network-trafic datasets is a costly procedure not always afordable in practice, possibly hindering DL solutions. Few Shot Learning (FSL) aims at tackling this shortcoming, providing means to leverage non-few knowledge to support trafic classification tasks with few labeled samples available. Although FSL has been largely investigated in other domains (e.g., computer vision), it has been only preliminarily adopted for trafic classification. In this work, we provide a first attempt in adopting FSL for classifying mobile-app encrypted trafic. We consider the two most popular FSL paradigms: meta learning (learn to learn) and transfer learning (knowledge transfer from related tasks). We consider a number of variants for each (i.e. MatchingNet, ProtoNet, RelationNet, MetaOptNet, fo-MAML, ANIL, Fine-Tuning, and Freezing) and provide an empirical assessment of these approaches when adopted for mobile-app trafic classification considering the trafic classification is feasible, reaching satisfactory results (up to ing in volume and changing in nature. In this context, classification approaches often take advantage of statistibilling, accounting, security as well as user-activity iden- trafic classes rapidly evolving over time, as they require ∗Corresponding author.

1. Introduction and Background

The difusion of mobile devices has dramatically modified

the landscape of network trafic, which is rapidly growtrafic classification has acquired a more and more fundamental role as it supports a number of activities related to network management, such as resource provisioning, tification and user profiling.

Indeed, trafic classification is an active research field, with the available approaches to be constantly adapted to the ever-evolving nature of the networks and the trafic traversing them. For instance, simple approaches relying on port numbers are hindered by port-independent applications or applications using standard ports to disguise their trafic. Similarly, encryption (e.g., via TLS and

HTTPS protocols) critically compromises the efective

nEvelop-O that falls short when only limited knowledge is available for some of the classes. acquired from abundant classes is used to train an em

Few Shot Learning (FSL) aims at tackling this situation, bedding function that is subsequently adapted to a new by leveraging non-few knowledge (about tasks related to task leveraging a limited number of samples. Herein, we the few one) in order to build a model capable of gener- consider a number of solutions, including M a t c h i n g N e t , alizing enough on new tasks (i.e. those with few samples P r o t o N e t , R e l a t i o n N e t , and M e t a O p t N e t from the modelavailable). Indeed, the problem of learning with few sam- based family, and f o - M A M L , A N I L , Fine Tuning, and Freezples (viz. shots) has strongly attracted the interest of the ing from the algorithm-based one. We underline that research community of several research domains, with data-based approaches are not considered at all because the way, the prior knowledge is exploited defining difer- training a generation model with few real samples may ent families of approaches [ 4 ]. FSL was initially tackled result in introducing biased synthetic samples. and in-depth explored in the computer vision domain. Accordingly, in this paper: (i) we investigate eight FSL Accordingly, the taxonomy of the available solutions is approaches that are properly adapted to the trafic classivery rich [ 4 ], including: (i) Algorithm-based approaches, fication , dealing with the problem of scarcity of data in using prior knowledge to alter the search strategy for mobile-app trafic ; (ii) we introduce a meta-learning pronew parameters, providing a baseline that may be useful cedure that takes advantage of an extra portion of classes for the specific few-shot task [ 5 ]; (ii) Model-based ap- (viz. validation classes) to enforce early-stopping in metaproaches, allowing to jointly learn a set of related initial training; (iii) we explore several directions of analysis by tasks in order to reduce the searching space of best pa- inspecting the impact of using diferent FSL setups in rameters [ 6–8 ]; (iii) Data-based approaches, leveraging terms of number of training classes (viz. ) and of shots prior knowledge to augment data of the few-shot tasks. (viz. ); (iv) we leverage the publicly available mobile

Given that such approaches proved promising in trafic dataset M i r a g e - 2 0 1 9 to foster reproducibility. other domains, recently research eforts have been made to apply FSL also in networking to address network-trafic classification. Hence, most networking- 2. Classifying Mobile-App Trafic related works dealing with FSL are typically based on with FSL previous computer-vision solutions, with minor variants being proposed [ 9–11 ]. In fact, most of the proposals 2.1. Input Data and Embedding Function leveraging FSL in networking tackle the problem of attack trafic classification [ 9–14 ]. To the best of our knowledge, We consider the bidirectional flow (biflow ) as the relevant we provide the first attempt in applying FSL to mobile-app trafic object of our analysis. A biflow is defined as an encrypted trafic classification . More in general, although aggregation of all network packets sharing the same 5the literature on FSL is rich, the studies that apply these tuple (i.e. source IP and port, destination IP and port, solutions to network-trafic classification are limited and and transport-level protocol) including both directions do not explore the whole range of possibilities that FSL of communication. provides. Often, the evaluation of the solutions investi- To feed the FSL methods exploited herein, we extract gated in the trafic-classification domain is also limited, as a set of informative fields from the sequence of the first proposals are only compared against others not tailored packets of each biflow: (i) the number of bytes in for FSL [ 9, 11, 13, 14 ] (thus, emphasizing the benefits of transport layer payload; (ii) the packet direction (can be FSL but not investigating those related to specific FSL −1 or 1); (iii) the TCP windows size (equal to 0 for UDP families or approaches). packets); (iv) and the elapsed time since the arrival of the

In this work, we provide an extensive empirical as- previous packet (i.e. inter-arrival time). The input data sessment of two popular FSL paradigms: meta learning are Min-Max normalized within [ 0, 1 ]. and transfer learning. Meta learning improves the per- Additionally, all FSL models embed the input data formance of a new task given the meta-knowledge ex- into a lower-dimensional space using an embedding functracted across tasks by a meta-learner. In a few-shot tion. The specific embedding function used herein is a context, meta learning is used jointly with episodic learn- state-of-the-art DL network widely used in and suited ing, which consists in organizing the training phase in a for the trafic classification domain [ 15, 16 ].1 Specifically, series of learning problems—also called episodes—in or- the embedding function is a single-modal bidimensional der to allow a model ( ) to quickly learn novel knowledge convolutional neural network (2D-CNN) whose architecby generalizing from previously encountered learning ture and hyperparameters are those originally proposed tasks and ( ) to distinguish unknown classes given few for trafic classification [ 16 ]. samples. Diferently, transfer learning aims to transfer knowledge from a task to a related one with the objective of fast adaptation, reduced complexity, and performance 1sWtaeteh-aovfe-tsheele-acrtetdoubtoctohminepsu[t15d,a1ta6]anadnde mabperdeldiimnginfaurnycetixopnebriamseedntoanl improvements. In the few-shot scenario, the knowledge campaign whose results are not reported for the sake of brevity.

2.2. Few-Shot Learning Paradigms

Meta Learning. Meta-learning approaches require a preliminary phase to manage the generation of N-way K-shot episodes. In detail, given a dataset = { 1, 2, }, the three subsets have a disjoint label space. includes the samples from the less populated (i.e. few-shot) classes, while 1 and 2 contain non-few classes. Detailing, each episode is formed by randomly sampling classes (N-way). 1, 2, and are used during training, validation, and testing phases, respectively. Defining an episode consists in constructing two (non overlapping) partitions: () the support set having × samples (where defines the -shot setup) and () the query set having × samples.

Based on this episode creation, () during a metatraining phase, the model learns from a set of tasks— i.e. N-way K-shot classification tasks —using samples from the support set and measures the error on the query set. Then, () the generalization ability of the classiifer is tested with a meta-testing phase, using an analogous episode-based procedure. Unlike the common meta-learning procedure, we perform an additional () meta-validation phase leveraging 2.

More specifically, meta-training is performed for a certain number of epochs in an episodic manner. On the other hand, 2 is exploited to enforce an early-stopping procedure based on the accuracy attained on it and to select the model showing the best performance on it after the training procedure is completed. Finally, meta-testing is performed on this best-performing model. We remark that during meta-testing the performance achieved by the selected model is properly evaluated on . Transfer Learning. Transfer learning aims at learning generic features from a set of non-few classes ( task) and then specializes in the few-shot context ( task). Accordingly, similar to the meta-learning, = { 1, }, with the latter encompassing the few-shot classes. However, transfer learning needs to further split 1 and (e.g., via hold-out) into a training set and a test set, which are analogous to the support and the query sets in meta learning, respectively. Specifically, learns (resp. evaluates the error) with the training (resp. test) data from 1. Similarly, enriches the knowledge obtained on with a set of data obtained from the samples belonging to . To mimic the support set of the meta-learning procedure, during we sampled from the training set shots per class.

2.3. Few-Shot Learning Approaches

Here, we describe the FSL approaches we deal with. They belong to the model- and algorithm-based families. All (but transfer learning ones) implement meta learning.

Model-based Approaches. Model-based methods learn by constraining the hypothesis space to a smaller one through the use of prior knowledge, with the aim of reducing the risk of overfitting. More specifically, they apply a dedicated embedding function—learned based on prior knowledge extracted from the meta-training tasks—to support and query samples mapping them in a lower-dimensional space. In such a way, similar samples are closer to each other, whereas dissimilar samples are more easily diferentiable (i.e. the hypothesis space is reduced). Then, the embeddings of the support set are used by a comparator to classify the embedded query set by measuring their similarity.

Model-based approaches difer by the comparator and then the similarity mechanism leveraged. We consider the following approaches: (i) Matching Networks (M a t c h i n g N e t ) [ 6 ] performs a generalized form of nearestneighbors classification based on Euclidean distance; (ii) Prototypical Networks (P r o t o N e t ) [ 7 ] classify a sample via a Euclidean distance function calculated between its embedding and a prototype, i.e. a centroid representative of a class; (iii) Relation Network (R e l a t i o n N e t ) [ 8 ] employs a relation module based on a convolutional network that measures the similarity between the embeddings of query and support samples of each class; (iv) M e t a O p t N e t [ 17 ] exploits a linear Support Vector Machine, trained on labeled support samples, as the comparator and measures the generalization error on query samples.

Algorithm-based Approaches. Learning models belonging to the algorithm-based category search in the hypothesis space for the parameter set corresponding to the best hypothesis in such a space. Unfortunately, in few-shot scenarios, the samples available for training are limited to properly update the parameter set, thus resulting in an unreliable risk minimizer. To deal with this issue, algorithm-based methods exploit the prior knowledge to influence how the parameter set is obtained.

In the present paper, we exploit M A M L [ 5 ] and A N I L [ 18 ], which are meta-learning approaches that aim at refining meta-learned parameters by learning an initial parameter set via a meta-learning procedure and further refining it using . M A M L —short for Model-Agnostic MetaLearning—continuously updates the initial meta-learned parameter set based on the performance attained on the episodic tasks (viz. inner-loop adaptation) with the aim of ifnding a highly adaptable set of parameters. To mitigate the well-known computational burden of M A M L , herein we employ a simpler but almost equally well-performing version named First-Order M A M L (f o - M A M L ) [ 5 ] that only uses first-order gradients during parameter optimization. In addition to f o - M A M L , we also evaluate A N I L —short for Almost No Inner Loop—a simplified version of M A M L being equally efective but computationally faster. A N I L removes the inner-loop updates for the embedding function during meta-training and meta-testing and applies them only to the model head.

3. Experimental Setup and

Evaluation

3.1. Experimental Setup

3.1.1. Dataset

In this study, we leverage the publicly released M i r a g e

2 0 1 9 dataset [ 2 ] containing trafic of 40 Android apps.2 It was collected at the University of Napoli “Federico II” in 2017–2019 by involving 300 voluntary students mimicking typical app use cases. Ground truth is obtained by using metadata log files collected during each capture: each biflow is labeled with the related Android-package name. The number of biflows per app depends on the specific app and ranges from 361 to 8246, despite the time that apps are used was roughly the same. This denotes a real-world and challenging scenario for the mobile-app trafic classification task considered. 3.1.2. Few-Shot Learning Setup

The FSL setup is described by detailing the common meta

learning setup and the configuration of FSL models. Meta-Learning Setup. We explain the meta-learning setup in terms of dataset partitioning and episode definition. The classes (viz. apps) of M i r a g e - 2 0 1 9 are partitioned into three disjoint sets corresponding to the apps considered to build 1, 2, and , respectively (Sec. 2.2 describes their use). More specifically, 1 includes the 24 most populous apps of M i r a g e - 2 0 1 9 , 2 consists of the most populous 8 apps besides those in 1, and the last subset includes the 8 remaining least-populated apps (i.e. with less than 1000 samples).

2https://traffic.comics.unina.it/mirage/mirage-2019.html

We also employ two transfer learning approaches Once the dataset partition is set, the definition of (i.e. Fine-Tuning and Freezing) that allow the model to meta-learning episodes is based on how (ways) and learn an initial parameter set from other tasks ( 1) and (shots) are selected (see Sec. 2.2). For meta-training, then refine it using . More specifically, Fine-Tuning we set and according to the goals of our analyses (T L FT ) involves: ( ) learning the initial model parameters (always considering = = ). For meta-validation through task and, ( ) refining the weights—during the and meta-testing, we set = 8 and = 100, to classify training phase of —starting from the values computed the samples belonging to all classes in 2 and and in the preceding task using only samples from few-shot improve the coverage and the stability of results. classes. This results in significantly faster execution. On Regarding the common hyperparameters, characthe other hand, T L FT is afected by the problem of forget- terizing all FSL models, the best configuration has 200 ting the old classes. Freezing (T L FZ ) freezes the weights epochs, each encompassing 100 episodes, and the Adam of the embedding function when a new task is presented optimizer set with 10−4 learning rate and a learning rate and allows the update of the weights associated with the scheduler having step size of 20 and decay of 1.0. To mitimodel head (diferently from T L FT where both the em- gate the overfitting, we exploit an early-stopping mechabedding function and head are updated). The expected nism that monitors the accuracy on 2 and has minimum result is that the model retains the previously learned delta of 0.01 and patience of 20 epochs. Finally, regarding knowledge, but also adapts to few-shot classes in case of input data (see Sec. 2.1), we use = 10. This choice is proper generalization capability. motivated by the outcome of the sensitivity analysis we conducted in [ 2 ] on M i r a g e - 2 0 1 9 using the same embedding function considered herein. 3.1.3. Performance Metrics To evaluate the performance of FSL methods, we use the macro F1-score and the silhouette score. For both metrics, we show the per-episode mean and related standard deviation attained on .3 The macro F1-score is the harmonic mean of per-class precision and recall arithmetically averaged over apps. We leverage the F1-score for both meta-learning (whose episodes are balanced by construction) and transfer-learning (sufering from data imbalance) approaches because it is more robust than the common accuracy when working with skewed data. The silhouette score quantifies how similar a sample is to its own cluster compared to the others, and it ranges from −1 (worst) to +1 (best). This is paramount since most meta-models behave like a nearest-neighbor classifier in the embedded space.

3.2. Experimental Evaluation

3.2.1. Sensitivity to N

This experimental campaign investigates the trend of

the F1-score when varying the number of train ways .4 More specifically, ranges from 2 to 8 with a step of 2, and the number of shots is kept constant at = 25 for both query and support sets (i.e. -ways 25-shots meta-training episodes).

Results in Fig. 1 show that the number of train ways has little impact on the performance of FSL approaches, with 3For transfer-learning approaches we perform multiple (starting from the same ) by randomly sample biflows over 10 runs. 4Note that the algorithm-based approaches (i.e. fo-MAML and ANIL) require that the same value of is used for both meta-training and meta-testing, not allowing for this specific analysis.

MetaOptNet MatchingNet RelationNet ProtoNet ANIL fo-MAML TLFT TLFZ MetaOptNet MatchingNet RelationNet ProtoNet ANIL fo-MAML TLFT TLFZ

This section presents the sensitivity analysis of FSL al

gorithms when considering a variable number of shots (i.e. a fully-connected layer) on a poorly separable latent . F∈or{5m,1e5t,a2-5tr,a5i0n, i1n0g0}epinisboodtehs, qwueercyonasniddesrup=po rt set=s sopf ascheoststililnocbretaaisness.aFhiingahlleyr, aucsciunrgacaysiwmhpelne tchoemnpuamrabtoerr and = 8 . For meta-testing and meta-validation ones, (i.e. M a t c h i n g N e t and P r o t o N e t using Euclidean distance) follows the values used in meta training. does not lead to an easily-separable latent space—which

Figure 2 exhibits that all FSL approaches significantly is less separable with higher values of —but it is increasincrease their F1-score for higher values of train shots. The ingly capable of distinguishing classes when the number best-performing approach (i.e. M a t c h i n g N e t ) achieves an of shots grows.

F1-score > 80% when = 100 , despite starting from 50% From this analysis emerges that the selection of the apwhen = 5 . T L FT , T L FZ , and M e t a O p t N e t similarly have proach has an impact on the embedding function training. satisfactory performance. P r o t o N e t , A N I L , and f o - M A M L Particularly, the way the embeddings are manipulated do not exceed 60% F1-score, despite a sharp rise when by the comparator/classifier has the most impact. passing from = 5 to = 15 can be observed. 3.2.3. Embedding Efectiveness vs Performance

Because of the relevance of the problem FSL aims to solve,

This analysis relates classification performance (mea- it has attracted the interest of research communities sured via F1-score) to the capability of the embedding working in several domains, including networking and function to separate app-clusters into the latent space trafic classification. Because FSL originated in the field (expressed via silhouette score). Accordingly, Fig. 3 de- of computer vision [ 5–8, 17, 18 ], applications of FSL to picts the related scatter-plot by considering 8-ways - the networking domain mostly exploit minor variants of shots episodes with ∈ {5, 15, 25, 50, 100} . Three main solutions designed for computer vision. Hereinafter, we trends emerge for diferent FSL approaches: (i) corre- discuss the most relevant studies facing trafic classificalated, for A N I L , f o - M A M L , and M e t a O p t N e t ; (ii) not correlated, tion in a few-shot context, underlining their key aspects for R e l a t i o n N e t and T L FZ ; (iii) negatively correlated, for and the changes made w.r.t. computer vision-tailored FSL P r o t o N e t , T L FT , and M a t c h i n g N e t . In detail, considering approaches that inspired them. Notably, all the relevant the peculiarities of each approach, these results can be studies [ 9–14, 19, 20 ] date to 2018–2022, witnessing the summarized as follows. First, exploiting complex com- recent interest of the networking community in this topic. parators (i.e. A N I L , f o - M A M L , and M e t a O p t N e t ) leads to more Additionally, we select these works mainly because they separable clusters: the better accuracy, the higher num- focus on the problem of classifying unseen trafic with ber of shots used. Then, T L FZ (and also T L FT ) behavior just few samples. On the other hand, we do not consider clearly demonstrates that training a complex comparator other studies dealing with FSL that perform the model 4. Related Work evaluation on seen classes (i.e. where the main goal is simply mitigating the class imbalance in data). Regarding the specific task addressed, a reduced number of works face (encrypted) trafic classification [ 11, 19, 20 ]. However, diferently from the present paper, none considers mobile-app trafic . Conversely, a higher number of works leverage FSL in the (similar) context of attack trafic classification and intrusion detection [ 9, 10, 12–14 ]. The FSL approaches applied in these contexts belong to modelbased and algorithm-based families. Most of them also exploit prior knowledge via meta learning, whereas only the oldest works [ 19, 20 ] leverage the transfer-learning paradigm. We underline that only two studies [ 10, 12 ] evaluate more than one FSL approach. This aspect highlights the lack of a wide comparison among the various few-shot families in the trafic-classification literature.

5. Conclusions

In this paper, an extensive empirical study of mobile

trafic classification through diferent FSL approaches has been performed, as a means to cope with the problem of scarcity of training data for some of the classes. The evaluation relied on M i r a g e - 2 0 1 9 , a publicly available mobile-app dataset. The analyses assessed: (i) the impact of changing the number of classes in episodic learning, (ii) the sensitivity to the number of samples available for each class, and (iii) the relation between the goodness of embedding space and the classification performance.

The performance figures of a number of meta-learning FSL algorithms have been assessed: P r o t o N e t , M a t c h i n g N e t , R e l a t i o n N e t , M e t a O p t N e t , M A M L , and A N I L . Transfer-learning approaches were also considered, i.e. Fine-Tuning, and Freezing. We found that (i) the number of train ways has little impact on the performance of FSL approaches, (ii) all FSL approaches significantly increase their F1-score for higher values of train shots, and (iii) performance of FSL approaches with less (resp. more) complex comparators obtains high F1-score and bad (resp. good) embedding separation.

Based on the outcomes of the experimentation, we identify various improvement avenues. These will explore novel FSL methods in multiple directions linked to (i) the optimization of the learning objective (using more complex loss functions to enhance the goodness of embeddings), (ii) the adoption of diferent embedding functions (e.g., multimodal architectures) to explore their benefits, and (iii) the implementation of a real prototype that can scalably run over a real network infrastructure.

Acknowledgements

This work is funded in the framework of the Huawei Innovation Lab project on “Network Trafic and AI enabled

Network Technologies” by Huawei Technologies France SASU at DIETI, University of Napoli Federico II.

References

[1] E. Papadogiannaki et al. A survey on encrypted network trafic analysis applications, techniques, and countermeasures . ACM Computing Surveys , 54 ( 6 ): 1 - 35 , 2021 .

[2] G. Aceto , et al. Mirage: Mobile-app trafic capture and groundtruth creation . In 4th IEEE International Conference on Computing, Communications and Security (ICCCS) , pages 1 - 8 , 2019 .

[3] C. Wang , et al. Appclassnet: A commercial-grade dataset for application identification research . ACM SIGCOMM Computer Communication Review , 52 ( 3 ): 19 - 27 , 2022 .

[4] Y. Wang , et al. Generalizing from a few examples: A survey on few-shot learning . ACM computing surveys , 53 ( 3 ): 1 - 34 , 2020 .

[5] C. Finn , et al. Model-agnostic meta-learning for fast adaptation of deep networks . In Proceedings of the 34th International Conference on Machine Learning , volume 70 , pages 1126 - 1135 , 2017 .

[6] O. Vinyals , et al. Matching networks for one shot learning . Advances in neural information processing systems , 29 , 2016 .

[7] J. Snell , et al. Prototypical networks for few-shot learning . Advances in neural information processing systems , 30 , 2017 .

[8] F. Sung , et al. Learning to compare: Relation network for fewshot learning . In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1199 - 1208 , 2018 .

[9] W. Zheng , et al. Learning to classify: A flow-based relation network for encrypted trafic classification . In Proceedings of The Web Conference 2020 , pages 13 - 22 , 2020 .

[10] Z.-M. Wang , et al. A few-shot learning-based siamese capsule network for intrusion detection with imbalanced training data . Computational Intelligence and Neuroscience , 2021 : 7126913 , 2021 .

[11] Z. Zhao , et al. A few-shot learning based approach to iot trafic classification . IEEE Communications Letters , 26 ( 3 ): 537 - 541 , 2022 .

[12] C. Rong , et al. Umvd-fsl: Unseen malware variants detection using few-shot learning . In 2021 International Joint Conference on Neural Networks (IJCNN) , pages 1 - 8 , 2021 .

[13] S. Huang , et al. A gated few-shot learning model for anomaly detection . In 2020 International Conference on Information Networking (ICOIN) , pages 505 - 509 , 2020 .

[14] T. Feng , et al. Few-shot class-adaptive anomaly detection with model-agnostic meta-learning . In 2021 IFIP Networking Conference (IFIP Networking) , pages 1 - 9 , 2021 .

[15] G. Aceto , et al. Mobile encrypted trafic classification using deep learning: Experimental evaluation, lessons learned, and challenges . IEEE Transactions on Network and Service Management , 16 : 445 - 458 , 2019 .

[16] M. Lopez-Martin , et al. Network trafic classifier with convolutional and recurrent neural networks for internet of things . IEEE Access , 5 : 18042 - 18050 , 2017 .

[17] K. Lee , et al. Meta-learning with diferentiable convex optimization . In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10657 - 10665 , 2019 .

[18] A. Raghu , et al. Rapid learning or feature reuse? towards understanding the efectiveness of maml . arXiv preprint arXiv: 1909 .09157, 2019 .

[19] Y. Xiao , et al. Common knowledge based transfer learning for trafic classification . In 2018 IEEE 43rd Conference on Local Computer Networks (LCN) , pages 311 - 314 , 2018 .

[20] S. Rezaei et al. How to achieve high classification accuracy with just a few labels: A semi-supervised approach using sampled packets . arXiv preprint arXiv:1812.09761 , 2018 .