Few Shot Learning (FSL) is a supervised Machine Learning (ML) problem which deals with learning with few labeled samples. To address the challenges of FSL in terms of low-resource perspective, in this paper, we describe the models submitted to "Profiling Cryptocurrency Influencers with Few-shot Learning" - a shared task in PAN@CLEF 2023. The task has a focus on English Twitter posts for three subtasks: i) Subtask1 - Low-resource Influencer Profiling, ii) Subtask2 - Low-resource Influencer Interest Identification and iii) Subtask3 - Low-resource Influencer Intent Identification, with a very small set of labeled data. Two models: i) FSL-Word2Vec - Linear Support Vector Classifier (LinearSVC) trained with word embeddings extracted from Google's pre-trained Word2Vec and ii) FSL-ST - LinearSVC trained with sentence embeddings obtained from stsb-bert-base, are submitted to the shared task. FSL-Word2Vec models obtained macro F1 scores of 46.66 and 50.42 for Subtasks 2 and 3 respectively and FSL-ST model obtained a macro F1 score of 37.92 for Subtask1.
Cryptocurrency is a digital currency created as an alternative form of currency using encryption algorithms. The transactions in cryptocurrency are verified and maintained by a decentralized system using cryptography, rather than by a centralized authority. Today, there are more than 22,789 diferent cryptocurrencies, with an estimated total value of 1 trillion dollars 1. Some of the top cryptocurrencies with billions of dollars in the market include Bitcoin, Ethereum, Tether, Binance Coin, and Dogecoin.
Social media platforms are playing a prominent role in expanding the cryptocurrency
communities to reach the general public. The rising ubiquity of speculative trading of cryptocurrencies
over social media has led to sentiment driven “bubbles” [
Crypto Influencers have built a sizable followers on social media sites by expressing their thoughts and observations on cryptocurrencies. However, profiling Cryptocurrency Influencers in social media is very challenging due the very informal kind of data that is available on social media platforms. Given the social media data about the impact the cryptocurrency Influencers have created and the category to which these Influencers belong, profiling the Influencers can be modeled as a text classification problem. However, in a real environment, data collection is a major challenge and real-time profiling needs to be done in a few milliseconds, which implies to process as little data as possible. This demands for the tools which can make predictions accepting very less data.
Conventionally, large amount of annotated data is required to train the ML models. However,
collecting, annotating, and validating large data is very expensive. Further, there are many
cases where it is just next to impossible to collect large datasets or the available large datasets
may not be accessible for public. For example, rare diseases would not have a large number of
radiological images or it would be frustrating if smartphones need to have thousands of pictures
of users to recognize them and get unlocked. In such scenarios, ML models trained on a small
number of samples results in low performance on the Test set. A solution to such scenarios
is FSL which aims to build accurate ML models with less training data [
"Profiling Cryptocurrency Influencers with Few-shot Learning" is a shared task 3 in PAN
@CLEF 2023 which aims to profile Cryptocurrency Influencers in social media, from a
lowresource perspective [
32 users per label with a maximum of 10 English tweets each 64 users per label with 1 English tweets 64 users per label with 1 English tweets each 160
220 320
402 256 292 less data provided by the organizers of the shared task. The proposed models makes use of Google’s pre-trained Word2Vec4 and stsb-bert-base5 - a Sentence Transformer (ST), to represent text which in turn are used to train LinearSVC models to predict the probabilities of the samples belonging to the classes in the predefined set of classes of a subtask and assign the label of the class having highest probability to the Test sample.
The rest of the paper is organized as follows: Section 2 gives a brief description of the related work followed by the methodology in Section 3. The experiments and results are discussed in Section 4 and the paper concludes with future work in Section 5.
Profiling Cryptocurrency Influencers and FSL are relatively the new fields for NLP researchers [
Wang et al. [
Parnami and Lee [
Many researchers have worked on Author Profiling considering a substantial amount of data. But, very few works are explored for Author Profiling with few labeled samples. Hence, exploring FSL for profiling Cryptocurrency Influencers with few labeled samples has enough scope and opens up new avenues of research in this topic.
Two models, FSL-Word2Vec and FSL-ST are proposed to address the challenges of the shared
task and these models difer primarily in pre-processing and the way of representing texts. The
dataset shared by the organizers of the shared task is collected from Twitter [
The framework of the proposed methodology is visualized in Figure 1 and the proposed models are described below: 3.1. FSL-Word2Vec • Pre-processing - emojis are converted to the corresponding text and the entire text is lower cased. Further, the noisy content, punctuation, and the stop words are removed from the text. English stopword list9 available at NLTK library is used to remove the stop words and Porter Stemmer10 is used to strip the afixes from the words. The remaining content in the pre-processed sentences are given as input to the text representation module. • Text Representation - deals with how eficiently text documents are represented. The introduction of Word2Vec by Mikolov et al. [11] gave rise to the representation of words by a fixed dimension dense vector of small size like 50, 100, 200 and 300. These vectors which are called as pre-trained vectors/embeddings will be trained on a very large corpus. With this representation, any text can be represented as an aggregation of the representation of words. Leveraging pre-trained embeddings for word representation will help to capture the semantic knowledge, even when the task-specific dataset is limited. Google’s pretrained Word2Vec11 is one such implementation trained on roughly 100 billion words 9https://www.nltk.org/search.html?q=stopwords 10https://www.nltk.org/api/nltk.stem.porter.html 11https://huggingface.co/fse/word2vec-google-news-300 from a Google News dataset. Word2Vec captures the general semantic relationships, patterns, and domains and contains vectors of size 300 for 3 million words and phrases. Using Google’s pre-trained Word2Vec model, the text is represented as follows: – for each word in a sentence, a vector of 300 dimension is extracted (if the word is not in the vocabulary, a zero vector of length 300 is used) – for each sentence, the mean of the vectors of all the words in that sentence is obtained – for each text, the mean of the vectors of all the sentences in that text is obtained With this arrangement, each text/sample in the dataset will be represented by a dense vector of size 300. 3.2. FSL-ST • Pre-processing - as ST represent the sentences, very limited pre-processing is carried out on the sentences to maintain the sentence structure. This includes converting emojis to the corresponding text and removing the noisy content from each sentence in the given text. Stopword removal, Stemming and converting text to lowercase, are not performed to retain the sentence structure. The pre-processed sentences are given as input to the text representation module. • Text Representation - ST is a Python framework for generating contextualized embeddings for sentences. By embedding sentences into a vector space, ST enables the proximity of similar sentences, allowing for various applications such as semantic search, clustering, and retrieval. With its user-friendly methods, ST simplifies the process of generating embeddings, exhibits state-of-the-arts performance, provides multilingual support, and is reliable as it belongs to open source community. Few-shot and zero-shot approaches have received a great deal of interest in the research community due to the availability of ST and the untapped capacity to use them in resource-constrained domains [9]. stsb-bert-base is a ST model that maps sentences and paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Using stsb-bert-base12, the text is represented as follows: – Each sentence in a text is represented by a vector of 768 dimension – for each text, the mean of the vectors of all the sentences in that text is obtained With this arrangement, each sample in the dataset will be represented by a dense vector of size 768. Hyperparameters and their values of stsb-bert-base are shown in Table 2.
The motivation for using FSL is to learn with few available samples. As the number of training samples for each subtask is very less, the vector representations obtained for the text samples of each subtask are grouped according to the classes/categories and the average of these representations is obtained. This arrangement reduces the number of training samples in the 12https://huggingface.co/sentence-transformers/stsb-bert-base subtask to the number of classes in that subtask. These new training samples which are the representative of each class are used to train the LinearSVC model.
To evaluate the model on the Test set, the samples in the Test set will be pre-processed and represented as discussed in Sections 3.1 and 3.2. These vector representations are fed to the LinearSVC model to predict the probabilities of the samples belonging to the classes in the predefined set of classes in the subtask and assign the label of the class having highest probability to the Test samples.
The statistics of the dataset for the subtasks shown in Table 1 indicates that the given datasets are very small in size. If ML classifiers are used in a conventional way, this small size dataset will pose challenges for achieving good performance and generalization on unseen data. This justifies the need for using FSL for profiling Cryptocurrency Influencers.
As the Development set is not provided by the organizers, several experiments were conducted initially by splitting the given Train set into Train and Test set and considering diferent word representations, ST and classifiers. The combination of word representation - Google’s pretrained Word2Vec, ST - stsb-bert-base and classifier - LinearSVC, which exhibited the best performance in the initial experiments are considered for the actual experiments to obtain the probabilites and predictions on the Test set which in turn are submitted to the organizers for evaluation. The predictions are evaluated based on the macro F1 score as it considers the average F1 score across all classes, providing a comprehensive measure of model efectiveness. Table 3 gives the performances of the proposed models for all the three subtasks. Among the proposed models, FSL-Word2Vec model obtained better macro F1 scores of 46.66 and 50.42 for Subtask2 and Subtask3 respectively whereas FSL-ST model obtained macro F1 score of 37.92 for Subtask1. The very low macro F1 score for Subtask1 may be because the number of training samples is only 160 which is very small to train any ML model. The performances of both the models for Subtask3 are better compared to that of the other two subtasks. Further, there are no much diferences in the performances of the two models in all the subtasks.
This paper describes the FSL models proposed for "Profiling Cryptocurrency Influencers with Few-shot Learning" shared task in PAN@CLEF 2023. The proposed methodology consists of two models: i) FSL-Word2Vec - LinearSVC model trained with embeddings extracted from Google’s pre-trained Word2Vec and ii) FSL-ST - LinearSVC model trained with sentence embeddings obtained using stsb-bert-base. The performances of both the models for Subtask3 are better compared to that of the other two subtasks. Further, there are no much diferences in the performances of the two models in all the subtasks. Eficient FSL techniques will be explored further to handle labeled data with few samples. [8] Y. Joo, I. Hwang, Profiling on social media : An ensemble learning model using various features notebook for pan at clef 2019, 2019. [9] L. Tunstall, N. Reimers, U. E. S. Jo, L. Bates, D. Korat, M. Wasserblat, O. Pereg, Eficient
Few-shot Learning without Prompts, in: arXiv preprint arXiv:2209.11055, 2022. [10] M. Chinea-Rios, I. Borrego-Obrador, M. Franco-Salvador, F. Rangel, P. Rosso, Profiling Cryptocurrency Influencers with Few shot Learning at PAN 2023, in: CLEF 2022 Labs and Workshops, Notebook Papers, 2023. [11] T. Mikolov, K. Chen, G. S. Corrado, J. Dean, Eficient estimation of word representations in vector space, in: International Conference on Learning Representations, 2013.