We report our models for detecting age, language variety, and gender from social media data in the context of the Arabic author pro ling and deception detection shared task (APDA) [32]. We build simple models based on pre-trained bidirectional encoders from transformers (BERT). We rst ne-tune the pre-trained BERT model on each of the three datasets with shared task released data. Then we augment shared task data with in-house data for gender and dialect, showing the utility of augmenting training data. Our best models on the shared task test data are acquired with a majority voting of various BERT models trained under di erent data conditions. We acquire 54.72% accuracy for age, 93.75% for dialect, 81.67% for gender, and 40.97% joint accuracy across the three tasks.1
The proliferation of social media has made it possible to collect user data in
unprecedented ways. These data can come in the form of usage and behavior
(e.g., who likes what on Facebook), network (e.g., who follows a given user
on Instagram), and content (e.g., what people post to Twitter). Availability
of such data have made it possible to make discoveries about individuals and
communities, mobilizing social and psychological research and employing natural
language processing methods. In this work, we focus on predicting social media
user age, dialect, and gender based on posted language. More speci cally, we use
the total of 100 tweets from each manually-labeled user to predict each of these
attributes. Our dataset comes from the Arabic author pro ling and deception
detection shared task (APDA) [
In the rest of the paper, we introduce the dataset, followed by our experimental conditions and results. We then provide a literature review and conclude.
For the purpose of our experiments, we use data released by the APDA shared task organizers. The dataset is divided into train and test by organizers. The TRAIN set is distributed with labels for the three tasks of age, dialect, and gender. Following the standard shared tasks set up, the test set is distributed without labels and participants were expected to submit their predictions on test. The shared task predictions are expected by organizers at the level of users. The distribution has 100 tweets for each user, and so each tweet is distributed with a corresponding user id. As such, in total, the distributed training data has 2,250 users, contributing a total of 225,000 tweets. The o cial task test set contains 720,00 tweets posted by 720 users. For our experiments, we split the training data released by organizers into 90% TRAIN set (202,500 tweets from 2,025 users) and 10% DEV set (22,500 tweets from 225 users). The age task labels come from the tagset funder-25, between-25 and 34, above-35 g. For dialects, the data are labeled with 15 classes, from the set fAlgeria, Egypt, Iraq, Kuwait, Lebanon-Syria, Lybia, Morocco, Oman, Palestine-Jordan, Qatar, Saudi Arabia, Sudan, Tunisia, UAE, Yemeng. The gender task involves binary labels from the set fmale, femaleg. 3
As explained earlier, the shared task is set up at the user level where the age, dialect, and gender of each user are the required predictions. In our experiments, we rst model the task at the tweet level and then port these predictions at the user level. For our core modelling, we ne-tune BERT on the shared task data. We also introduce an additional in-house dataset labeled with dialect and gender tags to the task as we will explain below. As a baseline, we use a small gated recurrent units (GRU) model. We now introduce our tweet-level models. 3.1
Tweet-Level Models
Baseline GRU. Our baseline is a GRU network for each of the three tasks.
We use the same network architecture across the 3 tasks. For each network,
the network contains a layer unidirectional GRU, with 500 units and an output
linear layer. The network is trained end-to-end. Our input embedding layer is
initialized with a standard normal distribution, with = 0, and = 1, i.e.,
W N (0; 1). We use a maximum sequence length of 50 tokens, and choose
an arbitrary vocabulary size of 100,000 types, where we use the 100,000 most
frequent words in TRAIN. To avoid over- tting, we use dropout [
Third submission. Finally, for our third submission, we use a majority
vote of (1) rst submission, (2) second submission, and (3) predictions from our
user-level BERT model. These majority class models (i.e., our third submission)
achieve best results on the o cial test data. We acquire 54.72% accuracy for
age, 81.67% accuracy for gender, 93.75% accuracy for dialect, and 40.97% joint
accuracy.
Arabic. Arabic is a term that refers to a collection of languages, varieties, and
dialects. The standard variety, Modern Standard Arabic (MSA), is the one
usually used in formal communication and educational settings. Arabic also has a
wide range of under-studied varieties and dialects that classically used to be
categorized in a coarse-grained fashion (e.g., Levantine, North African) [
Social Media Author Pro ling. Author pro ling is the term usually used
to refer to detecting a host of attributes of (often social media) users. This
include identifying attributes such as age, gender, educational level, economic
class, stance or ideology [
Age and Gender. A number of studies have been conducted on
Englishbased age and gender detection, including [
In this work, we described our submitted models to the Arabic author pro ling
and deception detection shared task (APDA) [
We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), the Social Sciences Research Council of Canada (SSHRC), and Compute Canada (www.computecanada.ca).