For most social media sites, how to find out (influential) users' professions is an important task. Much work has been conducted to explore this task through mining user-generated textual content or analyzing the social network structure. In this paper, we innovatively solve this task by only examining which microblog messages an influential user has forwarded. First, we define hot microblog messages under two standards and identify them from a large number of candidate messages. Each of the identified messages points to a specific hot event. Next, we group similar hot messages together based on their word similarity, semantic similarity, and forwarders' similarity. Last, we represent users with the hot messages they forwarded and design an identification method to identify their professions. Moreover, we collect a real-world dataset to conduct experiments and prove that our method performs significantly better than the traditional method.
Online microblogging services have become an integral part of the daily life for most
Netizens. These services expect to know more about their users’ profiles, since user
profile plays an important role in commercial services, such as personalized
recommendation and online advertising. However, user profile is usually not easily obtained,
because users are reluctant to expose their profiles to the public. Fortunately, some
work has been conducted to solve this problem. A traditional practice is cutting
users’ messages into bags of words and training a classifier. This practice can achieve an
acceptable result on simple tasks such as predicting gender and age [
Profession, which is founded upon specialized educational training, is a critical social profile of influential users. In Weibo, the largest microblogging service in China, influential users are mainly organized by their professions. They are more likely to follow other users that have the same profession with them. It is important to correctly identify influential users and their professions for microblogging services.
Message forwarding (e.g. retweeting on Twitter.com and reposting on Weibo.com) is one of the most popular functions in the existing microblogging services. In Weibo, users can forward messages or any interesting content on the web, such as real blogs, Copyright c 2017 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. photos and external links. In this paper, if a weibo message was forwarded by any user, we define it as forwarded message, otherwise we define it as non-forwarded message. Based on a large dataset, we find that about 60% of weibo messages are forwarded messages. For most users, the messages they forwarded are exactly what they are interested in. Users’ professions can be reflected by the messages they forwarded to some extent. But the traditional “bag of words” model will completely undermine the information contained in users’ forwarding behaviors. Naturally, in this paper, we ask and try to answer the following question: can we represent microblog users with the messages they forwarded, and predict their professions more accurately than the traditional method?
The task confronts some challenges which make it non-trivial. The first challenge is that there exist too many forwarded messages. If we consider each forwarded message as a feature, the feature vector will be very large and sparse. We observe that most of these messages only have been forwarded by no more than 3 weibo users. In this paper, we define them as non-hot forwarded messages and define other messages that are forwarded by more users as hot forwarded messages. In our experiment, we discard the non-hot messages. Another challenge is that even though we can filter out non-hot messages, the number of remaining hot messages is still quite large. We observe that, every hot message points to a hot event (e.g. a breaking news or a recently released movie). We should come up with some methods to group similar hot weibo messages together.
In this paper, we propose an efficient framework of Profession Identification by using Forwarding Behaviors (PIFB). As Figure 1 shows, first, we identify the hot forwarded messages from a large number of candidates. Each of these identified messages points to a specific hot event. Next, we introduce three methods to group similar messages together, downsizing our message sets. Then, influential users can be represented with the merged hot messages that they have forwarded. Finally, we predict users’ professions, and the results are more accurate than those in the traditional method. 2
We collect 41,531 manually annotated influential users from Weibo (http://weibo.com). To avoid robot users, we only collected verified users. Weibo conducts manual verifications to make sure that the verified users provide real and authentic information. These users belong to 11 representative professions. As Table 1 shows, the professions include “media”, “entertainment”, “sports”, and “IT”, etc.
We also collect users’ latest 500 weibo messages. These messages can be classified into two categories: forward action and post action. In general, forward action consists of trace and content. Trace contains the information that through which users the current user can see the final messages. Content can be extended to any forms as long as it can be shared by users with their followers, such as videos and blogs. A simple example is shown below: if a user froward the message:
RT @Raj RT @Sheldon : It took 50 years ::: | tr{azce } | con{tzent }
This forward action indicates that “It took 50 years ...” was originally posted by “Sheldon” and was forwarded by “Raj”, and now is forwarded by the current user again.
In general, post action only contains the “content” part, representing that the current user posted an original message. 3
In this section, we formalize our problem as a classification task and introduce the main steps of PIFB. 3.1
This paper focus on influential user’s behaviors about the forwarded messages. A critical step is to identify the hot forwarded messages. In this part, we define hot messages under two standards.
Absolutely Hot Message We argue that if a message has been forwarded by more users, the information behind it will be more. And the forwarding behaviors about this message can help our profession prediction more. Nowadays, Weibo has become the the biggest “News Site” in China. Most traditional news organizations open their official accounts in Weibo and these accounts are all very active. They usually publish the breaking news timely and make the news spread quickly. There also exist many Chinese celebrities in Weibo, including actors, singers and entrepreneurs, etc. They post their personal views or daily lives in their accounts. They generally have a great number of followers and their daily updates are likely to get thousands of forwards. So, in this paper, if a weibo message has been forwarded by more than a certain times (for example, 500), it will be regarded as the first kind of “hot forwarded message” (absolutely hot). Relatively Hot Message The 11 professions, showed in Table 1, are not “evenly matched” on attracting attentions. Nearly all the high forwarded messages are all posted by “entertainment” and “sports” stars. For an “estate” account, it is not easy to post an absolutely hot message, because “estate” accounts usually have relatively less followers and lower forwarding rate. If we only adopt the absolutely hot messages as described in the previous paragraph, it is very possible that we only get the messages posted by a small subset of that 11 categories (may be 2-4). Therefore, as a supplement to the first standard, we define another kind of hot message. In our dataset, if a message’s owner has f followers (f >500) and this message has been forwarded by more than f =5 times, it will be regarded as the second kind of “hot forwarded message” (relatively hot).
After identifying all these two types of “hot messages”, we can build a matrix M , whose columns denote hot messages and rows denote users. This matrix represents all the forwarding relationships between weibo users and hot messages. M will have too much columns, if we don’t filter out the non-hot messages. Even though we do only consider the hot messages, the number of column is also very big. To slim down M , we propose three methods to group similar messages together in the next. 3.2
In most microblogging services, users can be divided into two categories: information producer and information consumer. The information producer mainly includes the news site accounts, self-media accounts, and profit-seeking accounts with legions of followers. Their main purpose is making their microblogs broadcast as widely as possible to expand their influence and get more new followers. Whenever there is a news, producers will timely post their relevant microblogs. The producers are very likely to post similar contents, because the texts may be pasted from the same source. The information consumer mainly refers to normal weibo users. More than 90% weibo users can be classified into this category. Their most important action is reading and forwarding messages. Normally, hot messages are more likely to attract them.
If the hot messages only contain a video link or a web link, it is easy to determine whether they are similar. But if they contain some text contents, the task will be more difficult. In the next, we introduce three methods to solve it.
Simhash As described above, the information producers are likely to post similar
weibo messages. The most direct idea is that merging similar hot messages based on their
word similarity. Simhash [
In this method, we firstly calculate the simhash values of all the hot messages. After that, we can group the similar messages together, if the hamming distance of their simhash fingerprints is less than or equal to 3.
Paragraph Vector The simhash can only calculate the documents’ similarity based on
their word similarity. It can not deal with situation that, two documents have the similar
semantics but written with different words. [
We firstly calculate hot messages’ representative vectors by using the “Paragraph Vector” method. The length of vector is set to 400 according to the original paper. After that, we calculate their distances. A pair of hot messages can be grouped together if their distance is smaller than a threshold.
User-Weibo Matrix Factorization The first method is based on message’s word
similarity and the second is based on the semantic similarity. They are both directly
calculated by the weibo contents. As described in section 3.1, we have generated the user-weibo
relationship matrix M . So we can further find more similar messages based on which
users have forwarded these messages. Hofmann [
k P (ujd; ) = X p(ujt)p(tjd) t=1
(1)
Based on this model, we can transform the user-weibo matrix into two new matrices. The first is user-topic matrix, which represents each user with a vector of k topics. The second is document-topic matrix, which represents each document with a vector of k topics too. In the second matrix, if the documents contain similar topics, their vectors are more likely similar. We can group two similar hot messages together, if the distance between their vectors is under a threshold. In this paper, we empirically set k to 400 and name this method UWMF. 3.3
After merging similar hot messages, users can be represented as more compact vectors. Each element of these vectors represents a merged hot message, and the elements will be used as features in our multi-class classifier.
Over the last several decades, many kinds of discriminant classifier have been
created. In our experiment, we compare Logistic Regression (LR) and Gradient Boosted
Decision Tree (GBDT). We choose GBDT as our default multi-class classifier, because
we find that GBDT performs better in most instances. Hence, in the following part we
only show the results obtained with GBDT [
In this section, we first statistically study our dataset. After that, we identify the hot weibo messages and merge the similar ones. At last, we compare our methods with the baseline method comprehensively. 4.1
We firstly count influential user’s forwarding rates on different professions. As Figure 2(a) shows, different professions have different forwarding rates on average. It is a little surprise that the “estate” and “government” accounts forwarded more messages compared with the “finance” accounts. Overall, the difference between different professions is not significant. In our dataset, about 58% of weibos are all forwarded messages. For about 66% users, more than half of their messages are forwarded messages. Figure 2(b) shows the distribution of how many messages users forwarded (in their latest 500 messages) in our dataset. We find that about 95% users forwarded more than 50 messages. In this paper, our goal is to predict users professions only based on their forwarding behaviors, so we discard other 5% users who forwarded no more than 50 messages in our experiment.
As described in section 3.1, we define the absolutely hot message and the relatively hot message separately. To better understand these two types, we calculate how many times that users’ latest 500 weibo messages have been forwarded on average by category. As Figure 2(c) shows, these numbers of different categories are very unbalanced. The “entertainment” and “literature” accounts attract much more forwarding behaviors than “estate” accounts. The main reason is that the “entertainment” and “literature” accounts have relatively more followers. If we only adopt absolutely hot messages (for example, the threshold is set to 500), it is possible that we can not get any hot messages posted by “estate”. So identifying relatively hot messages is very necessary in our model.
Users forward how many messages in their latest 500 messages
Users forward how many messages
in their latest 500 messages 500 sd400 r a row300 ff o re200 b m uN100 3500 3000 2500 tn2000 u oC1500 1000 500
00 0 Me En Es FiPGroofeIsTsioSpnsFa Ed li Ga 00 F1o0r0ward h2o00w man3y00messa4g00es 500 (a) Users forward how many messages (b) The distribution of user’s forwarding behavior 7 x 104
How many times users’ latest 500 messages were forwarded
The length distribution of forwarded messages 0 Me En Es FiPGroofeIsTsioSpnsFa Ed li Ga 5L0engt1h00of for1w50arde2d00Weib2o50 300 (c) Number of users’ 500 messages (d) Length distribution of forwarded were forwarded messages
Weibo limits message length to 140 Chinese characters or 280 English characters. Figure 2(d) shows the length distribution of hot messages in our dataset. We can find that there exist two peaks. The first peak represents the hot messages that only contain 10-20 characters. These messages are likely to be posted by star users who have millions of fans. This kind of message usually additional contains a picture or a video link. The second peak represents the messages that contain 140 Chinese characters. This kind of message generally contains rich semantics. 4.2
As described in section 3.1, if a message has been forwarded by more than a certain number of times, it will be considered as an absolutely hot message. It is apparent that how to set the threshold is a double-edged sword. If we set the threshold to a smaller value (more hot messages), on one hand, user can be represented with more messages and our model’s expression ability will be increased; on the other hand, our model should handle more features and need to take the risk of over-fitting. As Table 2 shows, we set the threshold to 500, 2,000, and 10,000 separately. When the threshold is set to 500, we can get 731,153 hot weibo messages. This number is too large and most of these messages have been forwarded by no more than 5 users in our dataset (40 thousand users). Then, we filter out such messages from our hot message sets, leaving 100,219 valid messages. In the prediction tasks, we compare the performance of these three thresholds and choose 500 as the default value.
As section 3.1 shows, if a message’s owner has f followers (f >500) and this message has been forwarded by more than f =5 times, we regard this weibo message as a relatively hot message. Just as the absolutely hot messages, we also filter out the messages that have been forwarded by no more than 5 users in our dataset, and get 61,806 relatively hot messages.
Eventually, we collect 162,025 hot messages in total (100,219 absolutely hot & 61,806 relatively hot). 4.3
In this part, we evaluate the performance of our three methods on clustering similar hot
messages. As Table 3 shows: (1) In the simhash method, we choose 64 as the default
length of hash value. In this step, we group similar messages together, if their hamming
distance is less than or equal to 3. We can merge our 162,025 hot messages, identified
from section 4.2, into 57,624 hot events. (2) In the second method, we choose 400 as
the default size of paragraph vector, and merge similar messages according to their
Euclidean distances. In this step, we can merge the 162,025 hot messages into 32,118
hot events. (3) In the third method, we also choose 400 as the size of hidden variables,
and adopt Euclidean distance to measure their similarities. In this step, we can merge
the 162,025 hot messages into 27,129 hot events. In our experiment, the lengths of
these three vectors (64, 400, 400) are chosen empirically [
In practice, we serially combine all these three methods. At first, we adopt the simhash to find similar hot messages, making users’ representative vectors more compact. On the basis of this results, we adopt the second method, further compressing users’ vectors. At last, we perform the third method based on the current results. After
1 Simhash 162,025 57,624 2 P2V 162,025 32,118 3 UWMF 162,025 27,129 4 Simhash+P2V+UWMF 162,025 17,196 these three steps, our 162,025 hot messages can cluster together into 17,196 hot events. In the next, we will study whether these optimizations can improve our profession identification tasks. 4.4
We randomly divide our 40 thousand labeled users into training set (60%), validation set (20%), and test set (20%). We regard user’s labeled profession as the gold standard, and select accuracy, macro-averaging precision/recall/F-Measure as evaluation metrics.
To verify the validity of our method, we build a baseline model. The feature candidates of baseline model include: (1) Words in user’s original messages; (2) Words in user’s forwarded messages; (3) Mentioned user ids in messages; (4) URLs in messages; (5) Hash tags in messages. There exist hundreds of thousands of feature candidates and we have to perform feature selection to downsize our feature sets. Following the valid experience in feature selection for text classification, we use 2 statistic to select representative features. We evaluate performance with different numbers of features, and select 9200 feature candidates. We compare LR and GBDT on these features and find they have similar performance. To be consistent with our model, we also choose GBDT as the default baseline classifier.
From Table 4, we can observe the evaluation results. We find that the baseline model achieves a performance of 62.38% in accuracy and our three models all get better results than it. This comparison proves user’s forward behavior is effective in profession identification. As Table 4 shows, along with the implementation of three merging strategies, our three models can make the prediction gradually improved. Our model in the fourth line that serially adopts all three merging strategies achieves the best result (accuracy=73.98, F1=73.87). This result indicates that effective clustering of similar messages is necessary, for there exist too many forwarded messages.
To better understand the prediction errors, we present the details of the best result. In Table 5, the value of ith row and jth column represents the ratio of the users in profession i being identified as profession j.
To make the data more intuitive, we illustrate the ratio in each entry using different shades of color. We can observe that: (1) Our model performs differently on different professions. The recall scores (value on the diagonal) of most professions are bigger than 70%, with only “fashion” and “literature” less than 65%. The main reason is that the forward behavior of these two professions has no special characteristics. (2) The “media” accounts occupy about a quarter of our user collections. Our model tends to predict the uncertain user as “media” account, making the precision score of “media” relatively lower (51.3%). (3) The behaviors of some professions are quite similar. For example, the “entertainment” user and “fashion” user have the similar interests, they usually follow and interact with each other. It makes the boundary between these two professions not very clear for identification. 5
User’s attributes can be inferred from user-generated text data and social network
structure. [
Recently, there are some researches on identify users’ professions. [
In this paper, we present an efficient framework PIFB to predict influential users’ professions by only examining which microblogs they have forwarded. In the first step, we identify the hot weibo messages from a large number of candidate messages, and represent users with the hot messages they forwarded. After that, we group hot messages together if they talk about the similar topics. This step can make users’ representative vectors more compact. At last, we design a multi-class classifiler to predict their professions. The experiments on a real-world dataset demonstrate the effectiveness of PIFB. Our method performs significantly better than the traditional “bag of words” based method.
The authors would like to thank the anonymous reviewers for their comments. This work was supported by the National Natural Science Foundation of China under Grant No.61572044. The contact author is Zhen Xiao.