Since ancient times, humans have worshipped gods, feared witches, and been wary of the unknown. Today, beliefs in astrology, unlucky numbers, and government distrust persist. What sets believers apart from sceptics? Is it merely their beliefs, or is there a deeper distinction? Are these beliefs also manifested in digital behaviour? Can we use these digital behaviour traces to predict beliefs? In this work, we explore if music preferences, which could be extracted from digital behaviour traces on music streaming platforms, can predict an individual's inclination towards paranormal, pseudo-scientific, and conspiracy theories. This work follows two main research questions:
• RQ1: Is there a correlation between user music preferences and belief constructs? • RQ2: Can we successfully predict the belief constructs from music preferences of users and their respective song characteristics?
HCI SI 2024: Human-Computer Interaction Slovenia 2024, November 8th, 2024, Ljubljana, Slovenia Envelope 89242102@student.upr.si (E. Spirova); matej.golubovikj@famnit.upr.si (A. M. Golubovikj); marko.tkalcic@famnit.upr.si (M. Tkalčič) GLOBE https://markotkalcic.com/ (M. Tkalčič) Orcid 0009-0002-0378-400X (A. M. Golubovikj); 0000-0002-0831-5512 (M. Tkalčič)
Previous research has established a strong link between personality traits and preferences. Ferwerda et al. [1] demonstrated that personality, particularly the Big Five traits, can effectively predict music tastes across different age groups. Their findings revealed that musical preferences evolve over time, with individuals tending to narrow their variety of preferred genres. Motamedi et al. [2] investigated the potential for predicting personality traits through movie preferences, exploring eudaimonic and hedonic orientations in film viewers. Preniqi et al. [3] explored the relationship between music preferences and moral values, using both lyrical and audio analysis. They found that music can offer insights into an individual's moral framework. Our study differs from this body of literature as it includes a third dimension: unverified beliefs. In other words, our work will explore whether a transitive relationship exists: musical preferences, personality and beliefs. Personality and beliefs can be intricately connected (for example, Andersson et al. [4] finds a link between narcissism and belief in astrology), and analysis in this direction can deepen our understanding of musical preferences.
In the context of unverified beliefs, Drinkwater et al. [5] validated the Australian Sheep-Goat Scale (ASGS) [6] as a reliable measure of belief in the paranormal, a scale also employed in our study. Similarly, Tobacyk [7] introduced the Revised Paranormal Belief Scale (R-PBS), refining the original 1980 scale to enhance its accuracy in measuring paranormal beliefs. Both the ASGS and R-PBS are critical components of our work. Additionally, Lantian et al. [8] developed a brief yet effective scale for measuring conspiracy thinking, which we also use in our research. Lastly, in the domain of personality, Rammstedt and John [9] validated the shortened Big Five scale, emphasizing the benefits of reduced participant fatigue and increased engagement.
This body of work lays the foundation for our investigation into the triadic relationship between music preferences, personality traits, and beliefs, expanding the scope of existing research by integrating these elements into a unified model. To the best of our knowledge, no previous work has explored the relationship between musical preferences, personality and unverified beliefs in a single study
To our knowledge, there were no existing datasets that would be suitable for our research. We therefore performed a survey of 62 individuals (eligible after data cleaning) through a questionnaire. The questionnaire elicited the participants' (i) music preferences, (ii) personality and (iii) unverified beliefs (beliefs in paranormal phenomena, pseudoscience and conspiracy theories). We now present the contents of the survey in more detail.
Music preferences. We collected the participants' music preferences through song ratings. Participants were asked to rate 70 randomized songs on a 5-point scale. The songs were selected from a Spotify playlist with 350 songs, contributed to by 49 individuals 1 , each adding 5-10 songs that resonated with their character. Before inclusion in the survey, the songs were categorized using twelve common themes identified by Henard and Rossetti [10], based on Billboard hits from 1960 to 2009. To ensure balance, we excluded categories with fewer than five songs, leaving ten themes (detailed in Section 3.2). A Python script using GPT-3.5-turbo assigned these themes to the songs. The final songs were then randomized into five groups of seventy, resulting in five survey versions for maximum ratings. Each of these five survey versions was administered to a different set of participants.
Personality. To elicit the participants' personality, we used a shortened version of the Big 5 personality test [9]. The test consisted of 10 questions, on a 5-point (Likert) scale, gauging the participant's personality using the five-factor model [11]: (i) Agreeableness, (ii) Openness to (new) experience, (iii) Conscientiousness, (iv) Extraversion and (v) Neuroticism. We used a shortened version of the test to reduce participant fatigue during the survey.
Unverified beliefs. We used three tests to derive the participants' inclination towards unverified beliefs: (i) the Australian Sheep-Goat Scale (ASGS) [5], (ii) the Revised Paranormal Belief Scale [7] and (iii) the (Shortened) Belief in Conspiracy Theories Inventory [8]. The Australian Sheep-Goat Scale (ASGS) consists of 18 true/false questions giving a score from 0 to 36 on the participant's belief in pseudo-science and the paranormal [7]. The Revised Paranormal Belief Scale (RPBS) ascertains the level of the participant's belief, on a scale from 1 to 7, in (i) Traditional Religious Belief, (ii) Psi (belief in psychic powers, telepathy, clairvoyance, etc.), (iii) Witchcraft, (iv) Superstition, (v) Spiritualism, (vi) Extraordinary Life Forms and (vii) Precognition (belief in extrasensory perception, magical prediction of the near future). We excluded the questions for (i) Witchcraft, and (ii) Extraordinary Life Forms from the study (in total 19 questions were used from this test). Finally, the (Shortened) Belief in Conspiracy Theories Inventory (BCTI) measures the participant's inclination towards conspiracy thinking as a value from 1 to 7 using 5 questions on the same scale (each eliciting the degree of belief in a conspiratory statement, eg. the US government is spying on us).
In addition, the participants were asked to provide their age and gender (demographic information).
The survey was administered through the 1ka platform. It took 15-20 minutes to complete, with voluntary participation from individuals aged 18 and older. When the survey was concluded users with fewer than 10 song ratings were excluded, leaving us with 62 eligible responses.
The final dataset extracted from the survey contained (i) song ratings from each user of at least 10 songs, (ii) the participant's personality traits (Agreeableness, Openness to (new) experience, Conscientiousness, Extraversion and Neuroticism), and (iii) levels of inclination towards unsubstantiated beliefs (ASGS, Religious Belief, Psi, Superstition, Spiritualism, Precognition and Conspiracy Beliefs). We augmented this dataset by adding our 10 song categories (themes, discussed in Section 3.1), these were: Pain, Rebellion, Desire, Inspiration, Aspiration, Nostalgia, Breakup, Escapism, and Confusion. Each song was mapped to one category. Once mapped, the ratings of the songs in each category were aggregated and averaged for each user. This gave us an average rating for each category for each user.
To address RQ1 we performed correlational analysis on the derived data. Specifically, we analyzed the correlations between (i) the average rating of the participant per song category, (ii) the participant's personality scores and (iii) the participant's scores on our scales for unverified beliefs (ASGS, BCTI, and the 5 RPBS scores). We used the Pearson Correlation coefficient to derive correlations. The results are presented in Section 4.
To address RQ2, we employed three models (i) Linear Regression, (ii) Decision Trees and (iii) Matrix Factorization using Singular Value Decomposition (SVD) to predict beliefs based on music preferences and compared them to a Mean Baseline. For the purpose of this paper, we present our evaluation results for Singular Value Decomposition (SVD) our best-performing model 2 . A six-fold cross-validation was performed, and model performance was evaluated based on the Root Mean Squared Error (RMSE) of the predicted scores for each category of unverified beliefs (ASGS, Religious Belief, Psi, Superstition, Spiritualism, Precognition and Conspiracy Beliefs). The results are presented in Section 4.
We now present our correlational analysis between (i) preferences in music categories, (ii) Big 5 Personality traits and (iii) unverified belief constructs. The correlational coefficient presented here is the Pearson Coefficient which ranges between -1 and 1, where the closer a coefficient is to 1 the stronger the positive correlation is, while the opposite is true for coefficients leaning towards -1.
Figure 1a shows the correlation between (i) Big 5 Personality traits and (ii) preferences in music categories. The user's music preferences are operationalized as the average rating that the participant provided in each category of songs. We can see that most of these correlations are weak, even though previous work has shown that musical preferences and personality traits are related [1]. This might indicate an issue with the characteristics and/or size of the sample.
Figure 1b shows the correlation between (i) Big 5 Personality traits and (ii) unverified belief constructs. We can see some weak correlations between these constructs. For example, there are weak correlations between (i) Conspiracy belief and Neuroticism, (ii) Conspiracy belief and Conscientiousness and (iii) Superstition and Conscientiousness.
Figure 1c plots the correlations between (i) the average rating that the participant provided in each category of songs (delineated in Section 3) and (ii) the participant's unverified beliefs. Music categories are delineated in Section 3. The unverified belief constructs are: Consipracy theories (gcb), Australian Sheep-Goat (asgs), Religious Belief (trb), Psi (psi), Superstition (ss), Spiritualism (sp), Precognition (pc).
From Figure 1c we can see that there are some weak correlations. For example, (i) preference for the "pain" song category is positively correlated with the Australian Sheep-Goat Scale (asgs) and (ii) preference for songs in the "escapism" category is negatively correlated with superstitious beliefs (ss).
The majority of the correlations in Figure 1 are weak, even when a strong correlation was expected, such as between Big 5 Personality traits and music preferences (as shown in [1]). The results from our correlational analysis are inconclusive and point to the need of a more detailed analysis on a larger sample of participants.
The performance of our models for predicting unverified beliefs based on music preferences was evaluated using Root Mean Squared Error (RMSE). Table 1 summarizes the average RMSE values test sets across all folds and for all of the belief constructs. Models compared are, for each belief construct, (i) our best-performing model i.e. Singular Value Decomposition (SVD) and (ii) our mean baseline. The RMSE results indicate that the baseline model was consistent in
Average RMSE results per unverified belief construct for SVD and baseline models. The unverified belief construct are: Consipracy theories (GCB), Australian Sheep-Goat (ASGS), Religious Belief (TRB), Psi (PSI), Superstition (SS), Spiritualism (SP), Precognition (PC). All predicted belief construct quantifiers (GCB, TRB, PSI, SS and PC) are in the range of 1 to 7, except for the Australian Sheep-Goat (ASGS), which is in the range of 0 to 36. beating the SVD model in achieving lower RMSE values across folds. This suggests that the baseline model is more effective that the SVD model in capturing underlying patterns in the data. While the SVD model also performs reasonably well, the baseline model proves to be the better choice for this specific task. The baseline model also has a narrower error distribution, which implies that the ratings are closer to their actual values, whereas the wider distribution of the errors, which can be addressed with more advanced models.
What the findings in this study highlighted is the importance of selecting the correct model in predictive analysis. The dominance of the baseline model over the SVD model, underlines the need of a detailed evaluation of different modeling approaches when dealing with complex data. This insight can guide future research. Another significant point is the enhancement of the SVD model in future research. For one, gathering a more diverse and extensive dataset can greatly improve the performance of the SVD model. Additionally, some fine-tuning the hyperparameters of the SVD model, might also enhance its performance. Given more time, conducting more extensive experiments could lead to different results, as well.
Several limitations of this study can be addressed in future work: (i) Data Limitation: Lack of sufficient and diverse data is a limit for the models performance. Future studies should aim to collect more comperhensive datasets; (ii) More Exploration: A limited number of models have been explored in this work. Future research should consider introducing a more broader spectrum of models and algorithms to find the most effective approach; (iii) Computational Resources: Access to more powerful computing resources could potentially enable the exploration of more complex models.
In conclusion, this study investigates the potential for predicting unverified beliefs based on music preferences. While the results indicate some associations, the findings did not conclusively demonstrate a predictive relationship. Several limitations, including insufficient sample size and a limited exploration of predictive models, likely impacted the study's outcomes. These limitations underscore the need for more comprehensive data and the application of more sophisticated models in future research. Despite these challenges, this work serves as a foundation for further investigation, offering valuable insights that can guide future efforts to better understand and enhance the predictive relationship between digital behaviour and belief systems.