We like to see who is stronger, richer, better, more clever. Since we humans (1) love lists; (2), are competitive, and (3) are jealous of other people, we like ranking. Students ranked in ascending order based on their heights in a gym reflects objectivity. However, many “Top Ten” (and other) lists are based on subjective categorization and give only the illusion of objectivity. We don't always want to be seen objectively, since we don't mind to have a better image or rank than we deserve. While making objective rankings sounds like an appealing goal, there are at least two different reasons why we may not have objectivity: ignorance and manipulation. Persons with less knowledge suffer from illusory superiority due to their cognitive bias, and this phenomenon is called the ”DunningKruger effect.” Omnipresent in society is not only ignorance but also manipulation. Manipulators have the intention of gaining personal advantage by adopting different tricks. Computer scientists design ranking algorithms, and computers can now process huge datasets with these algorithms. As we have seen, we are not always happy with the results, so we might ask whether, when, and how the results of a ranking algorithm should be controlled by content curators. Recent public debates about the use and misuse of data reinforce the message: we need a combination of human and computational intelligence [1]. The lecture is based on the book: Pe´ter E´rdi: RANKING. The Unwritten Rules of the Social Game We All Play. Oxford University Press (in production), see aboutranking.com We humans are constantly evaluating ourselves and others across a variety of features, such as financial status, intelligence, attractiveness, success, etc. Social comparison theory [2] divides these evaluations into upward social comparison, which occurs when we compare ourselves with someone judged to be better than ourselves (e.g., by having more wealth or material goods, higher social standing, greater physical attractiveness); and downward social comparison, which occurs when we compare ourselves with someone judged to be not as good as we believe ourselves to be. Comparing ourselves to others is an elementary human activity, and we cannot avoid making comparisons and being compared. There is a trade-off: favorable comparisons make us happier (at least in the short term), but unfavorable ones
drive us to make things harder. Systematic comparison among many pairs of
elements generates a ranked list. Rating is, in principle, simpler—a score (generally,
but not necessarily, a number) is assigned to the object or subject being rated,
independently of the scores assigned to other objects or subjects. Teachers know
well that it is simply impossible to always be objective: there is some interaction
among the grades of individual students. Ordered lists are based on the rankings
of elements. We simply love to read and prepare lists, since they condense and
organize information. Like it or not, each day we read a good number of ranked
lists, many times in the form of a listicle, a style that bloggers and journalists
recently adopted to convey information via the ranking procedure.
Social psychologists have shown [
Brain imaging methods have helped identify the regions and neural mechanisms
responsible for upward and downward comparison [
We need a population of items in order to make a ranking based on pairwise comparisons. We should be able to make clear statements for any two items A and B, such as item A is ”ranked higher than,” ”ranked lower than,” or ”ranked equal to” item B. Continuing this procedure with every possible pair, a ranked list is formed. People, goods, and products have multiple features, so they can be ranked by multiple criteria. Often, different criteria are in conflict with one another: for example, price (or cost) and quality are often in conflict. We cannot expect to buy a cheaper and more comfortable car. Multiple-criteria decisionmaking thus encompasses mathematical techniques to help create ordered rankings of possible choices when many factors need to be taken into account. For example, if a student is going to college, the decision-makers (she and her parents) have to rank the alternatives (colleges). Candidate colleges can be ranked by multiple criteria (tuition, academic status, distance from home, qualities of facilities, etc.) Finally, to prepare a ranking we need an algorithm. The trick is that in order for an algorithm to work, the individual criteria should be characterized by a specific number, a weight, which specifies the relative importance of a criterion. Weights are subjectively determined. We live in a world where decision-making is a combination of subjective and objective factors. 1.3
Rating assigns a score, generally a number, to each item. The chess player’s Elo rating, for example, is a generally accepted system for rating and ranking chess players. Each player’s strength is characterized by a number. This number is subject to change after each game—if you win against a higher-rated player it matters more than winning against a lower-rated player. 1.4
The human brain generally does not have the ability to remember long lists of unstructured items. We aren’t very good at remembering a series of numbers, of nonsense words, or of goods to purchase in the supermarket. One of the pioneers of memory research, Hermann Ebbinghaus (1850 1909), made memory studies around 1885 on himself and tried to memorize nonsense syllables. Time and again, he tested his memory and realized that the quality of his memories decayed exponentially, and he theorized that the performance of his memory was quantitatively characterized by what is called the ”forgetting curve.” He also found that his performance depended on the number of items, and it was more difficult to memorize long lists of items as opposed to short lists.
There are big exceptions to these generalities. Some people are able to
remember lists of nonsense items for literally decades. Alexander Luria (1902 1977),
a Soviet neuropsychologist, studied a journalist named Solomon Shereshevski
(1886 1958), who apparently had a basically infinite memory. He was able to
memorize long lists, mathematical formulae, speeches, and poems, even in
foreign languages, and recall these lists 14 years later as well as he had on the day he
learned them. His performance did not depend on the length of the items,
deviating from the theory suggested by Ebbinghaus’ observations. Shereshevski was
diagnosed with synaesthesia, which is a neurological condition, in which different
senses are coupled. When he realized his ability, he performed as a mnemonist.
Despite the allure of having a perfect memory, his abilities also created disorders
in his everyday life, as it was difficult to him to discriminate between events that
happened minutes or years ago [
Hierarchy is the very general organizational principle that characterizes our
physical, biological and social systems [
Linear dominance hierarchies proved to be very efficient for community resource management among a wide variety of social animals, from insects to fish and from birds to primates. Since more and more data has been accumulated, it has become possible to test hypotheses in contemporary animal behavior studies about the mechanisms behind the formation of evolutionary hierarchies. There are two distinct mechanisms for navigating the social ladder: dominance and prestige. Dominance is an evolutionarily more ancient strategy and is based on the ability to intimidate other members in the group by physical size and strength. In dominance hierarchies, group members don’t accept social rank freely, only by coercion. Members of a colony fight, and the winners of these fights will be accepted as “superiors,” and the losers as “subordinates.” The hierarchy formed naturally serves as a way of preventing superfluous fighting and injuries within a colony.
Prestige, as a strategy, is evolutionarily younger, and is based on skills and knowledge as appraised by the community.
Modern neuroscience has combined brain imaging devices and computational
techniques to uncover some mechanisms related to how our brains process
information on social hierarchy [
3.1
Neoclassical economic theories are based on the concept that we are rational in
the sense that during decision-making, humans are concerned with maximizing
our expected gain (say, pleasure or profit), which can be expressed by a utility
function. If we want to undertake a quantitative analysis, say one that maximizes
the utility function for our dessert selection, we should be able to assign numerical
values to our desires to consume pie, cheesecake, or mousse. The development
of rational choice theory in social sciences [
Social choice theory provides a general theory of the aggregation of individual opinions into a single collective decision. Nicolas de Caritat and the legendary economist Kenneth Arrow were two major contributors to the field. Nicolas de Caritat (1743 1794), often known as the Marquis de Condorcet, pioneered a particular voting system, called pairwise majority voting, that has remained influential even in contemporary voting studies and systems. Condorcet analyzed the behavior of juries and developed his celebrated jury theorem from these studies. As always, when mathematical models are used to describe social phenomena, we should carefully discuss the assumptions underlying these models. In this case, assuming that each member of a jury has an equal and independent chance (which is better than random (i.e. greater than fifty percent) but worse than perfect (less than 100 percent)) of making the correct conclusion, the jury theorem holds that increasing the number of members of the jury increases the probability of the group as a whole making the correct decision. Importantly, the relevance of the jury theorem is restricted to situations in which there really is a correct decision. It works, for example, when the members of a jury should decide whether or not a defendant is guilty. Consequently, under certain conditions, majority rules is appropriate at ”tracking the truth.” Of course, in real life the opinions of the voters are not independent of one another. In addition, the theorem cannot be applied to situations in which there is no “objective truth,” but only individual preferences. This is the situation we encounter when we must choose among political candidates. Kenneth Arrow (1921 2017) published his famous impossibility theorem in 1950 (for which he received a Nobel prize in 1972), which showed that when voters rank candidates, some failures may occur. Arrow’s studies and the subsequent work of scores of economists and mathematicians have generated debates and comparative mathematical analyses about voting systems.
From the perspective of cognitive architectures, distributed cognition seem to be
relevant to implement social choices [
Condorcet realized that even when individual preferences are ”rational” (i.e., transitive), the resulting collective decision might be “irrational” (i.e., intransitive). The game ”rock-paper-scissors” is an interesting game since the mathematical feature called transitivity is violated. The violation of transitivity leads to a cycle, in which we are not in a position to generate a ranked list.
Cycling ranking occurs in the legal systems, and in the book there are examples from the Talmud to the modern times. My understanding is that the US governmental system was intentionally constructed to violate transitivity, since the goal was to avoid any ordered ranking among the three branches. 4
There are at least two different reasons why we may not have objectivity in a ranking procedure. In principle, ranking agents should be objective, but, more often than not, they are ignorant or manipulative. The ignorant lacks the knowledge of some facts or objects or the skills to do something. However, they (it is never we who) are not necessarily uninformed; rather, they are misinformed. Manipulators change, control, or influence something (or someone) cleverly, skilfully, and generally for their own advantage. The actions of the ignorant and the manipulative construct a deviation from “true ranking,” and they give the illusion of reality while producing artificial changes in reality.
The non-monotonic relationship between self-confidence and expertise is called
the Dunning-Kruger effect [
There are a number of manipulation techniques used motivate people to take a specific action or support specific decisions: – Appeal to fear – Black-or-white fallacy – Selective truth – Repetition – Appeal to authority Manipulation of digital reputation has become a big industry and emerged with the goal of making websites more visible. There are search engine optimization (SEO) companies that perform this task. Even Reputation Management Companies are subject to ranking. As in Western movies, there are characters with white hats and with black hats. There are heroes and villains. Some SEOs, referred to as ethical hackers wear the white hat, but others manipulate information and wear a black hat.
Black-hat optimizers attempt to game search engine algorithms. As always, in democratic societies, first the community promulgates rules. Then, some people try to evade these rules. We cannot do anything but attempt to identify and neutralize the effects of these troublemakers. Here is a warning you may find useful: a black-hat optimizer can take you to the top of a website ranking in a very short period of time. But strictly speaking, it is totally illegal. If you don’t want to get penalized and kill your Google ranking forever, it is strongly recommended that you avoid black-hat optimizers. 5 5.1
Recommendation systems are ubiquitous in our lives. It is difficult to make any purchase without being somehow influenced by large electronic commerce (ecommerce) systems. Recommendation systems are key elements of any e-commerce system. Nobody can force us to use them: we do if we trust them. While any such system can be gamed, fake reviews and other tricks can be filtered, and recommendation systems can help us make better choices.
Modern recommendation systems [
The process of measurement has a major role in any civilization. According to the optimistic perspective of positivism, measurement is the first step in making improvements. The social demand for accountability and transparency has made quantitative metrics a major tool for characterizing the performances of social institutions.
Computer scientists design ranking algorithms, and of course, computers can now process huge datasets with these algorithms. We also know that models and algorithms are based on human’s assumptions. Probably we don’t have better options but to trust in the power of the combination of human and computational intelligence.
Campbell’s law, however, states: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” It is a warning signal, that metrics can be (and often are) gamed. Still, we should not abandon algorithms in favor of our previous subjective and verbal evaluations. Instead, social scientists and computer scientists should cooperate to generate “ethical algorithms.” 6
We are in the process of understanding the cognitive architectures behind individual and institutional decision-making, in general, and related to ranking, in particular. In the age of the date deluge, public debates about the use and misuse of data and algorithms imply a message: we need to develop methods for combining human and computational intelligence. 7
The author thanks the Henry R. Luce Foundation for letting him serve as a Henry R. Luce Professor. He also thanks Natalie Thompson for useful discussions. Comments from the participants of AIC 2019 were very useful.