A recurring theme in the science- ction series Black Mirror is the consequence for society of an over-focus on social networking. The episode Nosedive imagines a future in which every public interaction a person has is rated by the other parties, and every aspect of ones life depends on the overall rating computed from these. In this paper, we show how such a scenario is already technically possible using existing technologies such as distributed ledgers, and discuss means by which the negative possibilities may be ameliorated using semantic approaches.
The television drama Black Mirror [
The episode Nosedive [
We rst show that the scenario of Nosedive can already be implemented using technologies available now, before discussing possible technical approaches which could minimise the negative social consequences of a hypothetical ubiquitous adoption of such a rating system. 2
An emerging area of research in the Web concerns the use of distributed ledgers,
based on the blockchain data structure, to serve as immutable trustworthy record
stores in environments where trust is lacking. Originally developed to underpin
the Bitcoin cryptocurrency [
A distributed ledger is based on a blockchain: a timestamped sequential series of records shared in toto across a network of nodes. There is no central or authoritative node, and every node has the potential to add new records to the ledger. New records are added by consensus: anyone who wishes to add data to a node submits a cryptographically-signed request to do so. Each node competes for the right to add (\mine") a new block of transactions to the blockchain, with that right awarded by consensus among all the nodes. The precise method of consensus varies between di erent types of blockchain. The important feature of any consensus mechanism is that there should be a cost involved in attempting to mine blocks, and a reward for the successful node, to encourage good behaviour among nodes. The winning node selects a set of pending record transactions and groups them into a block, which is added to the chain. Requiring a consensus between a large network of ( nancially, and organisationally) unrelated nodes in order to add any data to the chain guarantees that spurious data cannot be inserted by a malicious agent, and, due to the fact that modifying an older record requires the entire chain following the modi ed record be rewritten as if from scratch, the records entered in a blockchain are e ectively immutable.
Distributed ledger platforms such as Ethereum [
Records on a distributed ledger are public, and, as stated earlier, all nodes on the network have a complete copy of the entire chain. This does not mean, however, that everything must be readable by everyone; records can contain encrypted data, and, with a suitable cryptographic key management infrastructure, the data owner can implement a \selective visibility" system, only permitting chosen others to view the full or partial contents of a record. 3
We see two main technical requirements to implement the Nosedive rating system. To simplify the discussion, let us say that person A wishes to give person B a rating of n stars, where n is a natural number between 0 and 5 in relation to a face-to-face interaction I between A and B. Person C, at a later date, wants to view this rating's value. 1. the ability for A's mobile device to recognise B's proximity and provide A with the ability to select B's name and apply the rating. 2. Ratings must be unforgeable and incapable of being tampered with, and always attributable to both A and B in their respective roles in the rating transaction.
In the episode, A simply needs to hold her device up facing B rather than making a selection on the screen to achieve 1; potentially we can imagine advanced facial recognition in addition to the other techniques described here enabling this. And 2 is never explicitly stated; however, we must assume that it applies in order for the system to carry the weight in people's lives that it is shown to do.
To implement 1, it is possible to take advantage of the widespread availability of accurate GPS devices in modern smartphones. If each phone registered its current location in a public database, it would be possible for A's phone to use its own location to query for nearby phones, and then, from the returned list of devices, query each in turn, via some presumably standardised interface, for a user pro le, containing, e.g., B's name, picture, cryptographic public key and \rating account" details. To ensure as best as possible that ratings of B were tied to B's actual behaviour (and not, for example, the behaviour of someone who has stolen B's phone), it may be necessary to require some biometric identi cation of B. For non-proximate ratings, pro les can be shared via social media.
2 can be developed using a distributed ledger with smart contracts. If A
and B both have accounts on a distributed ledger, publicised via their user
pro les, then it would easily be possible for A to submit a numerical rating to
B's \rating" smart contract via a cryptographically-signed request. If C reads
B's pro le, then he can query B's rating contract to view the rating n. A suitable
public key infrastructure would support trustworthy attribution of the rating to
A and prevent forgery, and the properties of the distributed ledger would ensure
that ratings could not be tampered with once made { collectively making the
ratings e ectively irrefutable. In fact, we are currently experimenting with the
scenario of distributed-ledger-based reputation transactions ([
Signed and immutably timestamped records on the ledger would provide, therefore, an e ectively irrefutable record of all rated interactions in a person's history with the system. While it would be possible, and perhaps simpler in some ways, to implement a system using a centralised database without a ledger, one could not then be certain that ratings could not be tampered with by those with access to the database.
The technologies required are already developed and in use, and it would not be technically di cult to build the Nosedive system using them. 4
It should be noted at the outset that there are already currently-obtaining
situations in which people voluntarily take part in systems in which they are rated by
other users, either explicitly or implicitly. Every user of the auction site eBay [
The bulk of existing research into online rating systems has focused on
online sales systems or discussion forums such as these. Desiderata for an online
reputation system are suggested in [
It is clear, however, that the disastrous outcome for the protagonist in
Nosedive is not far-fetched; a system for ubiquitous ratings of people, for any reason,
and with profound social consequences, is ripe for abuse. We envisage this abuse
in the form of malicious, trivial or thoughtless low ratings directly, as shown
in the episode, as well as in over-interpretation of low ratings in inappropriate
contexts. In existing ranking systems, [
Let us suppose that some social or economic pressure leads to the widespread adoption of such a technology. In itself this appears to be unlikely, for reasons including the scenario which is the central plot of Nosedive, but for the sake of argument, let us assume that it takes place. And let us assume that such a system could be enforced, perhaps by economic pressures, to be close to universal, so that everyone had access to the relevant hardware and software to take part, and the options for \opting out" were limited to non-existent. (In practice, we assume that none of these are achievable, particularly the latter two, and that such a system has so many manifest and serious ethical and social problems with it that widespread adoption would be resisted very very strongly, su ciently to undermine its implementation at all.) But if it did, what measures could be put in place to protect users from malicious or punitive ratings? 5.1
The most obvious, and on the surface, likely to be e ective, approach would be if B had the power to accept or reject ratings at will, but of course, to do so would undermine the concept of a rating and would lead to the vast majority of, if not all, people having only the maximum rating. Even a slight modi cation, in which ratings are accepted or rejected, but the recipient does not know the value of the rating at the time of acceptance, would still tend to lead to overall high average ratings, as people would generally choose to accept ratings for interactions which they felt had gone well.
Take into account the rating of the rater Low-rated people's ratings do not count as much as those of high-rated people, i.e., A's ratings are weighted by her own rating.
This approach might seem good on paper, but is likely to be harmful in
practice. Given the hypothesis that the distribution of high and low ratings
would come to re ect existing power structures in society, there is the potential
for this approach simply to end up entrenching disadvantage. An episode of the
sitcom Community [
Take into account the rating history of raters. The ratings of someone who frequently rates people low are weighted less than the ratings of others.
This is also subject to being gamed. All a malicious person needs to do is to give high ratings to interactions which are not important to them in order to maintain rating in uence on those who do. It also would only a ect persistently low-rating people, and would have no e ect on the situation where B receives many low ratings from a large number of otherwise typically-rating people. (This may well have been the case at a number of points in the episode, such as when the protagonist receives a lot of low ratings because she's standing on the highway in tra c, or when she takes over the microphone at the wedding.)
This would also be relevant to other forms of gaming of the rating system. For example, family and friend groups might cooperate to increase their members' ratings, while antagonistic social groups could do the opposite. Analysis of rating history could identify patterns such as this, and allow for normalisation of ratings to re ect them.
Require mutual ratings If only A rates B in relation to I, nothing happens to B's rating until and unless B also rates A in relation to I.
This would provide B with some control and the ability to avoid ratings which are predicted to be bad. However, bearing in mind we are discussing a hypothetical situation where society as a whole has decided to adopt ubiquitous rating, it seems less than likely that this level of control would be accepted.
Mutual agreement would be one of the mechanisms by which erroneous ratings could be corrected (for example, in the case of misidenti cation of a person). Meta-rating Nearby people are randomly selected to \rate a rating", with low-ranked ratings being withdrawn or reversed.
This approach has the potential to make a di erence, and has been
implemented with positive e ects in discussion systems { a notable example being the
Slashdot technology news site [
Potential di culties with this method are the overhead of \meta-rating" { would people be willing to expend the extra e ort to do it? { and the observation that meta-ratings are just as likely to be a ected by conscious and unconscious bias as the ratings themselves.
Apply a cost to giving ratings If there were a cost to rating someone, or a limit on the number of ratings which could be given (within a particular time period, for example), then it may serve to deter frivolous ratings. If the cost were nancial, of course, this would again serve to entrench existing status. If simply a limit in the number of ratings, there is a chance that malicious ratings would be less likely. 5.2
Contextual ratings Rather than a semantics-free number, ratings are instead
applied to a particular category of interaction or behaviour, e.g., \this rating is
for a customer in a retail purchasing interaction". [
This allows ne-grained information to be conveyed. Speci cally, it allows for more subtle interpretation of ratings. So in hiring someone, an employer will not (indeed, should not) be concerned with what that person is like as a wedding guest, and can therefore exclude all ratings related to irrelevant transactions. More speci cally, an aggregate rating could be computed not across all interactions, but across all interactions of speci ed types, such as all nancial interactions. With more sophisticated interfaces and possibly smarter systems, the context could be extended to support the notion of evidence for a rating to be recorded alongside the rating itself.
For contextual ratings to function, there would need to be an ontology of human interaction describing the categorical relationships between interaction types (such as that a retail transaction is a nancial interaction, for example). Manual modelling, however, is only likely to get so far, given the range and complexity of human interactions which take place. Fine-grained categorisations could be crowdsourced, with, perhaps, some central editorial approval or moderation process to avoid discriminatory or o ensive categories.
Reasoning could be carried out on semantic representations to derive new categories or specialised ratings { this has potential for positive or negative consequences. For example, it might then be possible for malicious users to simulate discriminatory categories indirectly by deriving them. Positive possibilities include, among others, the ability to identify patterns in ratings received, which might help to limit or avoid negative e ects. (The same approach would apply if it were also possible to nd out the ratings that someone had given too, which would improve transparency in the process regardless.) Purely semantic ratings The idea of contextual ratings could be taken one step further, and the idea of numerical ratings could be dropped entirely. So instead of \B is rated 4", or \B is rated 4 in relation to I, which is a retail interaction", it would be \B is rated \helpful" in relation to I, which is a retail interaction". Models would need to be developed for relevant attributes, such as \helpful" which could be associated with particular interactions, or interaction types, and with su cient ontological semantics, reasoning could be carried out as with numbers. Such contextual semantic ratings would be more ne-grained still, and o er more exibility, while perhaps helping people to avoid the habit of overinterpreting numbers. Aggregation too need not be purely numerical; aggregate categories using vague quali ers, such as \mostly harmless", could be computed. Selective visibility of ratings It could even be possible, using smart contracts, to enforce restrictions on ratings that are taken into account; if C has to specify up-front the reason for reading B's rating, with B's acceptance of the reason, there could be hardcoded, crowdsourced, or legally mandated sets of interaction types which may be considered in computing the rating. Potentially the user could have the ability to override the request and add further limits onto the considered interaction types. However, this latter ability seems likely to be subject to the same social pressures as requiring approval or mutual rating before a new rating has e ect.
Two-way personalisation Combining semantic representation of contextual rating data with selective visibility gives the potential for all parties, A, B and C, to provide semantic descriptions of preferences and inputs which can be used for ne-grained negotiation of requests and permissions when it comes to rating, being rated and viewing other ratings, from which one would expect good practices to emerge from privacy-conscious users which could be adopted by those less skilled when it comes to technology and privacy protections. 6
We have described how a ubiquitous personal rating system, as shown in the Black Mirror episode Nosedive, could be implemented using technologies available today, backed up by smart-contract-enabled distributed ledger platforms which ensure the availability of e ectively immutable histories of e ectively irrefutable records of ratings. We have speculatively discussed potential moderation techniques and Semantic Web technologies which might aim to limit the negative consequences of such a system.
None of the speculated measures do more than mitigate the possibilities of abuse, and it is hard to see how anything other than avoiding the system at all could prevent it entirely. Currently-used personal rating systems are limited to very speci c contexts, such as eBay, with no consequences beyond those contexts, but a ubiquitous system is inevitably subject to many forms of abuse.
While we believe it is highly unlikely that society as a whole would adopt a Nosedive system, there have of course been historical and recent instances of societies choosing options not in the best interest of the majority of its members. Expecting a technological solution to scenarios such as this is nave; technology can be designed to support safe choices or minimise dangerous ones, but ultimately, it can rarely, if ever, enforce them. if there were a signi cant danger that Nosedive-style ratings were to become a ubiquitous phenomenon, the most e ective approach would likely be to tackle the economic and social pressures leading to this danger.