Examined is a structured play testing methodology based on an observational study to evaluate if a board game has intuitive features, and what features of design a user will find to be confusing. By presenting a game with only the elements of the game to a set of users and performing an observational study of play, mechanics, objects, and themes. The rules booklets should only be used as a method of teaching a player as a last resort and place intuitive embeddings into the parts of the game itself. We examine a small set of games via the process as an initial proof of concept. Observational studies of intuitive features will enable games designers to ensure a swift transition from unsealing the game's container to enjoying playing.
Intuitive design for board game design refers to ensuring that
player is able to effectively learn the rule of a game in a
reasonable time frame to allow players to enjoy playing the
game rather than reading rule books. In 2011 at MIT, Daviau
proposed what he deemed a sadistic experiment on a group
of smart people who “spend most of their time inventing
molecules or building cold fusion coffee makers”
This paper does not look at the results of any individual study but instead examines the refinements of the methodology and gives recommendations as to how to improve future studies with a focus on play testing for industry.
Use of qualitative play-testing alternatives such as
Personas
In (Hattula et al. 2015) managers were given example products to come up with a marketing plan for them. Those that were told to keep user demands were more likely to frame existing beliefs and did not respond well when challenged than those not primed to take into account their beliefs as to what a user would want. This is the same process which Personas use, taking into account a hypothetical user and such outcomes may expand from this study into these systems.
Hence, the best method in order to see if a design is
usable and insinuative is still user testing, even with all of its
inherent costs. Ergo, utilizing play tester time effectively to
show problems in the design becomes a priority.
Talking about object interaction, affordance theory is
significant to understand action possibilities with objects and
the environment. The word affordance was first introduced
by psychologist J.J. Gibson and explained in his book
Ecological Approach to Visual Perception
On the subject of board games, Browne examines the
elegance of rule systems which exhorts the designer to
produce rules which are self explanatory to players
Genchi Genbutsu, Americanized into “go and get your boots on”, encourages an engineer or manager in a manufacturing process to go and see the problems and issues themselves and not rely on just reports. The standing in the circle was a story, perhaps apocryphal, that a chalk circle would be drawn on the factory floor and new engineers and managers would stand in the circle observing the process until they would be allowed to propose any changes. The observations lead to an understanding of the state of the system before any actions could be performed; time must be taken to understand the reality of a system before acting. In order for these observations to be informative, the observers must ensure that the study is done with in situ game play.
In order to allow for such understanding the conditions of any observations should be made in an environment as close to in situ game-play as possible. This process is similar to how gaming groups will utilize board games, pulling a new game out of the box and attempting to grasp the rules and play the game. By ensuring that rules are intuitive, for a conceptual model to be produced in a short time frame, this allows players to quickly start a session of play and is thus a desirable and marketable feature. The adage which games should stride towards is that they should be “easy to learn but hard to master”; allow new players to join into the game, but allow veteran players to still feel engaged in the game.
An observational study is an empirical investigation to find
cause-and-effect relationships if it is not feasible to apply a
controlled experiment, for example when you cannot create
a random sampling or assign participants at random to
treatment and control groups,
This section imparts the methodological framework for examining intuitive design via the removal of rules during user testing. Subsection examines the operational process of the experimental method, how to set up the games and the process of receiving user responses. Subsection gives guidance to the observational team. Finally, subsection explores the limitations of the observational study method and possible solutions to factors which may lead to incorrect or inaccurate findings.
Observational studies require that a treatment is applied in order to demonstrate the cause-and-effect. The treatment explored by this study of games is the removal of all rules from the game, e.g. the game in Figure 1 has the treatment applied in Figure 2. We define Removal of Rules as broadly including the removal of any quick reference guides or cards, play mats, and the game container, if it has any flavor text or framing to the game. As a consequence, play tester’s actions should be based on solely their ability to produce a conceptual model of the game mechanics from the artifacts of game pieces and their embedded rules and framing. This refers to Norman’s idea of perceived affordances. This methodology also enables users to try out actions they render possible. The removal of rules release them from any constraints and, as the process goes on, play testers have the opportunity to recognize actions that were not readily perceivable. This approach is fundamental to understand all sorts of object interactions that can take place even if the users do not perceive them immediately.
Parts should be represented as close to how they will be
distributed in the game box as possible, perhaps even with
any box inserts. This means that objects which need to be
punched out of sprues and card, or parts which need
assembly into larger constructions. Placement of objects within the
box and their relations can, therefore, be accounted for as
well as any build in the organization on the part of the
developers to keep like objects with like. Developers could also
use blank boxes, without identifying information or text.
Note that it was stated in the Daviau anecdote that games
were presented only without the rule books, hence there
was no experimental control on the amount for information
which could be inferred from the box
The play tester groups are then assigned to a game and
informed of the task, see Figure 3. The size of a play
testing group should be no less than the number of players
allowed by the game itself. This ensures that the testers can
potentially attempt a round of game play. The testing teams
are given a short demographic questionnaire examining their
basic details as well as previous knowledge in playing board
games. Other personality tests such as OCEAN evaluations
could also be given as this point
The play tester groups are then given time in order to evaluate the game while under observation, see Figure 4. This evaluation should be done in a set time period which is based upon the expected time of game play — e.g. a game which is designed to take fifteen minutes should not reasonably take more than fifteen minutes to learn. Shorter and longer time periods can be tested with a shorter time period expected to yield the immediately intuitive ideas of the game. For example, teams looking at games for very short periods of time may only be able to work out the basic features from colours, shapes, and numbers. Those with longer periods will begin to see relations.
Finally, the play testers give their rules for the game during a wrap up meeting, see Figure 5. This may be either a written report or a verbal report. Written reports allowing for a clear matching of rules to functions, and play testers can give their comments on the features linked with rules, allowing for a quantitative analysis using such methods as word stemming in responses. Linking common features and parts of the game to the commonly appearing mechanical concepts. Verbal comments are best for extracting information about the qualitative ideas of the game, how frustrated they were by the process and what rules were decidedly confusing. Frustration is verbally easy for an observer to recognize, one will eagerly tell you how they were not able to fill the answers, but will be likely to not appear in written reporting, or appear as a blank to a question’s answer. Optimally an interview team should use both methods in order to provide a better evaluation.
This section explores the directions for observers, how they should question the play testing groups and open the dialog.
Observational teams should take care to see how such conclusions are made from the features of the games. For example, using chess as the game, the number of players in the game is determined to be two as there is a white set of pieces and a black set and both have the symmetrical types of units. Further, observers should pay attention to how the player testers manipulate the objects of the game; are they sorted by size, shape, colour, or other factors.
The observational teams should direct the players to answer some basic questions about the games during the testing, such as:
How many players does this game allow? (Min/Max) What is the order of play? (Turn-based, Turns with interrupts, all at once, etc.)
The questions should include elements of relating the findings back into what parts of the game led them to such conclusions and observers should encourage play testers to justify their answers to be clear about what features of the game are including in this decision. Asking probing questions to elicitate such answers can come from the observers. However, observers should not prevent mistakes on the part of the play testers, nor give feedback or direction as to the rule of the game in order to not direct players past legitimate difficulties in the understanding. Responses can be made on both the individual and team levels, though responses from the team should meet with a measure of conscientious, much like how house rules during game play are made.
The following presents a case study of this observational process.
Participants were all part of the Innopolis University Winter School and were attending a class on game design trends instructed by the authors.
There were ten male and four female participants. Their ages spanned from 21 to 32 with a median age of 25. Participants all had English as a second language at the intermediary level. All of the participants were university undergraduates.
The group was presented with a set of games, each pair was allowed to select one in turn by their randomly assigned group number. They were told to select a game by the criteria that it could not be a game which they had played before. Teams then returned to tables with their game to examine them. They were then informed that they would be presenting on the game, showing others how it was played. Finally, they were informed that all rule books had been removed from the games (in this early study we did not remove all the boxes and rules objects in the box to conform with the Daviau study - though we do advise it for future works as it conflates information from the game objects and from the box’s theme as mentioned above). We used a limit of fifteen minutes, longer than that used by Daviau, in order to account for potential issues in language, the games presented except one were all in English. Carcassonne was presented in the Russian edition, which was the native language of the participants, but other than the box, no other language was presented on the objects.
As said, our aim is to determine if specific game design choices may impact how intuitive a game is. To perform such determination, we need to define the key set of features that characterize a games design, these are the independent variables. Once we have found them we will compare how the different games perform when the instructions are removed and we will try to link back these performances to such features. Since this is an observational study, we will not employ any inferential statistical technique to draw conclusions.
The key features that we have identified are:
Game Boards or Play-space — where we define a playspace to be an inferred board created by actions in the game such as the discard, playing a card in front of you, and draw piles on a table. These actions could have a board created for them, and such play-mats exist for a number of card games such as poker or blackjack tables.
The selected tabletop games are examined in this section.
The majority have widely varied items used in their
construction: cards, wooden cubes, meaples (a portmanteau of
‘my people’), tiles, dice, counters, and pawns. Also, they
have a mix of both European and North American style
designs. Some have won awards for their design, such as
Carcassonne winning the Spiel des Jahres and the Deutscher
Spiele Preis awards in 2001
To simplify the analysis in this section we summarize the features we have in Table 1. We omit the board as a feature, as in all cases but the two which use score boards, Carcassone and Condottiere, it is defined by the play.
From the observation above, Table 2 summarizes the outcome of students and their ability to use the parts examined to create an understanding of the game, that is, which features of the game helps the most in its understanding it without having to go through its manual.
The final results show that a particular assemblage of parts is not enough to ensure the imparting of knowledge about the game rules. For example, games which came in boxes were likely to have the number of players printed on the box, yet the team for Flip City did not find this information, whereas in Hive Pocket without a number of players printed on the bag this information came from the games other parts, the number of players was defined by tiles having only two colours. Thus, the connection from a game part type to a game rule is rather weakly connected via our initial observations. Our summary does show that PirateFluxx and Carcassone were the easiest game for the teams to understand. PirateFluxx embeds the rules to the cards almost completely and Carcassone uses both colours and forcing functions, the poke-yoka, for tile placements.
Our least well defined game was Condottiere, which has the most interacting parts. A player must conceptually understand the scoring board of Italy and a set of tokens, and a large set of cards. These cards give no text as to their context in the game and symbolically give information to the player by a combination of colour, number, and image. The cards also cause effects, such as where tokens will be placed on the map, who can take the first turn next, if there needs to be a modification to all values, or if they are protected from such a value modification. It is very unlikely that even an advanced game player would be able to infer all the rules from just the parts alone. The contextual framing of the images helps the game, it being about two armies clashing in the Italian Renaissance, but it would be unlikely for a player to realize that the scarecrow card has the mechanical of taking an army card from the playing area and drawing it back into one’s hand for a later turn. Though in framing context this obvious in hindsight once described — it was a decoy army.
Observing the students and their methods in order to find the factors which would allow them to make rule discovery, we saw a commonality in the process used in the discovery. The students used the game boxes to allow for the inferences of the theme of the game and used the text on it as a stating point. As some of the game boxes made mentions a number of players and of various mechanical actions within the game — e.g. Flip City: “The citizens of Flip City want you to expand and renovate their neighborhoods – but if you ask for too much in taxes all at once, they won’t be happy! To keep them at ease, you will:
Push your luck Build your deck
Upgrade Cards by flipping them”(Zhifan 2014)
The theme of the game was an important factor in understanding, as identified above in Condottiere as well. Parts then underwent some type of classifications, by factors such as colour, shape, and type. Items which expressed or inferred numbers were more heavily scrutinized by the students, such as the values in love letter. For games without descriptive boxes, i.e. Hive, the name of the game made an implication which was hard to ignore — the idea of the the tiles acting as cells in a hive structure. The colour also was able to allow for implications of the number of players, the two players in Hive, five in Carrcassone, or six in Condottiere, directly stem from the colour on the tokens used in their representation. Note that games, where there were the most confusions on a number of players, were the card games without the use of tokens, except the player number for PirateFLUXX was discovered via the package.
Further, the findings highlight some games were quick reference cards and other rules aids can improve the play. Looking for example in PirateFLUXX which has the majority of its rules and play embedded in the cards, the student’s mistakes as to the number of cards at the beginning of play in one’s hand, and what the initial actions be fully embedded on the single removed “Basic Rules” card — those rules being that you draw three cards to start and when no other rules are overwritten you must draw one card and must play a card on your turn. Similarly, love letter’s quick reference card gives the turn order, a short description of the value of each card, and the number of that type of card in the deck. Such information as the number of each type of card would be known to advanced players but not apparent to novices. Given that some of the card’s powers are based about guessing as to opponents hands, such knowledge of a number of potential cards in play allows players to card count. This information being part of a quick reference card held by all players allows for both novice and advanced players to apply such strategies and increases the speed of formation of the conceptual model.
Decks of cards, meaples, marbles, and even chessmen, have had a number of games created for their use. Early rules for chess limited the movement for the queen to one to three spaces, and later this was changed to the entire board1. A 1The change occurred sometime in the late Medieval to Renaissance period and was known as the Mad queen variant or in Italian alla rabiosa and was historically mentioned in Scachs d’amor (de Castellv´ı, Fenollar, and de Vinyoles 15th century) as part of a poem dialog. The chessmen in this game were also stated to be red and green rather then the current white and black designations. Cthulhu Hive Pocket PirateFLUXX Love Letter Flip City Condottiere
Container Box with game art and description None Bag with title of game Box with game art and description Bag with title of game Box with game art and description Box with game art and description None None Keeper, Action, Creeper, Surprise, Rule, and Goal Cards Face cards with value Double sided building cards Battle Cards Tokens Meaple followers and a meaple scoring token Score Counters Hexagonal titles None Score Counters None Faction score tokens, Pope, Battle deck of 52-cards (or the 78-card French Tarot deck) can be easily used for poker, hearts, bridge, and euchre. Variations and house rules have been common in such games. The process of rules discovery has shown in this study to allow for the creation of new games with old parts. The team for Hive seeing the game more like droughts or Stratego than chess, placed all the tiles in a rank order, and allowed for moves based on rank. A move would be allowed if a piece outranked the one over which it was moving. Given some play testing such a game might be found to have a consistent set of rules allowing for the playing of the new game as a variant.
You cannot expect in mechanically complex games that all rules will be in the end meet with an intuitive requirement. Even in the relatively simple game of chess, the rule of en passant2 does not come naturally to even players well versed in the rules, and situations for its application are limited during play.
During observations, the emotional state of the observed
before the study and their demographics should be
controlled features. Players with experience of numerous board
games may be able to make a determination of rules which
are not expressed by the game itself but via the use of these
previous experiences. This “filling the gaps” via the
scaffolds of previous learning can both assist in
understanding if the conceptual models are close, or can hinder
learn2en passant coming into existence sometime in the 1800s when
pawns were allowed to move twice on the first move but not being
accepted for a number of years after
Further, game framing and cultural experience can lead to
differing results and should be controlled as a biasing
factor. For example, the game Condottiere
Through the use of a guided play-testing method removing
the rules from the game and observing game play sessions,
designers can extract the features which allow for intuitive
play. While Daviau presented an anecdotal account
These studies seek to quantify, via existing games, how
well the design ideals rule as presented by
Finally, the process can be used as part of not only a
rules evaluation but as part of a rules development method
for common parts. The rules seen in According to Hoyle
This study applied to a set of random parts asking players to come up with rules for them could become a practical means of brainstorming. Particularly, companies with intellectual property rights3 to game parts could utilize this as a method to create new games from said proprietary parts. This protects and utilizes their current patents with new innovations.
An extension of this experiment over a wider group in a more controlled setting would be able to provide a clearer picture of the role of intuitive design. Are there common themes what makes a game intuitive and can we use this method as a classifier of games. By looking at a proper measure in the difference between the rules created by the participants against the real rules. If there is also a large difference between participant teams on the same game then this should be identified as unintuitive points in the design.
Our games were not selected to meet with a well defined selection over the set of board games. Nor did we apply more than one team to each game, as the immediate concern was providing an engaging class and not a detailed experimental design. Such a design, now that we have seen the encouraging outcome of this observation, should be undertaken. We 3Such as Loony Labs’ Pyramids for which developer is reported to aim for construction of games which utilize this IP. would also remove more of the packaging in this study, in order to better look at only the parts themselves and not the framing of the game box.
Developments upon this process first examined in this
study have lead to findings in the differences of age
The authors wish to thank Innopolis University and all the participants of the Winner School. Further, we thank Drs. Qu, Kassab, and Ivanov, and Aliya Lutfullina for organizational support for the school.
This winter school was partly funded via the Russian Foundation of Basic Research under grant number 34 1637-10031n16.