Towards the Simulation of Large Environments Jorge J. Gomez-Sanz and Rafael Pax and Millán Arroyo 1 Abstract. The development of a smart environment working into smart environment is even built. In order for the simulation to be real- large facilities is not a trivial matter. What kind of intelligence is istic, the simulation has to reproduce the interplay among users and needed and how this intelligence will interact with individuals is a between users and the environment. The later includes too the de- critical issue that cannot be solved just by thinking about the prob- vices that are supposed to make the users behave differently. These lem. A combination of social and computer science methods is nec- devices do provide stimulus previously validated by experts as can- essary to learn and model the interplay between the environment and didates to produce the desired kind of effects. the environment inhabitants. This paper contributes with an ongoing The paper is structured as follows. First, section 2 analyses if a case study that exemplifies this kind of combination. The case study particular behavior alteration/induction is really possible. The con- considers two faculty buildings and a behavior to be modified. The tribution of social sciences to the requirements gathering is made in goal is to design a set of devices that sends signals to passing-by 3. A guideline is proposed that combines interviews, surveys, but also pedestrians in order to make them use more the staircases. Banners, field studies, as well as analyses of the captured information. Exam- videos, and directed intervention are used. The effect of each one ples of the analysis phase is made in section 4, where a domain ex- is measured and such measurements are reproduced into computer ample is introduced. The design of simulations that aid to create the simulations. This information is necessary in order to determine the smart environment capable of enacting the new behaviors is made in duration, the intensity of the stimulus, and the response of the indi- section 5. The related work is introduced in section 6. Conclusions viduals. Opposite to most works, the measurements do not provide are presented into section 7. full information of what is going on in the large facility. As a con- sequence, algorithms and software to fill in the gaps consistently are needed. The paper describes the current state of the simulations and 2 Stimulus for Behavior Alteration the difficulties in modeling with precision the results in a case study. Literature shows that humans are sensible to external stimulus in sub- tle ways and that our behavior can be altered. The extent of the alter- 1 Introduction ation may depend on the individual. Some may react notably while If a large facility is expected to host an embedded system, as in a others hardly react. Nevertheless, the average person ought to notice Internet of Things or Ambient Intelligence scenario, the definition this. The nature of the stimulus matters too. In certain conditions, of such system and its early validation is largely missing in the lit- such as evacuations, humans pay more attention to other humans erature. Given a particular building or large space, a first question rather than other artificial elements, such as banners. is whether the goal to make the visitors of the facility show a new Humans have sensibility towards the behavior of other humans. behavior or to alter an existing one. This cannot be done in the tradi- If an individual finds a group along the way, depending on its size, tional way, by consulting a few stakeholders. Interviewing and sur- will either stop and look what is happening and stay or keep walking veying the occupants of the environment seems more adequate. [11]. The larger the group, the greater the effect. This is explained How this information is captured and reused later on, it is still as a mirroring behavior effect. If sufficient people stare at an arbi- an issue. Documenting is out of question. However, the format of trary point, a passerby individual will unconsciously look at the same the documentation can be subject of discussion. Also, how this doc- place [5]. Gaze copying happens mainly within 2 meters range and umentation is used and accessed in order to ensure the problem is the response depends on the physical layout of the environment, the completely understood. social context, and the sex of the individual. The hypothesis of this work is twofold. First, that social sciences When the stimulus come from artificial sources, the results are still methods can be used in order to capture better the behavior of the promising. Sound and images can affect the behavior of pedestrians. humans inhabiting the large facility and what stimulus can trigger Beyer et al. [2] introduce an experiment where an interactive large this behavioral change. Second, once there is a preliminary solution banner display affects the audience. Through visual stimulus, authors to the creation of new behaviors or modification of existing ones, the manage to attract approaching pedestrians and distribute them along enactment of such behaviors can be something better documented if the display. Miller [12] shows that noise can affect people’s perfor- computer simulations are used. mance. A sleepy person may be aroused by noise, but it has also Besides, documenting something as dynamic as the behavior of negative effects, like affecting the performance of complicated tasks, big groups of humans through simulations allows too to experiment affect negatively the mood and disturb relaxation. Negative effects with the expected effect of different stimulation procedures. This could be used to influence pedestrians. In this paper, it is assumed way, responsible of large facilities can have tools that enable them that, since it can annoy people, this could be used to clear out areas to discover how they want the facility to be altered before the actual or to reduce the pedestrian traffic around some places where the noise comes from. 1 Universidad Complutense de Madrid, email:jjgomez@ucm.es The context matters too. Foster [4] analyzes different domains in order to promote healthy habits. Each context is different. A shop- their collaboration is needed too along the design stages. Human sci- ping center and railway station involve different behaviors on behalf ences scientists, such as psychologists or social scientists, provide in- the population. sight into the behavior of the users beyond common wisdom. Hence, Also, sensibility towards stimulus changes depending on the con- they are needed in order to properly design the field experiments, to text. In an airport, passengers pay special attention to information study the results, and to assess the validity of the simulations. panels. A change in one panel may trigger movements of user groups, It is assumed that there is a simulation parameterization whose be- such as changing one boarding gate ten minutes before the boarding havior is close to the observed behavior. Such simulation should be starts. Fun parks also influence the behavior of their visitors through possible because the behavior of users into installations is not het- information panels that tell expected waiting time for each attraction. erogeneous and tend to fit into standard behavior patterns, that we associate with activities of the daily living typical of the installation. The definition and parameterization of the simulation is considered 3 Guidelines for Developing Ambient Intelligence following. in Large Facilities The system to be developed aims to interact with several inhabitants 3.1 Specifying the crowd simulation of a large space. These inhabitants may be transient ones or perma- An important part of the simulation is the description of the physical nent inhabitants of the considered space. It is assumed that the peo- space inhabitants, which is called here population description. For ple in this physical environment can be either a management staff, in this goal, it is necessary to identify a set of possible actor behaviors, charge of the facilities and aiding to the occupants of the facilities to an enumeration of the number of instances of these behaviors, and a fulfill the identified system goals; and the visitors, who are the clients timestamp of when the behavior&actor instantiation happens. of the facility. In general, the staff interacts with the visitors in order Actor instances are created along the simulation and destructed for helping them perform certain activities. In the physical space, it when the behavior of the character finishes. It is assumed the de- is assumed the staff is expected to modify the behavior of the clients signer determines a suitable place where this destruction happens. in a way that clients perceive an benefit. After all,actors cannot vanish from the scenario just anywhere. These To identify what behavior modifications are possible and how to actors instantiate a particular set of behaviors with particular param- best convince inhabitants of the facility to commit to such behavior, a eters. The different parametrization determines individual variations guideline affecting particular system development is introduced fol- of the behavior. lowing: It is assumed that actors can belong to two distinguished groups: • Analysis phase. The facility to be analyses is assumed to fulfill one those responsible of operating the facilities and those visiting the fa- or many purposes. The staff is expected to alter the behavior of the cilities. The first are expected to perform different activities oriented inhabitants in order to achieve certain behavior. This behavior is towards coordinating the behavior of the second group within the fa- compatible with the purposes of the facility, and it is supposed cility. The second group are executing activities of the daily living to be regulated or activated through some environmental devices. related to the main purpose of the facility. It is not expected that one There is a review of the meaningful behaviors, according to the actor belonging to one group suddenly becomes an actor belonging literature, on the expected behaviors (domain or non-domain spe- to another. Even thought there maybe cases where this role switch cific) for the chosen facility. A selection of stimulus is made based makes sense, it is not considered in this paper. Within each group, on the available resources (the budget of the modification, for in- there can be further decomposition of responsibilities, but it depends stance). Also, field studies have to be planned to know more of the on each particular domain. visitors and also to evaluate the effect of those stimulus over time. An actor behavior specification consists of a sequence of param- Effect of each stimulus is measured and annotated so that it can be eterized activities of the daily living plus an initial location. The reproduced later on. Each stimulus is expected to have a duration amount of instances of each actor behavior specification determines and an intensity. the composition of the population. • Design phase. The different stimulus and the expected reaction Actors are not allowed to alter their behavior and they constantly is modeled into a simulation that serves as reference. The sim- perform the same sequence. The sequence terminates with the de- ulation includes the physical space, the inhabitants of the space, struction of the character. This enforces designers to define precisely the expected behaviors of those inhabitants according to the field what actors are expected to do since their creation until completing studies, and the simulated devices that are going to provide the their part in the simulation. stimulus. The measurements made in the field studies are interpo- lated to guess the overall behavior of the whole population. Ac- 4 The case study cordingly, the expected behavior is studied, taking into account the reaction to the stimulus. As a result, an expected orchestration The crowd simulation has been applied to a scenario situated in two of the stimulus is obtained. faculties. The goal is to alter the behavior of the inhabitants in order • Deployment phase. The synchronization of the stimulus is de- to make them choose an activity that requires additional effort over ployed into real devices already working in the facility. The sim- another activity that does not. The behavior to be altered is using ulation is expected to have identified several critical observation the elevator, which ought to be replaced by using the staircases. The points whose measurements indicate if the stimulus is working or experiment is run into two different faculties, the Computer Science not. Faculty and the Political and sociological Sciences faculty. The application of the methodology starts with a field study struc- The role of human scientists is important in the development of tured as follows. First, the managers of both faculties are interviewed this kind of systems. In this guideline, it is assumed that human sci- to know more of the daily problems they have to face. This provides entists involve themselves mainly into the analysis stages. However, an insight on the students and other staff using the facility. It also helps to identify possible incompatibilities between the planned stim- to use the elevator is reduced up to 4 points in phase C2. This is a ulus and the current activities. The chosen stimulus are: variation of 13,65% over the original use of the elevator. The results are not shocking, but it should be taken into account that each stimu- • Human-to-human interaction. A person playing the role facility lus lasted for one week, and not months. operator interacts with another playing the role visitor and tries to suggest the use of staircases is better. Table 1. Variation of the traffic in elevators into two faculties • Banners. Banners are proposed containing information of interest to the visitor and that may aid in suggesting an alternative behav- ior. It is important to notice that there ought to be an evident profit % use elevators A B C1 C2 D for the visitor, otherwise the behavior modification will not occur. Total 23,1 21,9 21,4 20,3 22,2 In this case, the banner is presented at figure 1. It suggests the vis- Departures 29,3 28,2 26,2 25,3 26,4 Arrivals 14,4 14,4 15,2 14,0 16,2 itor will gain health improvements, will arrive faster to the desti- #total= 9730 9797 9459 9165 9088 nation, and will save electricity. These facts, specially the savings #departures= 5688 5371 5335 5109 5345 in time during travels, has been proven to be true. #arrivals= 4042 4426 4124 4056 3743 • Multimedia. A video shows a dramatization of a person that uses the elevator for everything even though can perfectly walk. The With the obtained traffic data, a simulation is arranged so as to video is shown through a short distance beamer sufficiently visi- reproduce the observed behaviors. ble and the equivalent of a 55’ screen. The short distance beamer is projecting vertically and permits a less disturbing installation. The projection is made close enough to the elevator. Due to safety 5 Reproducing the experiments concerns, it was not allocated right next to the elevator. The result of the experiments is being transferred to computer sim- ulations, to identify complexities and capture individual behavior as precisely as possible. SUBIENDO LAS ESCALERAS Y EVITANDO EL ASCENSOR… In the simulation, all actors are belonging to the visitor role. Their actions consists in entering the building, visiting a previously un- 1. MEJORA TU 2. LLEGAS MÁS known number of rooms, and exiting the building. Hence, a parame- FORMA FÍSICA RÁPIDO. Según mediciones terization of the problem includes an account of the rooms each actor Fortalece las piernas, mejora la hechas en esta Facultad bajar es un visits and how long they stay there. actividad cardiaca y bajas calorías. 20% más rápido por escaleras y subir más o menos igual. The computer simulation has to capture emergent behaviors. UCM Rather than organizing dynamically the behavior of a whole popula- 3.AHORRAS tion and letting a central node orchestrating everything, the approach ENERGÍA…y is multi-agent based one, where individual behaviors of characters contribuyes a mejorar el medio ambiente is coded. The individual behaviors is explained along the next para- graphs, but the goal is to attain the same, or close, traffic data to those obtained from the different experiments. Since the data from each phase is available, the simulation ought to capture the effect of the stimulus over the visitors. Henceforth, if the stimulus is a banner and the measured effect is a 25% variation, then the simulated traffic ought to show such change as well. Figure 1. Banner for motivating users to use the staircase. It written in The total aggregation of the traffic ought to provide with num- spanish. The main title says stair climbing and avoiding elevators at the top. The alleged reason are 1. improving your health, 2. You will get faster to bers similar to those of table 1. Achieving this traffic data while cod- your destination, and 3. you will save energy ing individual behaviors is a hard task because of two reasons. First, there are several elements whose interplay affects the outcome of the simulation. Actors interact among themselves and with the environ- A plan for measuring the effects of these stimulus was made. The ment, specially elevators and the physical layout of the environment, plan consisted on a five week schedule. The first week (week A) a building with several floors. Second, the gathered information is there was no stimulus and it was used to collect a base line of stair- partial, since only a few pedestrian traffic check points were estab- case/elevator traffic stats; during the second week (week B) the ban- lished in the field study from section 4. This means there were not ner stimulus was introduced; along the third week (week C1) the cameras recording the full activity. As a consequence, there may be videos were added; and in the fourth week (week C2), the human- many populations of simulated actors whose movement along the fa- to-human interaction. Then, there were some days of no stimulus to cility matches the obtained measurements in the field experiment of let users decide whether they want to keep the new behavior or get section 4 back to the old one. Therefore, the fifth week (week D) is dedicated The problem has been studied in [13] and the provisional solution to measure the resilience of the stimulus. is a greedy algorithm that produces a population of actors whose Collected data was a set of pedestrian traffic into strategic check- behavior matches to some extent the expected behavior of the whole points of the faculties. Measurements indicate whether visitors come population. A first attempt is presented in figures 2 and 3. or go, and whether they are using the elevator or the staircases. An The behavior of each individual can be summarized as follows. account of persons per minute is provided. The resulting influence of Each character has a navigation path from the starting point to a the stimulus along the field experiment stages is included in table 1. particular location determined by the greedy algorithm [13] and go- The number of people arriving through the elevator remains mostly ing through some intermediate points that are part of the parame- the same along stages. However, the number of people choosing not terization. Intermediate points may correspond to specific rooms the characters may or may not visit. Along the navigation, the character may find obstacles. Fixed obstacles are already avoided by the nav- igation algorithm. Mobile obstacles are avoided through maneuvers around the expected collision points. Afterwards, the navigation path is rechecked and resumed. Figure 2 shows a part of the 3D simulation created with the greedy algorithm. In the simulation, to compare the simulated vs the real sce- nario, the simulation assumes there is a device in the area capable of counting people as they cross the section corresponding to the check- point. The counting is compared against the real measured traffic in the bottom part of the figure. Figure 3. Elevator carrying people from one floor to the other precisely the lectures to finish exactly at the same time. Such indi- vidual behaviors are relevant to be modeled too. Also, the simulations may lead to inconsistent results. For in- stance, most of the resulting populations according to the algo- rithm [13] have in common that upper floors are mostly empty. Upper floors only have offices and not classrooms, what would explain this result. Then, it may be subject of discussion if a better occupation of the building was possible. If the space allocated in upper floors is the same as lower floors while the traffic is much lower, perhaps a higher number of offices could be arranged without compromising an eventual evacuation of the building. Capturing complexity at the simulation allows to realize the Figure 2. Simulation of pedestrian traffic along checkpoints and simulated software-in-the-loop approach. It is a goal of the project to include traffic data gathering sensor/actuator devices in the simulation so that a designer can ex- plore the effect of the stimulus of those devices on the population. The simulated devices would be operated using control software that There are many possible populations of actors whose movements was close to the simulated one. This approach has been essayed in have the same effect in terms of traffic through the checkpoints, at [6] for gesture recognition devices design using 3D simulated envi- least, in theory. The greedy algorithm from [13] achieves the perfor- ronments generated with the AIDE environment [3]. mance shown in figure 4. This figure focuses on the traffic data and compares the simulated to the real measured traffic along the experi- ment. The considered time window is different from 2. In figure in 3, there is a small variation in the obtained simulated traffic measure- ments. The main reason for such variations is the interplay of actors along their paths, which is not taken into account. Collisions and bot- tlenecks happen too, and they cause a different transit time. This is a positive sign the simulation is more complex than the simplified model the greedy algorithm uses. Another source of complexity is the modeling of elevators, as shown in figure 3. The characters that occupy the interior of the ele- vator must coordinate to exit into each floor. Problems happen when one character situated at the back of the elevator wants to get out, but no one of those situated at the front wants to move. Again, this alters Figure 4. Measured traffic in the simulation compared to observed real traffic using a different time window from figure 2 the traffic. To prevent this, the simulated actors have to be aware of what is the right use of an elevator. The problem becomes more complex when the activities of the daily living is added to the considerations. The protocol of lectures in a classroom is simple: students come to the classroom; they sit down; 6 Related work a teacher comes and starts the lecture; more students may come dur- ing the lecture; the lecture finishes and then all, or a few, students There are works dealing with the design of smart systems, but they do leave the room. The uncertainty in the process, such as teachers fin- not frequently consider human sciences and stimulus to plan the kind ishing sooner or later, makes the evacuation of students from class- of system which is needed and what performance it will have. Harri- rooms more smoother than it should be if all teachers coordinated son [7] claims the analysis of mutual and incidental user interaction has not been accounted and proceeds to apply fluid flow analysis to [3] Pablo Campillo-Sanchez and Jorge J. Gómez-Sanz, ‘Agent based sim- understand it. This kind of analysis is necessary, but, it does not re- ulation for creating ambient assisted living solutions’, in Advances in Practical Applications of Heterogeneous Multi-Agent Systems. The place a more conventional study and cannot assume a 100% response PAAMS Collection - 12th International Conference, PAAMS 2014, of the individuals every time. Other works focus on the devices ex- Salamanca, Spain, June 4-6, 2014. Proceedings, pp. 319–322, (2014). pected to provide the stimulus at small scale, such as [14]. Thought [4] Charles Foster and Melvyn Hillsdon, ‘Changing the environment to authors stress the involvement of human scientists too, the behavior promote health-enhancing physical activity’, Journal of sports sciences, of people in small spaces cannot be compared to that of large spaces. 22(8), 755–769, (2004). [5] Andrew C Gallup, Joseph J Hale, David JT Sumpter, Simon Garnier, There are precedents too in reproducing observed data as simu- Alex Kacelnik, John R Krebs, and Iain D Couzin, ‘Visual attention and lations. In [8], video recordings were used to reproduced later on the acquisition of information in human crowds’, Proceedings of the a crowd simulation of simulated actors. Behavior of the individu- National Academy of Sciences, 109(19), 7245–7250, (2012). als were obtained from a multiple checkpoint observation that al- [6] Jorge J Gómez-Sanz, Marlon Cardenas, Rafael Pax, and Pablo Campillo, ‘Building prototypes through 3d simulations’, Advances in lowed. The project introduced in this paper, however,assumes incom- Practical Applications of Scalable Multi-agent Systems. The PAAMS plete information about activities and traffic. The less information Collection, 299, (2016). is used, the less expensive a real installation would be. Following [7] Michael D Harrison, Mieke Massink, and Diego Latella, ‘Engineering the same paradigm, Lerner et.al [9] propose the creation of an ex- crowd interaction within smart environments’, in Proceedings of the 1st ample database for evaluating simulated crowds based on videos of ACM SIGCHI symposium on Engineering interactive computing sys- tems, pp. 117–122. ACM, (2009). real crowds. Bera. et.al [1] also developed a behavior-learning algo- [8] Kang Hoon Lee, Myung Geol Choi, Qyoun Hong, and Jehee Lee, rithm for data-driven crowd simulation, capable of learn from mixed ‘Group behavior from video: A data-driven approach to crowd simula- videos. Zong et.al [15] developed a framework for generating crowds tion’, in Proceedings of the 2007 ACM SIGGRAPH/Eurographics Sym- for matching the patterns observed on video data,taking into consid- posium on Computer Animation, SCA ’07, pp. 109–118, Aire-la-Ville, Switzerland, Switzerland, (2007). Eurographics Association. eration the behavior both at the microscopic level as at the macro- [9] Alon Lerner, Yiorgos Chrysanthou, Ariel Shamir, and Daniel Cohen- scopic level. Finally, Yi Li et.al [10] developed a technique for popu- Or, ‘Data driven evaluation of crowds’, in Motion in Games, 75–83, lating large environments with virtual characters, cloning the trajec- Springer, (2009). tories of extracted crowd motion of real data sets to a large number [10] Yi Li, Marc Christie, Orianne Siret, Richard Kulpa, and Julien of entities. Pettré, ‘Cloning crowd motions’, in Proceedings of the ACM SIG- GRAPH/Eurographics Symposium on Computer Animation, pp. 201– 210. Eurographics Association, (2012). 7 Conclusions [11] Stanley Milgram, Leonard Bickman, and Lawrence Berkowitz, ‘Note on the drawing power of crowds of different size.’, Journal of person- The paper has introduced a guideline with recommendations inspired ality and social psychology, 13(2), 79, (1969). into human sciences and the realization of field experiments. Such [12] James D Miller, ‘Effects of noise on people’, The Journal of the Acous- tical Society of America, 56(3), 729–764, (1974). experiments are necessary to fine tune devices that aim to influence [13] Rafael Pax and Jorge J. Gómez-Sanz, ‘A greedy algorithm for repro- the behavior of the facility inhabitants. With this information, com- ducing crowds’, in Trends in Practical Applications of Scalable Multi- puter simulations have been created. These simulations reproduce the Agent Systems, the PAAMS Collection, 14th International Conference, observed behavior and can be used to experiment with different se- PAAMS 2016, Sevilla, Spain, June 1-3, 2016, Special Sessions., pp. 287–296, (2016). tups and stimulus until a suitable combination is found. The next step [14] Norbert A Streitz, Carsten Röcker, Thorsten Prante, Daniel Van Alphen, is to devise the control software capable of synchronizing the stim- Richard Stenzel, and Carsten Magerkurth, ‘Designing smart artifacts ulus over the population and then deploying such software to real for smart environments’, Computer, 38(3), 41–49, (2005). devices having that capability. [15] Jinghui Zhong, Wentong Cai, Linbo Luo, and Haiyan Yin, ‘Learning Simulations can be used also to reason about what is happening behavior patterns from video: A data-driven framework for agent-based crowd modeling’, in Proceedings of the 2015 International Conference inside the facility. In particular, the produced simulations show there on Autonomous Agents and Multiagent Systems, pp. 801–809. Inter- are concerns in how the building is actually used. The identified national Foundation for Autonomous Agents and Multiagent Systems, traffic, and assuming most of the people that goes through a stair- (2015). case/elevator do it only once, permits to infer that the upper floors of the building are less occupied than expected. ACKNOWLEDGEMENTS We acknowledge support from the project “Collaborative Ambient Assisted Living Design (ColoSAAL)” (TIN2014-57028-R ) funded by Spanish Ministry for Economy and Competitiveness; and MOSI- AGIL-CM (S2013/ICE-3019) co-funded by Madrid Government, EU Structural Funds FSE, and FEDER. REFERENCES [1] Aniket Bera, Sujeong Kim, and Dinesh Manocha, ‘Efficient trajectory extraction and parameter learning for data-driven crowd simulation’, in Proceedings of the 41st Graphics Interface Conference, pp. 65–72. Canadian Information Processing Society, (2015). [2] Gilbert Beyer, Vincent Binder, Nina Jäger, and Andreas Butz, ‘The pup- peteer display: attracting and actively shaping the audience with an in- teractive public banner display’, in Proceedings of the 2014 conference on Designing interactive systems, pp. 935–944. ACM, (2014).