Collaborative Recommendation System for Improved Information Logistics: Adaption of Information Demand Pattern in E-Mail Communication

Abstract

Today e-mail communication information is widely used in organizations to distribute information. The increasing volume of received emails hinders an efficient work. It becomes more and more difficult to identify relevant e-mails inside this enormous volume of information. This work presents a solution in a multi-user environment by improving an established email client extension based on information demand pattern with a recommendation system. The contributions of this work are (1) a concept and implementation of a solution for a single-user environment using information demand pattern and (2) a concept and an architecture to use the solution in a multi-user environment.

Keywords

• Information Logistics
• Recommendation Systems
• E-Mail

1 Introduction

In spite of many isolated applications information overload is still a problem in our modern working environment. In order to face this problem not only a single application but also the integration of well-established solutions is the key. Every day, a wide variety of information is produced. This emergence has greatly increased in recent years. The society has changed to an information society. The availability of information can be viewed as an obstacle for a demand-oriented information supply, but finding the information by the person who needs them. This development does not stop in front of companies, so that they see themselves confronted with an everincreasing amount of information, as studies have shown [ 11 ].

Information logistics as an application field of business information systems takes care of this problem and has the goal to achieve an improvement of the information flow in organizations by providing a demand-oriented information supply. Information logistics offers methods, concepts and tools to achieve an improvement. Under the concept of demand-oriented information supply the right information at the right time, in the right quality, in the right form and at the right place for the seeking person is understood [ 12 ].

For a demand-oriented information supply in addition to the lack of information, especially the flood of information in companies is regarded as obstacle. Öhgren and Sandkuhl showed in an empirical study that around two-thirds of the Swedish Manager of the top management and middle management suffer from information overload. In that survey 37% of the participants answered that they receive "far too much information" and another 29% receive still "too much information” [ 13 ]. The phenomenon of information overload is often equated with a too much of information. Speier et.al. however point out that the amount of information, as well as the complexity and the time available for the working person are in relationship to each other [ 14 ].

In companies, e-mail has established itself as an important communication medium. Also here a flood of e-mails can be observed which Volnhals and Hirsch have shown in a study. This has negative effects on the quality of decision-making by managers and can lead to economic consequences as well. It is pointed out that information overload can be defined as the amount of information that a cognitive processing capacity exceeds [ 15 ].

Soucek and Moser, however, have identified in a survey three facets of information overload through e-mail. These are the mass of incoming e-mails, inefficient workflows and a poor quality of communication [ 16 ].

To improve the demand-oriented information supply within e-mail communication an extension of an e-mail client was proposed by Stamer [ 17 ]. This extension uses an information demand pattern (IDP) as input about the information a worker needs. The extension filters the e-mails according to his or her needs and provides a better information supply. This solution fits well to a single-user work environment. But normally workers within a team interact with each other during their work. Therefore integrating a recommendation system in order to enable interaction between the local extension installations enhances this solution.

The contributions of this work are: (1) a concept and an implementation of a solution to filter e-mails in a single user environment by using information demand pattern; (2) an architecture and a concept to establish the aforementioned solution in a multi-user environment such as a workgroup using recommendation systems technologies.

This work is structured as follows: Section 2 gives an overview about recommendation systems and information demand patterns in general. Section 3 describes the architecture and functionality of the proposed IDP-based recommendation system. Section 4 summarizes the results, gives some conclusions and an outlook on further research. Due to a better understanding of the solution presented in section 3, this section introduces the key principles of recommendation systems and gives an overview about information demand patterns.

2.1 Recommendation Systems

Recommendation systems have become widely used nowadays as they help to mitigate information overflow of current life [ 3, 4 ]. Any recommendation system involves two entities: users and items. User is a person (we may also imagine some software agent playing the role of a user, but this scenario is rarely addressed in the field, if at all) interested in interacting with items of certain kind. Products, services, web pages, blogs etc. on the other hand, may represent items. There are two crucial points that justify the development of a recommendation system in some domain: a) some items are more interesting (or useful) for a particular user than others; b) there are plenty of items, and the user has no chance to examine them all in order to find the most useful ones. Recommendation systems have much in common with search engines, but then differ in a sense that a user must query a search engine, but recommendation system acts more in a proactive way offering a user items that might be useful without explicit requests. Classical examples are movie, books, and music recommendation systems.

There are three basic approaches to recommendation systems development (not to mention hybrid recommendation systems, which usually employ some ensemble of basic approaches) [ 5 ]: • Content-based recommendation systems (CB), • Collaborative filtering systems (CF) and • Knowledge-based recommendation systems (KB), also known as constraintbased recommendation systems.

These approaches differ in the rationale that is behind the recommendation process, information used as well as information and mathematical models of users and items.

Content-based recommendation systems [ 6 ] are based on the premise that if a user likes some item he or she will probably like similar items. So, there are two pieces of information that make this kind of systems viable: the information about which items a user likes and a pairwise similarity between other items. The former is usually collected during the user’s interaction with a system and the latter requires some domain-specific analysis of item’s properties and characteristics. Similarity measures for movies, blogs and books are quite different. Typical pitfall of this kind of systems is the lack of diversity.

Collaborative filtering systems [ 7 ] are based on the premise that if two users share some significant part of their interests, other interests may also be common. For example, if it is known that two users assigned high ratings to “Green mile” and “Apollo 13” movies and one of these users did not watch “Forrest Gump” the system may infer that “Forrest Gump” may be of some interest for that user. It is important to note that this inference have nothing about the fact that Tom Hanks starred in all the mentioned movies, but it is based solely on the fact that the users similarly rated some movies. So, the only information that is used by this kind of systems is users’ attitude to various items. This attitude usually takes the form of ratings assigned to items by the users, but also may be derived from some user’s behavior peculiarities. As this approach does not rely on item’s properties it is rather universal and can be applied to almost any domain. The significant drawback however is that without significant number of ratings the statistical inference becomes unreliable.

Knowledge-based systems [ 8, 9, 10 ] are powered by a set of rules that connect users, context and items. Recommendations here are provided as a result of logical inference and/or constraints resolution. Such systems can also be seen as a kind of expert systems. The development of a knowledge-based system requires a significant effort, as all the rules and trends that are automatically inferred (and updated during the system lifetime) by recommendation systems of the other approaches must be formalized and manually represented in some machine-readable form by knowledge engineers. Hence, KB recommendation systems are usually developed in domains where there are experts that can provide comprehensive formalization of item space, for example, for browsing product/services catalogues of some company.

2.2 Information Demand Pattern

The concept of information demand pattern originates from work in the research and development project Information Logistics for SME (small and medium-sized enterprises) (infoFLOW). infoFLOW included seven partners from automotive supplier industries, IT industry and academia. The objectives were to develop a method for information demand analysis [ 21 ] and to identify recurring elements in information demand, i.e. patterns of information demand.

Lundqvist has shown in a study in companies that the information demand of an employee depends on the role in the organization that he or she fulfills. The structured collection of this information, which is necessary for the processing of work tasks, was underpinned by the development of a methodology for information demand analysis and validated [ 18 ].

After detecting the information needs of a role in a company, Sandkuhl presented the concept of information demand pattern. As with patterns in other disciplines of computer science, these patterns have the purpose to detect a proven solution to a problem in order to reuse it in other application scenarios. With information demand patterns, the identified organizational knowledge is collected in a structured and reusable way. The term information demand pattern is defined as follows according to [ 19 ]:

An information demand pattern addresses a recurring information flow problem that arises for specific roles and work situations in an enterprise, and presents a conceptual solution to it.

Information demand pattern consist of five integral components: • Name of the pattern, • Organizational context, • Problems, • Conceptual solution and • Effects.

The name is used to identify the pattern. This is usually the name of the role, which the pattern describes.

The organizational context explains the application context in which the pattern can be applied. This can be departments, functions or domains.

Problems represent the difficulties and challenges that the person is facing in filling their role in the company. In addition, the duties and responsibilities of the role are subsumed under this point also.

How the described problems of the role can be solved is shown in the section conceptual solution. It is divided into three areas to consider: information demand, quality criteria and timeline. Information demand describes the information that is necessary to fulfill the duties and responsibilities of the role. The quality criteria describe the quality in which the information must be available such as the general importance of the accuracy, the time and the completeness of the information. The timeline represents the time at which the required information must be available at the latest.

The effects part describes effects that may occur if the information is not available or not in time. The possible effects occurring may be associated with the following dimensions: economic effects, time and efficiency, quality of work, motivation, learning and experience and customer [ 19 ].

The concept of information demand pattern has been studied in several other works and its applicability has been validated [ 20, 25, 26 ].

3 Architecture and Functionality of IDP-based Recommendation

Systems

The overall architecture of the proposed contribution is described first in this section. It will be shown how an e-mail client extension can work together with a recommendation system in order to provide a demand-oriented information provision by using e-mail filtering technologies. It will be shown how the solution can be implemented in organizations.

3.1 Architecture in Enterprises and Organizations

Demand-oriented provision of information is important to avoid disadvantages by information overload. Therefore the extension of e-mail clients described in section 3.2 can be used to present the right information at the right time to a user as information logistics proposes. The quality of the results of this solution for a single person depends on the quality of the given input – here information demand pattern. On the one hand this solution does not consider the changing demands of the user depending on their current working context. On the other hand there are no connections between the users to benefit from a continuous information exchange. Therefore the solution will be enriched with a recommendation system, which will be described in detail in the following section.

Figure 1 shows how our proposed solution can be implemented in an organizational context. Inside this organizational context there are several user contexts. These user contexts represent users in his or her working environments, who fulfill a role inside the enterprise. During carrying out their duties, roles in organizations have a typical information demand, which is determined by their tasks and responsibilities. This information demand can be gathered with an information demand analysis and described with an information demand pattern. This information demand is somehow abstract, which enables the pattern to be used in different contexts like in other enterprises. Therefore it is necessary to specialize or to tailor the information demand to the exact user context. Due to the fact, that information demand pattern are right now only textual descriptions, these patterns will be transformed into a machine-readable und machine-interpretable format by using technologies like indexing. This indexed information demand pattern is used by the next step as input. After preparing the information demand pattern for use in an organizational structure the specialized information demand of a role helps to provide a demand-oriented user support, which shall help to increase e.g. efficiency of the role while accomplishing his or her tasks. From a technology perspective this will be implemented by extending the e-mail clients of the users with a plug-in. This plug-in will do locally filtering of e-mails to offer the user e-mails he or she needs at the moment to accomplish their tasks. Due to the fact, that most of the employees in organizations use e-mails to communicate with internal or external partners. Since the above described user contexts are somehow isolated from each other, we propose context-based recommendations for organizational support. This means that there should be interaction in between the different user context in order to generate benefits through information exchange. Therefore we propose to use a recommendation system. The previous installed plug-in provides feedback about the user behavior to the recommendation system. The recommendation systems itself responses to all installed e-mail plug-ins in order to adapt the e-mail filtering with newly gained information by other users.

3.2 E-Mail Client Extension

Information demand patterns were presented as a way to capture the information demand of roles in organizations in a structured way to make them reusable. These collected information demands associated with tasks and responsibilities, should continue to be used now to offer a solution to reduce the problem of information overload while communicating via e-mail.

The target is to support a person, who fills out a role with defined tasks, through an automated provision of information at the time when the information is needed. The field of application of the concept can be a company or an organization, which uses email as a communication medium and is suffering from information overload. The user will be presented for the completion of his duties necessary e-mails in an appropriate manner at the appropriate time. Interpreting the information within the email will remain with the user. The presence of an existing information demand pattern, describing the supported role, is adopted as a precondition for the application of the concept.

Information demand patterns are described so far in a semi-structured textual form. This form is neither readable nor interpretable by computers. Therefore, it is necessary to make the information demand patterns first machine-readable. It is proposed here to use ontologies or the Extensible Markup Language (XML). Within information demand patterns the specific information needs are only described linguistically and are not interpretable by computers, therefore it is proposed to assign first keywords by hand, which describe the information and thus make them identifiable. Automated indexing may help later to reduce this effort.

Information has its highest value for business at the time it is needed. Information received after that date may be worthless. Obtaining it previously, its value is still low. The information must be presented to the user, if needed to fulfil the assigned tasks. This point emerges from the timeline of the information demand pattern. In identifying the relevant e-mails not only just incoming e-mails are recorded, but also already existing e-mails are optionally labelled in order to carry the point account that draws closer a defined time point increases the value of information in e-mail. Cyclically repetitive algorithms can implement this.

The relevant e-mails to the performance of a task are now identified; they must be presented at the defined time. Highlighting about the marking of the e-mail can do this. The use of virtual folders provided by any modern e-mail application can be helpful to present the relevant e-mails to the user. It is to be noted that only the relevant e-mails are presented to the user. The user coordinates and interprets the content.

The inclusion of the proposed approach in the e-mail communication can be done locally by the user as well as on the side of the e-mail server. As an advantage of this implementation, broad support from any devices such as PCs, smartphones, Tablet PCs or Web access to e-mails is conceivable because filtering the content of e-mails happens centrally. The contrary is a higher cost to the implementation, as well as the possibility to extent the e-mail server with new software. The concept can be implemented as an extension of the used e-mail program as well. Modern e-mail programs offer interfaces to do this. As a disadvantage, it is here to note that, in this case, no support from other devices is feasible.

The concept is offered as a way to reduce information overload within the e-mail communication in organizations. It can be expected that the automated provision of emails for the user will lead to a timesaver. As economic benefits, an increase in efficiency and an avoidance of wrong decisions can be stated.

An appropriate and previously specified information demand pattern can be viewed as a limitation of the proposed approach. Also, the collected information demand must be made sufficiently identifiable by keywords. It is likely that the quality of the presented e-mails to the users thereof will be significantly dependent on. It is conceivable to transfer the duty to create the information demand pattern and the duty to determine keywords to central organization units. Economies of scale can be used if there are several similar roles in the organization.

3.3 Recommendation System Supporting E-Mail Client Extension

Goals of the recommendation system

Goals of the recommendation system: • Adjust IDP-formed tasks based on actual workers activity. Initially, all IDPs are created by some authorized entities and reflect general view on specific worker role or tasks. • Help to classify pieces of information (e-mail messages) as relevant/irrelevant to some IDP.

Informally, the goal of the recommendation system is to provide dynamic adjustments to the IDP-based structure of the information workflow.

In the context of this work, recommendation system analyzes an interaction between workers and e-mail messages within the scope of each IDP and adjusts importance of e-mails in other workers’ mailboxes based on this interaction.

Generalization and propagation of user-item relations is usually achieved through collaborative filtering that is the main principle of the proposed recommendation system. However, holistic approach to tailoring an IDP structure to the organization workflows goes beyond the traditional collaborative filtering scheme and involves variety of information processing techniques and models.

Recommendation system input data

One of the crucial aspects of recommendation system development is identifying input data useful for fulfilling the recommendation system goal. For each type of input data, the rationale that underpins its usage for recommendations should be identified. Input data selection then affects mathematical models and algorithms that are used for making recommendations. For the proposed recommendation system the following types of input data are used: • E-mail messages’ textual content and additional attributes (message id, sender etc.), • Workers’ actions on e-mail messages, • IDP descriptions and • Workers’ profiles.

Each type of input information is discussed in detail below.

E-mail messages’ textual content and additional attributes. To propagate actions that a user applies to e-mail messages in his/her mailbox to other users’ mailboxes a system must relate messages in different mailboxes. It can easily be done with multiple recipient messages as a sender program usually assigns message identifiers for outbound messages and these message identifiers will be the same for each recipient. Message identifier is put into “Message-ID” field of the message header ([RFC 5322]) and can be read by the receiver. Single recipient messages need some other approach. So the similarity measure between e-mail messages is introduced for relating messages of different mailboxes. This similarity measure accounts not only for message contents, but also for supplementary message attributes (sender, list of receivers). Furthermore, the textual contents can be used to automate message classification to IDPs.

Workers’ actions on e-mail messages. These actions are interpreted as implicit information about how useful an e-mail message is for a given user in a particular IDP. Captured actions paired with their interpretation are listed below: • A user deletes an e-mail message. Means that the e-mail message is irrelevant to this IDP and probably should be ranked lower for other users or even removed from the respective IDP-folders of other users. • A user ignores (does not open) an e-mail message. Means that the e-mail message is likely to be relevant to this IDP however is received too early. • A user opens a low-ranked e-mail message before a high-ranked one. Means that there is a sign of ranking inversion and opened e-mail probably should be ranked higher for other users. • A user marks, flags, highlights an e-mail. Means that the e-mail message is important and should be ranked higher for other users.

IDP descriptions are employed to determine the scope of actions propagation. For example, if two workers perform similar tasks (and therefore, follow similar IDPs), then there is a chance that they consider the same information as important. So, similarity measure between IDPs is introduced. The similarity measure accounts for information demands, their keywords and structural relations between organization units, performing respective IDPs.

Workers’ profiles. The set of roles assigned to a worker and workers’ efficiency measures that can be used to share the experience of highly professional employees.

Recommendation system output

Recommendation system produces an output in the form of expected importance of each e-mail message in the mailbox. Expected importance is then used to rank e-mail messages presented to the user to make sure that most relevant and actual e-mail messages are placed on the top of e-mails list, attracting most of the worker’s attention.

An important feature of the proposed recommendation system is that there are two principal components affecting e-mail ranking: (a) a set of rules from an IDP description, reflecting the information workflow design; (b) usage-based rules inferred from the practice of information processing by workers. During our research, these components are considered not reducible to one another as they aim different goals: conceptual description of the IDP vs. tailoring this IDP to actual information workflows and informal information processing patterns of an organization. Therefore, the recommendation system must merge e-mail rankings produced by either of these components.

Technological model

XML IDP descriptions

IDP model

IDP usage model

User actions analyzer

E-mail ranking engine

Data storage

Recommendation system

Browse emails

  As it is shown in figure 2, recommendation system functionality comprises four blocks discussed in the rest of the current section.

First of all, entire IDP model is divided into two parts: static (or, IDP model) and dynamic (or, IDP usage model). Static part is enforced by organization information workflow engineering and is composed of human-generated rules about how to classify e-mail to IDP and how to assign importance to e-mails inside some IDP. This part is knowledge-based and pieces of knowledge here are classification rules connecting message attributes (sender), keywords, and optionally facts, extracted from the message by a context-free pattern analysis algorithm (see [ 1, 2 ]), with IDPs and current message importance according to some IDP.

The dynamic part, or IDP usage model, is employed to adjust base IDP model to actual information workflows of the organization. This part also has a form of classifier but it is learned from workers interaction with e-mails by some machinelearning algorithm.

User actions analyzer. E-mail client tracks user actions and passes their descriptions to this component of recommendation system. Each action description includes: action type (removing, flagging, opening an e-mail etc.), IDP instance, user, action time, message browsing context (identifiers of other e-mails that are ranked higher in current users’ browsing context). Actions data are used to estimate current e-mail importance in the context of given IDP. These estimations are saved in recommendation system data storage. Furthermore, user actions and estimated importance are used to build adjusted IDP models by training classifier that predicts IDP from message text and attributes.

E-mail ranking engine. Hybridization of knowledge-based and collaborative approaches. Knowledge-based part is powered by IDP description provided by knowledge worker and a processed form of e-mail message. Collaborative engine looks for similar users and then for each e-mail retrieves estimated importance of similar e-mails from recommendation system data storage. Knowledge-based and collaborative lists are then merged to provide user with resulting message list.

4 Summary and Conclusion

Due to the increasing amount of e-mails received every day and the resulting information overflow, this paper proposes a conceptual architecture for enterprises and organizations to support demand-oriented information supply. Therefore wellestablished information demand patterns are used. Information demand patterns itself are the results from an information demand analysis, which leads to the information demands needed by a worker in an enterprise to accomplish his or her tasks. Information demand patterns are used to feed the proposed e-mail client extension, which enables to provide at the moment needed e-mails to the user by filtering and presenting them in an appropriate manner. Due to the fact that this e-mail client extension is just locally used at the work place of one user, we extended the solution by a recommendation system. The recommendation system monitors users’ behavior like deleting, reordering or ignoring messages and proposes the results to other users with a similar information demand. This might reduce negative consequences of information overload like reduced efficiency, wrong decisions and excessive demands of the employees.

The biggest shortcoming of our approach so far is that is has not been fully implemented and validated in practice. The core elements of the proposed architecture, the recommendation system and the plug-in for the e-mail client including transformation of textual IDP, were both implemented and evaluated in practical application, but separately from each other (see, e.g. [ 17 ] and [ 24 ]). The integration of both into a joint system so far only happened on a conceptual level. From this perspective, we presented work in progress, which has to be continued technically and conceptually.

From a technical perspective, the implementation of the proposed architecture in a collaborative recommendation system with IDP use and e-mail frontend has to be finished. Since the interfaces of both components are well known and suitable for integration, we expect this to cause substantial efforts but no principal problems. The configuration of the system for the actual use in an organization using different IDPs will probably create additional insights regarding the need for further automation. Even for a human actor, to identify overlapping information demands between different IDPs sometimes is not straightforward since the vocabulary used in different IDPs is not necessarily fully adjusted. Automating this mapping might require techniques from text matching and ontology matching [ 22 ].

From a conceptual perspective, we plan to investigate the utility and value of the collaborative recommendation system. For this purpose we need a model how to measure or at least estimate the value and a set-up for performing measurements in everyday practice. Regarding the model for measuring the value, we intend to use our experiences in balanced scorecards [ 23 ] and develop a specific scorecard for this purpose. The set-up for practical evaluation will in the first step probably be a team at a university and the demand of the team members regarding information about education and research activities of the team. Later on, we intend to extend this to an industrial setting.

Acknowledgements

The work presented in this paper was supported within the project KOSMOS (Konstruktion und Organisation eines Studiums in offenen Systemen) funded by the BMBF (Federal Ministry of Education and Research, Germany) and the European Social Funds of the European Union.

The research was also supported partly by projects funded by grants 13-07-00271, 1307-00039, 13-07-12095, 13-07-13159 of the Russian Foundation for Basic Research, project 213 (program 15) of the Presidium of the Russian Academy of Sciences, and project #2.2 of the basic research program “Intelligent information technologies, system analysis and automation” of the Nanotechnology and Information technology Department of the Russian Academy of Sciences. This work was also partially financially supported by Government of Russian Federation, Grant 074-U01.

References

1. Masaru Tomita . Efficient parsing for natural language . Kluwer Academic Publishers, Boston, 1986 .

2. Yandex free tool for fact elicitation , http://api.yandex.ru/tomita/

3. Park , D.H. , Kim , H.K. , Choi , Y.II , Kim , J.K. : A literature review and classification of recommender systems research . Expert Systems with Applications . Vol. 39 , iss. 11 , 10059 - 10072 . ( 2012 )

4. Bobadilla , J. , Ortega , F. , Hernando , A. , Gutiérrez , A. : Recommender systems survey . Knowledge-Based Systems . Vol. 46 , 109 -- 132 . ( 2013 )

5. Burke , R. , Ramezani , M. : Matching Recommendation Technologies and Domains . In: Ricci, F. , Rokach , L. , Shapira , B. , Kantor , P.B. (eds.) Recommender Systems Handbook. Pp. 367 -- 386 . Springer US ( 2011 )

6. Lops , P. , Gemmis , M. de, Semeraro, G.: Content-based Recommender Systems: State of the Art and Trends . In: Ricci, F. , Rokach , L. , Shapira , B. , Kantor , P.B. (eds.) Recommender Systems Handbook. Pp. 73 -- 105 . Springer US ( 2011 )

7. Su , X. , Khoshgoftaar , T.M.: A survey of collaborative filtering techniques . Advances in Artificial Intelligence . Vol. 2009 , art . No. 4 . ( 2009 )

8. Burke , R. : Knowledge-Based Recommender Systems. Encyclopedia of Library and Information Science 69 ( 32 ), 180 - 200 . ( 2000 )

9. Lorenzi , F. , Ricci , F. , Tostes , R. , Brasil , R. : Case-based recommender systems: A unifying view . In: Intelligent Techniques in Web Personalisation, no. 3169. LNCS , pp. 89 -- 113 . Springer ( 2005 )

10. Felfernig , A. , Burke , R. : Constraint-based recommender systems: technologies and research issues . In: Proceedings of the 10th International Conference on Electronic Commerce, ICEC'08 , pp. 1 - 10 . ACM, New York, NY, USA ( 2008 )

11. Edmunds , A ; Morris A ( 2000 ): The Problem of Information Overload in Business Organisations: a Review of the Literature . International Journal of Information Management 20 ( 1 ): 17 - 28 .

12. Sandkuhl , K ( 2008 ) : Information Logistics in Networked Organizations: Selected Concepts and Applications . In: Filipe, J ; Cordeiro, J ; Cardoso, J (Hrsg), 9th International Conference, ICEIS. Madeira.

13. Öhgren , A ; Sandkuhl K ( 2008 ) : Information Overload in Industrial Enterprises - Results of an Empirical Investigation . In: ECIME. London.

14. Speier , C ; Valacich, JS ; Vessey, I ( 1999 ): The Influence of Task Interruption on Individual Decision Making: An Information Overload . Decision Sciences 30 ( 2 ): 337 - 360 .

15. Volnhals , M; Hirsch, B ( 2008 ) : Information Overload und Controlling . Controlling & Management 52 ( 1 ): 50 - 57 .

16. Soucek , C ; Moser, K ( 2010 ) : Coping with information overload in e-mail communication: Evaluation of a training intervention . Computers in Human Behavior 26 ( 6 ): 1458 - 1466 .

17. Stamer , D. ( 2014 ). Reduktion der Informationsüberflutung durch Informationsbedarfsmuster in der E-Mail-Kommunikation . In D. Kundisch, L. Suhl , & L. Beckmann, (pp. 1226 - 1233 ). Presented at the Multikonferenz Wirtschaftsinformatik 2014 , Paderborn.

18. Lundqvist , M; Holmquist, E ; Sandkuhl, K ( 2009 ) : Information Demand Context Modelling for Improved Information Flow: Experiences and Practices . In: IFIP. Aachen.

19. Sandkuhl , K ( 2011 ) : Information Demand Patterns . In: PATTERNS 2011 , The Third International Conferences on Pervasive Patterns and Applications . Rom.

20. Sandkuhl , K ; Stamer, D ( 2012 ) : Towards Improving Structure and Content of Information Demand Patterns . In: Aseeva, N , Babkin, E , Kozyrev, (Hrsg), Perspectives in Business Informatics Research (BIR) . Nizhny Novgorod.

21. Lundqvist , M. ; Holmquist , E. ; Sandkuhl , K. ; Seigerroth , U. & Strandesjö , J. : Information Demand Context Modelling for Improved Information Flow: Experiences and Practices. 2. IFIP Working Conference on Practices of Enterprise Modelling, LNBIP , Springer Verlag. ( 2009 )

22. Feiyu Lin , Kurt Sandkuhl: A Survey of Exploiting WordNet in Ontology Matching . IFIP Artificial Intelligence in Theory and Practice . Springer, 2008 .

23. Kaplan , R. S. , & Norton , D. P. ( 1996 ). The balanced scorecard: translating strategy into action . Harvard Business Press.

24. Smirnov , A. , Kashevnik , A. , Ponomarev , A. , Shilov , N. , Schekotov , M. , & Teslya , N. ( 2013 ). St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St . Petersburg, Russia. In Open Innovations Association (FRUCT), 2013 14th Conference of (pp. 148 - 155 ). IEEE.Sandkuhl, K. ( 2010 ). Validation and Use of Information Demand Patterns in Higher Education (Vol. 57 , pp. 204 - 213 ). Presented at the Business Information Processing , Berlin, Heidelberg: Springer Verlag.

25. Sandkuhl , K. ( 2011 ). Improving Engineering Change Management with Information Demand Patterns . Presented at the 8th International Conference on Poduct Lifecycle Management.