Design of Search Interfaces for Mathematicians

Abstract

The availability of mathematical knowledge on the Web changes mathematical practices. In a study of professional mathematicians' search behavior we identi ed ten behavioral patterns particular to mathematicians. In this paper, we present a set of design goals for mathematical search interfaces implied by these patterns and insights derived from them.

Introduction

The availability of mathematical knowledge on the Web is constantly increasing. Research especially in the Mathematical Knowledge (MKM) and Digital Mathematical Libraries (DML) communities have looked into the representational and organizational aspects of this. In the Mathematical User Interfaces (MathUI) workshop we are especially interested in the aspects of use and handling of this fast growing mathematical web knowledge. In this paper we are especially interested in improving search for math. For that, we might look at general resources like [ 10 ] or [ 2 ]. But even a complete redesign of e.g. the reviewing service Zentralblatt Mathematik (zbMath.org) according to the newest best practices didn't result in a jump in usage levels. We suspected that the issues lie deeper as mathematicians have speci c, idiosyncratic needs and thus require distinct design goals.

To assess the usability of the European Digital Math Library in [ 4 ] Kitchen conducted interviews with (professional) mathematicians | mathematicians that not only have degree(s) in mathematics, but also are actively pursuing mathematical research. It provided the insight that mathematicians didn't know how to use the o erings of mathematical search interfaces. The reason can either reside in the content, i.e., not o ering resources interesting to mathematicians, or in the form, i.e., o ering valuable resources in not suitable ways. In [ 16 ] Zhao analyzes user-centric and math-aware requirements for math search. He lists \general keyword search, browsing math-speci c resources and personal contact " as main math search activities. This doesn't yet answer the general question, whether there is a need for a speci c math-oriented design for search interfaces.

To nd out what distinguishes mathematicians from non-mathematicians, we interviewed professional mathematicians and other people interested in math to nd idiosyncratic, but typical behavioral approaches, expectations or understandings with respect to mathematical search interfaces (reported in [ 5 ]). We elicited 10 patterns that give us insights about the special user group of professional mathematicians.

In this paper, we want to turn to the design aspect of these 10 patterns. We will argue for some principles that don't seem to have been taken into account in current designs of math software systems. Note that you should not expect a set of handy statements of the kind \Do this, but not that" as the underlying power of design consists in the fair balance of concurrent and partially contradictive requirements. The formulation of math search speci c design goals is consequently done rather cautiously.

In Human-Computer Interaction (HCI) it is common practice to establish a set of \best practices" for the design of user interfaces, which are directed towards a general user pro le as a human being. Therefore, they hold for professional mathematicians as well. Note that we cannot provide best practices for designers of math UIs, as we haven't conducted any usability study with a system that adopted our principles yet. The presented design goals are supposed to serve as a basis for discussions about future math user interface design. 2

Ten Patterns of Mathematical Approach to mSIs

In [ 5 ] we used the set of exemplary math search interfaces (\mSI") and established the following 10 patterns of mathematicians when approaching math search interfaces:

Pattern 1 (\Familiarity"): \Mathematicians do not assess mSIs based on familiarity." Pattern 2 (\Community"): \Mathematicians trust human and community resources." Pattern 3 (\Finding"): \Finding is the primary mathematical search task." Pattern 4 (\Social Interaction as Tool"): \Mathematicians appreciate social interaction as a mathematical tool. In particular, it is a mathematical practice to collaborate and exchange feedback." Pattern 5 (\Medium"): \Mathematicians aim at adopting a search tool as a medium." Pattern 6 (\Function"): \Mathematicians appreciate function over form." Pattern 7 (\Outcome"): \Mathematicians care more for the outcome than the input." Pattern 8 (\Empowerment"): \Mathematicians want to be empowered in the search process." Pattern 9 (\Transparency"): \Mathematicians base their information search process on transparency of the search result." Pattern 10 (\Expectations"): \Mathematicians expect to nd meaningful information in the search result."

Design Implications for mSIs

The ISO standard for Ergonomics of Human System Interaction ISO 9241-11 (1998) sets usability to be the \extent to which a product can be used by specied users to achieve speci ed goals with e ectiveness, e ciency and satisfaction in a speci ed context of use". The elicited patterns didn't even mention the keywords \e ectiveness", \e ciency" or \satisfaction" once. The underlying reason is not the author-speci c phrasing of the patterns in [ 5 ] but rather the disregard the mathematicians showed for these concerns in the common form.

This might be just as well as usability has lost its status of being the criterion for a well-designed product in modern HCI. There, the new criterion is user experience (UX), which comprises according to ISO 9241-210 (2009) \a person's perceptions and responses that result from the use or anticipated use of a product, system or service". This replacement seems much more tting, if we recall that the math-speci c properties of the above mSIs were cooked down in [ 5 ] to evaluating the \preconditions of the search", the \precision of the input" and the \relevance of results driven by either user or data". The expectations concerning the input and the anticipated meaningfulness of the output clearly indicate an holistic approach by mathematicians.

According to [ 1 ] user experience consists of four components: value, desirability, adoptability, and usability. Therefore, we will analyze the patterns in the following with respect to these four components. 3.1

Value

The value aspect of UX concerns the question of usefulness.

Finding As pointed out in [ 5 ] the term \search" comprises numerous di erent search processes. In particular, it entails for example: { Finding = already knowing what one is looking for { Browsing = getting an overview over a topic or an idea of a concept { Sur ng = surrendering to the links, drifting from one to another { Solving/Information Gathering = creating a search plan, i.e., specifying a sequence of actions that achieves the solution of a problem { Asking = posing a question to nd an answer In di erent phases (like selection, formulation and collection) a user seeking information translates a search intension into a query or series of queries optimizing for the relevance of the nal collection of search results (see [ 8 ]).

The value aimed for by mathematicians according to Pattern 3 (\Finding") consists in the satisfaction of the search intension. It can only be redeemed, if the entire search environment is streamlined towards nding.

The presumptions for such a streamlining Shneiderman et al. summarized in [ 14 ] under the formulation phase with the following distinct aspects: a.) Sources: How large is the underlying database? b.) Fields: How structured is the underlying database? c.) What to search for: How expressive is the input language? d.) Variants: How big is the error tolerance? Additionally, in the collection phase they point out the following aspects: e.) Entropy: How much information is conveyed via the presentation of the search results? f.) Correctness: How good is the precision and recall according to the search intension? g.) Re nement: How easy is a re nement of the search results (towards the search intension)? To optimize the nding process, thus, several handles can be used: Goal 1 (\Optimization of Resources"): \Optimization of sources, elds, query language, presentation of variants, low entropy, and re nement features | with respect to the search intension " Expectations The Pattern 10 (\Expectations") from [ 5 ] stresses the importance of the collection phase. If the precision of the information retrieval is not high, then mathematicians need tools to turn this data into meaningful information, e.g., for aligning their subsequent search plans.

Goal 2 (\Structure"): \Maximization of structure in the search results " Note that mathematicians do value function over form (Pattern 6 (\Function")), so a complex presentation (maybe causing cognitive overload) is allowed, as long as it provides value. In [ 9 ] Libbrecht posed an interesting question: Is there something like a \too high precision" of (mathematical) search queries with the e ect that search results become too small? Considering Pattern 10 (\Expectations") we can assume that mathematicians didn't plan for low recall with their search query. Goal 2 (\Structure") therefore should include a query extension or netuning mechanism like the one presented in [ 9 ]. There, a bad result gives, e.g., hints about query variants.

Function Obviously, Pattern 6 (\Function") strongly supports the need of mathematicians for value, as they evaluate form less than function. Even though the di erence in layout in the set of mSIs is perceived by most (notes taken in interviews in [ 5 ] con rmed that), it hasn't found its way to the assessment schemata of mSIs, therefore can be deemed irrelevant for mathematicians. Empowerment But the question arises whether mathematicians appreciate functionalities independent of their search intension? In particular, is there something like too much functionality for mathematicians, where too much value turns into less value? Pattern 8 (\Empowerment") suggests that this is not the case, as they want to have all the world at their ngertips and decide autonomously what to do with it in their search strategy. The side condition \serving their search strategy" should be considered nevertheless. Goal 3 (\Flexible Search Strategies"): \Addition of as many functionalities to support exible search strategies as possible " Transparency The value of a mSI for mathematicians also consists in a correct mental model of its inner workings. Mathematicians do not appreciate black boxes, they want to evaluate themselves how to make use of a service. For that (according to Pattern 9 (\Transparency")) they need to be able to assess its value for them, e.g., when working out a search plan to ful ll a search intension. Therefore it might be helpful if math designers adopted Goal 4 (\Semantic Transparency"): \Semantic transparency, i.e., if the user interface allows access to the relevant information and its underlying semantic relations [ 6 ] " In combination with Pattern 10 (\Expectations") we might even venture to say that only a not understood, that is, unpredictable search result is a bad one. Such a search result breaks the original search plan as it does not allow a re nement of the search process.

Social Interaction as a Tool Mathematical literacy is an ability that computers as of now do not possess, but mathematicians do. The more they have dealt with mathematics the stronger their literacy becomes (see [ 7 ]). Not only re ection, but also communication within the center of the mathematical Community of Practice (CoP) gets so much more e cient with this growing literacy. This might be the underlying reason why we found in [ 5 ] that mathematicians could approach social interaction within their CoP as if it were a mere search tool for mathematics. The value of social interaction is not about establishing social bindings or networking, it is about e ciency. Therefore, we suggest: Goal 5 (\No Social Media"): \No inclusion of social media in an mSI " Note that we refer to social media with a focus on establishing social networks. A math search interface like mathoverflow (mathoverflow.net) shows how a service can successfully introduce social interaction in accordance with Pattern (\Social Interaction as a Tool").

Community mathoverflow draws on Pattern 2 (\Community") as well. Goal 6 (\Community Idiosyncracies"): \Integration of community aspects " Recall, for instance, the following mathematical community aspects: polite and distinguished feedback, anecdotes, regular e.g. small community meetings in Oberwolfach, etc. 3.2

Desirability

To understand the desirability of an interactive product we ask whether it is fun and engaging. But what exactly does this mean for mathematicians? The disgust for playfulness shown in the study in [ 5 ] by them indicates that it is not about the engagement factor in the normal sense. Entertainment and gaming do not come into the fun experience of mathematicians when searching for math. But maybe the beauty of Math does? In [ 15 ] there is even some scienti c evidence that mathematicians desire elegance. What concretely does for example the following statement mean for design? \Mathematics, rightly viewed, possesses not only truth, but supreme beauty a beauty cold and austere, [...], sublimely pure, and capable of a stern perfection such as only the greatest art can show." [ 13 ]. We can derive an interpretation of \desirability" for mathematicians: Goal 7 (\Beauty"): \Utmost reduction of complexity | without losing functionality "

One best practice of a well-known set of usability guidelines [ 12 ] points in the same direction. It states that \any extra unit of information in a dialogue competes with the relevant units of information" and therefore a minimalist design is to be preferred (see also [ 11 ]). For a search interface Google's layout is not only a prototypical example, it has even reached an idol status, the Google search design factor, which 'has' to be aimed for. In particular, Google achieves this (see [ 3 ]) based on the maximes of an ethics of simplicity, rewarding openness and drawing on the e ects of the long tail. In [ 5 ] we have explained that the redesign of zbMathNew interface towards a Google-like design does not need to be a successful move with mathematicians. Reduction of form is good, but the impression of loosing functionality destroys the basic idea.

Transparency Here, the Pattern 9 (\Transparency") kicks in. For instance, in the zbMathNew example above the new Google-like design did not disable functionality, it merely neglected to point it out to the user. That is, the redesign was not transparently carried out. The desirability of a math search interface is improved when the interface is transparent enough, so that search results can be anticipated. Pattern 7 (\Outcome") is also about the ability of looking-ahead. Therefore, we note that a rather coy but with a lot of potential design might be desirable for mathematicians: Goal 8 (\Cinderella"): \Di dent design with the potential to dig deep" The metaphor \Cinderella" is based on Cinderella's di dent attitude and appearance contrasting her brilliant heritage and future. Empowerment To be able to showcase mastery of a tool is the more impressive the more the tool empowers one to do. Here, mathematicians are no exception, they like to be experts. Thus, Goal 2 (\Structure") and Goal 3 (\Flexible Search Strategies") support fun and engagement, that is, desirability.

Community Note that the desirability is strenghtened with Goal 6 (\Community Idiosyncracies") as well. 3.3

Adoptability

Familiarity It is an HCI best practice to bridge the gap between new and old services by extending old and therefore known interaction rules into the new ones. In particular, the old service can usually serve as a metaphor for the use of the new service. E-readers, for instance, do use the good old book metaphor to be handled.

Pattern 1 (\Familiarity") breaks this best practice. It seems to suggest that professional mathematicians, from here on abbreviated to \mathematician" only, can be used as guinea pigs for whatever new designs one can come up with as long as some basic usability guidelines are followed. But in fact, it is quite the opposite. Even though they are open to new designs in general, they have to be convinced in the rst place to adopt the new service. These criteria seem to be much higher than with non-math services as some of the other patterns like Pattern 6 (\Function") or Pattern 5 (\Medium") are not really optional. Community One reason for adoption of an application consists in the added value o ered. As the community is highly appreciated by mathematicians, Goal 6 (\Community Idiosyncracies") supports the adoptability. 3.4

Usability

Outcome One main usability goal of applications is the e ciency of its user interface. But for mathematicians the outcome is more important than the input according to Pattern 7 (\Outcome"). In particular, e ciency is not a main goal for them. Other main usability goals are learnability and memorability, which didn't come up in the interviews in [ 5 ] at all. Yet another usability goal consists in the fun factor of using an application. We discussed this in regard to desirability already and decided that fun has to be reinterpreted for mathematicians. So, all in all, we can state that the usability criteria for mathematicians have to be rede ned. The suggested goals in this paper lead towards such a rede nition. 4

Conclusion

We have presented and argued for design implications based on the patterns elicited in [ 5 ]. In particular, we have: G1 (Optimization of Resources): \Optimization of sources, elds, query language, presentation of variants, low entropy, and re nement features | wrt search intension" G2 (Structure): \Maximization of structure in the search results " G3 (Flexible Search Strategies): \Addition of as many functionalities to support exible search strategies as possible" G4 (Semantic Transparency): \Semantic transparency, i.e., if the user interface allows access to the relevant information and its underlying semantic relations [ 6 ] " G5 (No Social Media): \No inclusion of social media in an mSI " G6 (Community Idiosyncracies): \Integration of community aspects " G7 (Beauty): \Utmost reduction of complexity | without loosing functionality " G8 (Cinderella): \Di dent design with the potential to dig deep" Future work now entails the implementation of these design suggestions in, for example, Mathematical Knowledge Management applications. Once this is done, the value of the goals can be veri ed with usability studies. If so, then we can establish the above set of design goals as best practices for mathematical search interfaces.

Note that this study didn't prove common usability goals invalid, but when considering a fair balance of essential requirements for a good design of math (search) interfaces, the weights should be speci cally assigned according to mathematicians. If those requirements are satis ed, the netuning regarding general best practices concerning form but not content can begin | if there are resources left. Mathematicians will be the happier.

Acknowledgement This work has been funded by the Leibniz association under grant SAW-2012-FIZ.

References

[1] Frank Guo. More Than Usability: The Four Elements of User Experience . Available at http://www.uxmatters.com. Accessed on 2014-04-01 . 2012 .

[2] Marti A. Hearst . Search User Interfaces . 1st . New York, NY, USA: Cambridge University Press, 2009 . isbn: 0521113792 , 9780521113793 .

[3] Je Jarvis. What Would Google Do? HarperCollins Publishers Limited , 2009 . isbn: 9780007312108 .

[4] Tim Kitchen . The European Digital Mathematics Library: Usability Study . url: https://wiki.eudml.eu/eudml- w/images/D6.1.pdf (visited on 03/14/ 2014 ).

[5] Andrea Kohlhase . \Search Interfaces for Mathematicians" . In: Intelligent Computer Mathematics 2014. Conferences on Intelligent Computer Mathematics (Coimbra, Portugal, July 7 { 11 , 2014 ). Ed. by Stephan Watt et al. Lecture Notes in Computer Science . accepted. Springer, 2014 . url: http: //arxiv.org/abs/1405.3758. Forthcoming .

[6] Andrea Kohlhase and Michael Kohlhase . \ Semantic Transparency in User Assistance Systems" . In: Proceedings of the 27th annual ACM international conference on Design of communication (SIGDOC) . (Bloomington, Indiana , USA, 2009 ). Ed. by Brad Mehlenbacher et al . ACM Special Interest Group for Design of Communication. New York, NY, USA: ACM Press, 2009 , pp. 89 { 96 . doi: 10 . 1145 / 1621995 . 1622013. url: http : //kwarc.info/kohlhase/papers/sigdoc09-semtrans.pdf.

[7] Michael Kohlhase . \ Mathematical Knowledge Management: Transcending the One-Brain-Barrier with Theory Graphs" . In: EMS Newsletter ( 2014 ). in press. url: http://kwarc.info/kohlhase/papers/ems14.pdf.

[8] Carol C. Kuhlthau , Jannica Heinstrom, and Ross J. Todd. \ The information search process revisited: is the model still useful?" In: ir information research 13 .4 ( 2008 ).

[9] Paul Libbrecht . \ Escaping the Trap of Too Precise Topic Queries" . In: MKM/Calculemus/DML. (Bath, UK , July 8 { 12 , 2013 ). Ed. by Christoph Lange et al. CEUR Workshop Proceedings 1010. Aachen , 2013 , pp. 296 { 309 . url: http://ceur-ws. org/ Vol- 1010 .

[10] P. Morville and J. Callender . Search Patterns: Design for Discovery. O'Reilly Media , 2010 . isbn: 9781449383060 .

[11] Jakob Nielsen . Designing Web Usability : The Practice of Simplicity . New Riders Press, 1999 . isbn: 1 - 56205 -810-X.

[12] Jakob Nielsen and Rolf Molich . \ Heuristic Evaluation of User Interfaces" . In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '90 . Seattle, Washington, USA: ACM, 1990 , pp. 249 { 256 . isbn: 0 - 201 -50932-6. doi: 10 .1145/97243.97281. url: http://doi.acm. org/10 .1145/97243.97281.

[13] Bertrand Russell . Mysticism and Logic. Dover Books on Western Philosophy. Dover Publications , 2004 . isbn: 9780486434407 .

[14] Ben Shneiderman , Don Byrd, and W. B Croft. \Clarifying Search: A UserInterface Framework for Text Searches" . In: ( 1997 ). issn: 1082 - 9873 .

[15] Semir Zeki et al. \ The Experience of Mathematical Beauty and its Neural Correlates" . In: Frontiers in Human Neuroscience 8 .68 ( 2014 ). issn: 1662 { 5161 .

[16] Jin Zhao , Min-Yen Kan , and Yin Leng Theng. \Math Information Retrieval: User Requirements and Prototype Implementation" . In: Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries. JCDL '08 . Pittsburgh PA , PA, USA: ACM, 2008 , pp. 187 { 196 . isbn: 978 - 1- 59593 -998-2.