Computer and mobile device screens are increasing in size and resolution. This increase in resolution causes problems with graphical user interfaces designed for lower resolution screens, as all information gets smaller and smaller. We present two novel techniques to make graphical user interfaces on high-resolutions screens more accessible and usable. We introduce a new in-place, localized zooming technique that works on a per widget basis. We also present a novel widget magnification technique that implements special modalities for common user interface elements, which affords widget-dependent magnification.
Computer monitors are getting bigger and bigger and resolutions, i.e. the number of pixels, continue to increase. This trend exists both on handheld devices, such as smartphones, as well as traditional laptops and desktops. In general, bigger displays are preferred as users can display more information and see more detail simultaneously. More relevant to our current work is that the pixel density of screens, i.e. the number of pixels per inch (ppi), is also increasing steadily. Currently smart phones and portable book readers use screens with 150-300 ppi, laptops 100250 ppi, and desktops 100-200 ppi.
However, most graphical user interfaces (GUI’s) are designed for monitors with a constant pixel density, typically about 100 ppi, the lower end of today’s displays. This makes all user interface elements, including text, appear (too) small on displays with higher densities. This causes problems for users with poor eyesight, but affects even average people when they upgrade their system. In many common GUI toolkits there is little or no support for scaling of widgets and their elements, such as images or text, to resize or render them differently depending on the resolution. Partial fixes to this problem exist in the form of a global, system wide scale factor or full screen pixel-wise zoom. However, this reduces either the information content per window, or provides less screen space for other programs. This quickly becomes a problem if the user is working with more than one application at a time. The zoom/pan facilities associated with the “canvas” widget used in browsers, text editors and drawing programs etc., address this problem for that type of widget, but this solution does not generalize. Reworking existing GUI toolkits to provide scalable widgets is labour intensive and time consuming and will likely require significant adaptation in existing applications. Hence, methods are needed to enhance the readability of existing GUI’s without reducing content.
In this article, we present a novel technique to make user
interface widgets more scalable for different screen resolutions
A GUI consists of many densely tiled targets, i.e. widgets. In such
arrangements, the user needs to be able to quickly select and
easily interact with any given widget. For this situation,
dynamically expanding targets have been proposed based on their
implications on human performance [
Another area of relevant work is interface customization, i.e.
adaptive and user-adaptable interfaces. Among this large body of
work, Façades is particularly relevant [
For this localized widget zoom, we present two new techniques using pixel magnification and widget replacement. Both use the accessibility API of modern toolkits to retrieve information about widgets, such as position and/or interaction possibilities.
In this version of a WidgetLens, the pixel image of the current widget is used as a texture for an enlarged version, which is overlaid on top. The widget in focus is thus enlarged in each direction (by default a factor of two). We move this enlarged version proportional to the distance of the corresponding edge of the original widget from the cursor, which provides a sliding effect. Mouse motions are mapped to guarantee that as soon as the mouse cursor leaves the original widget, the local widget zoom switches to the corresponding next widget. Any mouse or keyboard interaction with the fully zoomed version is passed through to the original widget.
As neighbouring widgets may be hidden by the expanded widget, we generate partially enlarged versions and overlay them at the respective corner closest to the cursor in an intermediate layer, which increases the visibility of these surrounding widgets. Figure 3 depicts the local widget zoom. 3.2
Straightforward magnification causes an undesirable blur in text and images. To address this, we present a novel form of widget lens that replaces widgets with higher resolution versions. This is implemented by traversing the widget tree of the underlying application and by duplicating each widget on a one-to-one basis in a separate window. The replication then uses larger fonts. If the system recognizes also the filename of an icon, it also uses a higher-resolution version of the corresponding image. Then, each of these enlarged versions is instantiated as a façade and overlaid on the original window as described in the local widget zoom (Figure 4). For simple widgets, such as buttons or menus, all interactions, such as mouse clicks are simply redirected to the original widget. The display mechanism, mouse events, and partially obscured widgets are handled as described above.
More complex widgets, such as combo-boxes or text fields, are managed differently. In this situation there are more interaction possibilities, such as selection, scrolling, or editing. We address this by allowing the user to interact directly with the overlaid widget and pass the resulting higher-level events, such as text changes or selection events, to the original widget. Another issue is that simple magnification may cause parts of the widget to fall outside of the monitor. Hence, we limit the number of lines, list items, and/or the number of characters to make sure that the widget stays within the boundaries of the monitor. We currently do not provide WidgetLens functionality for canvases. A naïve magnification would only lead to an explosion of white space. Moreover, we can assume that the application already provides zoom facilities for a canvas widget.
We presented a new method for end-users to interact with user interface elements on screens with high pixel densities. Based on the Façades system, we implemented two new methods that increase the visual accessibility and interactivity of GUI widgets.
We are working to implement various other effects, such as high-contrast display of widgets. Moreover, we are currently improving the animations to make the zoom effect less noticeable and dependent on cursor speed. Also, we are expanding the set of smart WidgetLenses further. Finally, we are considering a user study to analyze the benefits of WidgetLenses.