=Paper=
{{Paper
|id=Vol-2811/Paper03
|storemode=property
|title=Dynamic Generation and Exploration of Symmetrical Textures from Real-time Camera Input on Mobile Devices
|pdfUrl=https://ceur-ws.org/Vol-2811/Paper03.pdf
|volume=Vol-2811
|authors=Nikitas M. Sgouros
}}
==Dynamic Generation and Exploration of Symmetrical Textures from Real-time Camera Input on Mobile Devices==
https://ceur-ws.org/Vol-2811/Paper03.pdf
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
DYNAMIC GENERATION AND EXPLORATION
OF SYMMETRICAL TEXTURES
FROM REAL-TIME CAMERA INPUT
ON MOBILE DEVICES
Nikitas M. Sgouros
Professor, Dept. of Digital Systems University of Piraeus, Greece
sgouros@unipi.gr
Abstract: Symmetrical patterns play significant roles on the development of the
aesthetics of various art forms. The wide use of such patterns in modern design creates
the need for digital tools that can facilitate their creation, experimentation and use by
artists and designers. Responding to this need, we describe Symmetricon, our Android
art/design app for mobile devices that supports real-time visual exploration of a
multitude of symmetry types. In particular, we explore animated texture design based
on the application of plane symmetry groups on real-time camera input and projected on
various type of surfaces. Our art/design app is freely available on the Google Play store.
We provide example videos for its use and possible application on YouTube.
Keywords: Geometric design, symmetry, digital authoring tools, mobile computing.
Introduction
Symmetrical patterns have been a constant source of inspiration for various visual art
forms by evoking feelings of order, harmony or balance [1]. Furthermore, the existence
of detailed mathematical descriptions for generating all possible symmetries on the
plane has opened up the possibility of efficient, algorithmic generation of such patterns.
However, the kinds of iconography that can be generated from symmetry is vast and can
be made even bigger by combining symmetrical forms with color or motion animation
or by introducing noise in such patterns in a principled way. In this case, the important
question that concerns us is how we can efficiently help and motivate digital artists to
create and experiment with symmetrical designs that are visually interesting and
aesthetically pleasing. Our efforts in dealing with this question focus on the creation of
digital environments for creative visual exploration of symmetrical patterns and their
interplay with geometry and animation. We want to create tools that can be easily
deployed by artists and designers, therefore we target mobile devices as our main
development platform. To this end we have created and deployed Symmetricon [2], a
free Android app for generating, animating and displaying, in real-time, plane symmetry
patterns. In the rest of this paper, section 2 gives a brief description of the types of
Copyright © 2018 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0) DCAC 2018.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
symmetry generated by Symmetricon, while section 3 explains the algorithms used for
the efficient creation of such patterns in real time. Section 4 describes the use of the
system and provides some examples of the designs that are possible with it. Finally,
section 5 is a conclusions and future work section.
Symmetry
We think of symmetry as a system for organizing design components, referred to as
tiles or cells, that are repeated in space. The repeated filling of space with tiles creates
tessellations, i.e., complete coverings of the space without any gaps or overlaps. When
we restrict ourselves to the two- dimensional space there are 17 ways by which cells can
be combined to generate tessellations. We refer to them as the 17 wallpaper groups [3].
The basis of symmetrical surface design is to have a cell that orients in the same
direction and repeats with no gaps or overlaps. Therefore, only certain shapes can be
used as primary cells (i.e. the shapes that will be combined to form the final repeating
cell). These are the triangles, quadrilaterals (all planar shapes with four sides) and
the hexagon. In the case of quadrilaterals, if opposite sides are not equal in length or
parallel to each other, then the tessellation process becomes more involved with various
translation and rotations applied to each one of them in order to fit seamlessly in the
tessellation. Triangles must also be combined with one of them turned upside down to
form a quadrilateral cell.
All final repeating tiles are generated by four motions of the primary cell (see Fig.
1); translation, rotation, reflection and glide. Translation moves the primary cell
sideways, horizontally, vertically or diagonally without a change in its orientation.
Rotation turns the cell a certain number of degrees either around its geometrical center
or around one of its vertices. Reflection flips the cell either horizontally or vertically.
Finally, glide is a combination of a translation and a reflection. In this case, the cell can
either be translated horizontally/vertically or then flipped vertically/horizontally.
All 17 possible symmetry groups on the plane are labeled with a notation system
stemming from the one used by crystallographers to classify crystals but restricted to
the plane [5]. The notation describes the combination of all the operations that are
performed on a primary cell (translation, rotation, mirror, glide) to create a final tile that
repeats through horizontal and/or vertical translation to produce the tessellation.
Symmetricon uses this standard notation. It works as follows [6]:
• The first letter in all the groups is either a p or a c. The p stands for the primary cell,
the c stands for centered cell which is akin to a primary cell.
• A number in the second position indicates the maximum order of rotation that is
done in the symmetry. The center of rotation is determined by the letters that follow
the number. An m directly following the number means that the cell must be
mirrored first and then rotated along the angle formed by the mirror. If m comes in
the fourth position after another letter, the primary cell must first be rotated and
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
then the mirror applied to the result. For example, p2 means the cell is rotated two
times (180°); p3 means it is rotated three times (120°); p4 means the cell is rotated
four times (90°); and p6 means it is rotated six times (60°) to produce the final
repeating cell. If there is no number in the label, it means that no rotation is applied
during the creation of the final tile.
• The third letter (or second if there is no number in the label) indicates the first
operation that is done to one of the sides of the primary cell (m = mirror, g = glide,
1 = no operation prior to rotation).
• The fourth position (or third if there is no number in the second position of the
label) is also a letter and indicates what is done to an adjacent side of the cell.
For example, p4g means that the final cell consists of the primary cell rotated four times
and then the resulting cell is glided, while pgg means that the primary cell is glided both
horizontally and vertically.
Generation of Symmetrical Textures
The symmetrical texture generation process in Symmetricon receives as input the
frames captured from the camera stream of a mobile phone. These frames comprise the
primary cells for the subsequent operations. Symmetricon then generates the final
repeating tile which is either a rectangle or a hexagon depending on the symmetry group
specified by the user. This tile is a bitmap image which is passed as a texture unit to the
fragment shader of an OpenGL environment. This is straightforward in the case of
rectangular tiles. In the case of hexagon tiles, given that textures are rectangular in
OpenGL, the hexagon tile is turned into a rectangle by adding the top left triangle to the
bottom right side of the hexagon and so on. The rest of the operations consisting of all
the motions (translations, rotations, reflections and glides) applied to the repeating cell
in order to create the tessellation are being performed in the fragment shader in parallel
thus speeding the tessellating process and achieving real-time performance. In parallel
with the generation of the tessellation the fragment shader computes the animation
pattern specified by the user in the GPU as well. This can either be a horizontal/vertical
translation of adjacent rows/columns of the tessellation, a rotation of a 4x4 tile block
around its geometrical center or non-linear vortex and tunnel effects.
The resulting tessellation is being pasted upon either a square surface, a sphere surface,
a square grid (i.e., a square surface sampled with a grid of points) or a combination of
the above depending on the user choice. These surfaces are created in the vertex shader
of the OpenGL environment and are centered on a 3D coordinate system. The user can
select a variety of animations that affect the position of the vertices of each surface
displayed thus creating various wave and other geometrical deformation special effects.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
Figure 1: Examples of (a) horizontal translation, (b) rotation about the point of intersection of the diagonals,
(c) vertical reflection, and, (d) glide. The black line in (c) and (d) depicts the axis of reflection.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
User Interface (UI)
Symmetricon allows the user to select the types of symmetry groups he wants to use, the
size of the tessellation along with the specific texturing surface (square, sphere or rugged
plane) he wants to create and the type of animation he wants to employ on the pattern
(none, horizontal, vertical, cyclical, vanishing, tunnel or pulsating effects). Based on
these user specifications, the system creates a tessellation of the desired symmetrical
pattern and embeds the resulting texture in a 3D scene. The user can then view the
generated texture from any distance or viewing angle in this 3D space. He can also store
the textures he finds interesting as images in his phone. Given that Symmetricon is a
mobile app, we have sought to minimize all UI elements so that we can maximize the
display of the tessellation on the device screen. The app window consists of a toolbar at
the top that contains all the user interaction options and the main screen for displaying
the tessellations (see [7] for a YouTube video depicting the system in use).
Toolbar
The toolbar contains a series of menu items for interacting with the app:
• the Symmetry option displays a menu of the 30 possible symmetry types supported
by the system. Selecting any of the symmetries in this menu results in the application
of this patricular symmetry on the app input. The names for each symmetry type
correspond to their crystallographical labeling. The complexity of generating each
symmetry group is not the same, therefore some groups (e.g. P6) may take
more time to be displayed than others.
• the Repeats option determines the size of the tessellation. The size is the number of
times that the cell will repeat in the horizontal and vertical directions. The current
version of Symmetricon creates only square tessellations in which the number of
repetitions is the same horizontally and vertically.
• the Input option opens a menu for selecting the input for creating the primary cell.
Possible inputs are the front/back camera or a user-supplied image/video.
• the Animation option opens a menu for selecting the animation pattern that will be
applied
• to the tessellation. Possible animation patterns include sliding of the tessellation
rows or columns along with cyclical motions of adjacent cells. Furthermore, we
offer versions of the vanishing, tunnel and vortex effects.
• the Surface option opens a menu for selecting the surface on which the resulting
image will be textured. Possible options include a square (default), a sphere, a
combination of both or a rugged square surface.
• the Take Snapshot option allows the user to take a jpeg snapshot of the screen
contents and store the resulting image.
• the Set/Exit full screen options allow the app to use all available space in the
device screen.
• the Help option displays a help dialog.
Main Screen
The main screen allows the user to change the angle and distance from which he
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
observes the pattern by dragging gestures left/right and up/down. In each of these cases
the textured surface always appears at the center of the screen. Using pinching gestures
the user can zoom in or out of the scene.
Examples
Symmetricon provides the artist/designer with the opportunity to generate and test
various motifs under an unlimited set of continuous and constantly changing imaging
conditions. This enables the app to be used as a visual exploration environment that
allows its users to create infinite color and geometrical variations of symmetrical
textures using as raw material real-world scenes captured by the camera of a mobile
device. In this case, the artist/designer can control the camera position and motion of the
mobile device during the capturing of the raw material while also applying on it various
animation patterns and projecting it on various surfaces. A video showing a selection of
patterns created by an extended version of Symmetricon while randomly scanning a
typical office space with the camera of the mobile device can be found in [4]. The
particular app version used in the video is able to project the textures simultaneously on
an unlimited set of texturing surfaces where each of these surfaces is specified by the
user using 3D mathematical functions. Figure 2 provides some more examples of the
possible tessellations that Symmetricon can create. The images used for the creation of
these patterns are the frames captured from the camera stream of a mobile phone.
Figure 2: Three examples of tessellations generated by Symmetricon.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
Conclusions & Future Work
We have described Symmetricon, our art/design app for creative exploration of planar
symmetrical textures and their interaction with various surface geometries and animation
patterns. Symmetricon is able to create 30 types of symmetries in real time from either
the camera stream, or images and videos stored in a mobile device.
There exist a variety of mobile apps that focus on the generation of symmetrical
patterns (e.g. Kaleidoscope Pro, Picasso, etc.). Most of them are drawing apps that allow
the user to create the primary cell and then generate mainly kaleidoscope tessellations
based on it. To the best of our knowledge Symmetricon is the only existing app that can
generate animated, symmetrical tessellations from real-time camera stream and apply
them as textures to various surfaces.
Our future research efforts will focus on speeding up further the tessellation
process by parallelizing the primary cell generation which currently is executed on the
CPU. In addition, we are exploring the dynamic generation of symmetrical patterns and
the iconography they produce in their use in non-Euclidean geometries and in mobile
virtual reality environments. Furthermore, we are pursuing collaborations with visual
artists for the use of this technology in common projects along with the development of
novel methods for the integration of such camera-based symmetrical texturing in virtual
and/or mixed reality environments.
Acknowledgements
Presentation of this work was partially supported by the University of Piraeus
Research Center.
References
Beyer, J. (1999). Designing Tessellations: The Secrets of Interlocking Patterns. McGraw-
Hill. https://www.youtube.com/watch?v=Sseob7zSNDU&feature=youtu.be
Farris, H. A. (2015). Creating Symmetry: The Artful Mathematics of Wallpaper Patterns.
Princeton University Press.
Kappraff, J. (1991). Connections: The Geometric Bridge between Art and Science (2nd
edition). World Scientific Publishing Company. https://play.google.com/store/
apps/details?id=org.epinoetic.symmetricon
Schattschneider, D., (1978). The plane symmetry groups: Their recognition and
notation, The American Mathematical Monthly, 85 (6): 439–450.