Oil Painting Rendering Changeable by Light Effect Sungkuk Chun* Keechul Jung† HCI Lab., Soongsil University effect generates the oil painting-like image as an output of our ABSTRACT system. Traditional oil painting works enable the spectators to feel the We give a detailed description about our algorithm in Section 2. various impressions because it can be shown differently by the The experimental results are illustrated in Section 3. We then changes of light effect. The reason of oil painting’s distinguishing conclude the paper in Section 4. feature is that it contains the texture and volume of color 2 PROPOSED SYSTEM expressed by thickness of used pigments and brushing. In this paper, we present a novel method that reproduces oil painting-like This paper proposes a non-photorealistic rendering system to image from a source picture based on a virtual light and non- create an oil painting-like image from an input picture. The photorealistic rendering technique. To generate the oil painting- system consists of three modules, stroke distribution, painterly like image as an output, the system first performs stroke rendering, and intermediate image transformation. Figure 1 shows distribution, which is to determine where a brush is located for the process of proposed system. stroking, using edge detection and image segmentation on an input picture. And the intermediate image is constructed with the suitable color, orientation and size of brush at each stroke point. At last, the system applies light effect to the intermediate image and generates the oil painting-like image. KEYWORDS: Non-photorealistic Rendering, oil painting, aesthetic INDEX TERMS: I.3.3 [Computer Graphics]: Picture/Image Generation — Display algorithms; I.4.0 [Image Processing and Computer Vision]: General 1 INTRODUCTION Figure 1. Process of proposed system In the recent decade, artists express their creativity not only through the intuitive expression but with the help of computer Stroke distribution process as the first step is to decide the technologies such as image processing, computer vision, and stroke point where the brush texture defined by user is located. computer graphics. And using these research fields, a more For the determination of stroke point, the system analyzes the aesthetically evolution of digital art is being represented into a local complexity of an input image using edge detection and fascinating digital form. The aim of these researches, such as non- image segmentation. photorealistic rendering techniques [1][2], is to make use of In painterly rendering process based on the existing method [2], computer techniques to reproduce an aesthetic digital art the system draws an intermediate image from an input picture by representation from a still image. using local image moments [4] and the stroke distribution. The lots of existing non-photorealistic rendering methods have Intermediate image transformation retouches the intermediate a tendency to focus on intrinsic and technical aspects of how to image by using light effect and the number of stroke times at each paint the image similarly to real painting works. In these methods, pixel. After this work, the oil painting-like image that changeable it is important to determine the order, the direction, and the by light direction and light power is created. number of strokes for painterly rendered image generation. 2.1 Stroke Distribution However, for the oil painting works, the extrinsic and environment points such as light effect also acts essentially, As artists decide initially where they paint, the proposed system because the texture and the volume variable by the different light also determines by first the stroke point to be painted. In order for conditions enable to give spectators various impressions. stroke distribution, the system analyzes the local complexity of In this paper, we present a novel method that reproduces oil the input picture using edge detection and image segmentation. painting-like image from a source picture based on a virtual light This process is based on two assumptions; 1) complicated region and non-photorealistic rendering technique. The system first must be painted using lots of small and delicate brushes, 2) simple performs stroke distribution to determine the point to be stroke, by region must be painted using a suitable brush to the region. using edge detection and image segmentation [3] on an input Edge detection is used to extract the location where a rapid and picture. And the intermediate image is constructed with the complex color change between neighboring pixels is appeared. suitable color, orientation and size of brush at each stroke point. Through this method, it is possible to recognize the complicated At last, intermediate image transformation based on the light region having large color variation. In case of simple region extraction, the system applies image * e-mail: k612051@ssu.ac.kr segmentation, which is generally used for grouping the neighbor pixels that consists of similar colors. And then the central points † e-mail: kcjung@ssu.ac.kr of segmented regions are defined as the stroke point. 2 2.2 Painterly Rendering Painterly rendering as second process is to generate a painting- like image as an intermediate image based on the stroke distribution computed from the previous step and local image moments. For rendering the image, the following three properties must be defined; 1) brush texture, 2) stroke properties, 3) stroke order. Brush texture that means the style of brush is defined by user. And stroke properties, such as suitable brush color, location, orientation, and size to each stoke point, can be obtained by local Figure 3. Results of different light directions; (a) right light, (b) top image moments which are used for calculating the centoid, width, light height, orientation of local image. Stroke order is in order to paint large regions first, and depict small regions on the painted large regions based on the brush size at each stroke point. 3 EXPERIMENTAL RESULTS We have tested 50 pictures obtained from the web for the experiments. Two examples of them are shown in Figure 4. Figure 2. Stroke process: (a) input image, (b) local image, (c) local Figure 4. Result examples of our system. binary image, (d) equivalent ellipse based on local image moments(width: 33, length: 26, θ: 1.4358), (e) defined brush texture, (f) rendered brush style. 4 CONCLUSION Figure 2 describes the image moments based painterly This paper proposed a novel method of reproducing oil painting- rendering process. For the calculation about image moments, the like image from a source picture based on virtual light effect and local image is converted to the binary image (Figure 2(c)). And non-photorealistic rendering technique. Computational then the width, the height, and the orientation of brush are methodology such as edge detection, color quantization based computed by image moments (Figure 2(d)). After that, the color image segmentation, and image moments serves as the core of brush is applied to brush texture (Figure 2(f)). All of these steps engine for stroke distribution and painterly rendering. And are computed at every stroke points. through applying light effect into the intermediate image, the system generated oil painting-like image changeable by virtual 2.3 Intermediate Image Transformation light condition. For the future work corresponding to this, we will try to apply the more reasonable light effect through analysis of To create the oil painting-like image, the system transforms the light flow on an input picture. intermediate image by using light direction, light power, and the number of stoke times at each pixel. Acknowledgements For application of light effect, the system utilizes two kinds of data, depth map (D) and gradient map (G). Depth map contains This research was supported by the MKE(The Ministry of the number of stroke times at each pixel, and gradient map is Knowledge Economy), Korea, under the ITRC(Information obtained by differentiation of depth map along the defined light Technology Research Center) support program supervised by the direction. Through adding the gradient image to the intermediate NIPA(National IT Industry Promotion Agency)(NIPA-2009- image, a transformed image (T) as the oil painting-like image is (C1090-0902-0007)), and the Soongsil University BK 21 Digital completed. The following equations represent intermediate image Media Division. transformation. In these equations, , , , , and , mean respectively a pixel value in the transformed image, the REFERENCES gradient image, and the intermediate image. [1] A. Hertzmann. Painterly rendering with curved brush strokes of , , , ,0 . (1) multiple sizes. Proceedings of SIGGRAPH ’98, Computer Graphics Proceedings, Annual Conference Series, pages 453–460. ACM Here, is a light power parameter and , is calculated by the SIGGRAPH, ACM Press, 1998. following equation, [2] M. Shiraishi and Y. Yamaguchi. An algorithm for automatic , 1, if left light painterly rendering based on local source image approximation. , 1, if right light Proceedings of 1th International Symposium on Non Photorealistic , , (2) Animation and Rendering, pages 53-58, 2000. , , 1 if bottom light [3] A. H. Dekker. Kohonen neural networks for optimal colour , , 1 if top light quantization. Network: Computation in Neural Systems, volume 5, where , is a pixel value in the depth map. pages 351-367, 1994. Figure 3(a) and (b) are the results by using right light and top [4] C.H. The and R.T. Chin. On image analysis by the methods of light respectively. As shown in Figure 3, the system enables to moments. IEEE Transactions on Pattern Analysis and Machine generate the oil painting-like image changeable by the light effect. Intelligence, volume 10, pages 496-513, 1988. 3