As machine learning models have been effectively applied to various real-world problems, collecting data from various sources became crucial for many service providers (Park, Hah, and Lee 2017). In this situation, privacy issues have become a major problem in the learning society. Therefore, it is necessary to preserve the privacy of the data and the security of the learning process without compromising the performance of the learning algorithms.

Homomorphic encryption (HE) enables computations on encrypted data (ciphertext) which are equivalent to operations on decrypted data (plaintext) (Gentry 2009). In recent years, privacy-preserving machine learning applications have developed, but they have high computational cost and huge memory burden. Cheon et al. developed a homomorphic encryption scheme for approximate arithmetic (HEAAN) (Cheon et al. 2017).

Support vector machine (SVM) is one of the most effective machine learning algorithms for classification (Cortes and Vapnik 1995). The training data and the model parameters should be protected to apply the SVM model to the security-preserving scenario. Therefore, in this paper, we propose the implementation for the training phase of the SVM classifier on the encrypted domain with the HEAAN scheme.

Fully homomorphic encryption (FHE) is a cryptographic scheme which aims to enable homomorphic operations such Copyright held by authors as additions and multiplications on encrypted data. Recently, HEAAN scheme (Cheon et al. 2017) was developed to carry out approximate computations efficiently.

For the (leveled) HEAAN of depth L, we set the parameters such as a power of two M , integers p, q 0 , q L = p L • q 0 , h and P , and a real σ for λ-bit security. The specifications of the algorithm are as follows.

-

q at level . For more details, we recommend to see (Cheon et al. 2017).

The SVM aims to find the maximum margin hyperplane, given the training examples {(x 1 , y 1 ), ...,

To train the SVM model over encrypted data, we used a nonlinear least-square SVM with a basis function φ(x) : R d → R l that minimizes the following optimization problem (Suykens and Vandewalle 1999):

where w ∈ R l . To solve the problem (1), we construct the Lagrangian function:

With the optimality conditions of the Lagrangian function, the following linear system is obtained by removing w and e:

where Ω ∈ R (n+1)×(n+1) s.t. Ω i,j = k(x i , x j ). We introduce an additional least square problem with gradient descent method for the system (2), which convergence can be

where represents a Hadmard product. By Schur product theorem, the first term becomes PSD with Mercer kernel. Therefore, the linear system (2) is well-posed with some η > 0, and the convergence of iterative method can be gauranteed. With these design components, the intermediate dycryptions for training the SVM model can be reduced.

In this paper, we propose a protocol which can reduce the communication to construct kernel matrix and to learn the model parameters with FHE operations. Figure 1 illustrates the whole procedure of our protocol which secures serverside and data provider-side information because all operations can be performed on encrypted domain.

As mentioned previously, we first compute the kernel matrix from the encrypted data. For simplicity, we assume that the kernel matrix is encrypted as a ciphertext. We want to build a parallelizable procedure because of computational cost and large memory of ciphertext. The HEAAN scheme supports slot-wise operation over ciphertext, so the operations of matrices should be replaced with addition and Hadmard product. We propose data structure for calculating the inner product matrix. Assume that Z ∈ R n×d and ZZ T , Ẑj ∈ R n×n .

The multiplication of cihpertexts in this operation can be performed on d different machines in parallel.

In this study, to efficiently implement the gradient descent method, we utilize the symmetricity of the matrix (3) in

Our procedure replaces the dual convex optimization with numerical gradient descent to implement the securitypreserving LSSVM. To illustrate the result of this replacement, we compare the classification performances with RBF and polynomial kernels for some real datasets used in (Suykens and Vandewalle 1999). Table 1 shows that the replacement does not severely affect the performances. From this results, we can expect to develop a secure-preserving SVM without compromising the performance.

In this paper, we present a new framework to train the LSSVM model using HEAAN. This framework includes protocol, data structure and secure-preserving iterative training procedure. Our method considers the reduction of computational cost and memory burden that are the common problem for the application of HE scheme. The proposed solution can be helpful to show the potential for the practical utilization of machine learning models without concerns on security and privacy issues.