Improving Classification Accuracy by Optimizing Activation Function for Convolutional Neural Network on Homomorphic Encryption
Abstract
A secure machine learning technology that performs prediction while encrypting data using homomorphic encryption is being developed. However, Convolutional Neural Networks (CNN) on homomorphic encryption cannot use general non-linear activation functions, Thus, the classification accuracy is low. We proposed a novel method to create an activation function that improves the classification accuracy by performing a pre-training optimization on the coefficients of the polynomial approximation of the Mish function. We confirmed the improvement of classification accuracy for MNIST, Fashion-MNIST, and CIFAR-10 by optimizing the Mish function through pre-training. The classification accuracy can be improved by 4.27% for CIFAR-10. Furthermore, we showed that classification accuracy improves for Fashion-MNIST and CIFAR-10 even when different networks and datasets are optimized by pre-training the activation function. These results show that the activation function of CNNs on a homomorphic encryption can be optimized to improve classification accuracy.