Imparting Soft-Biometric Privacy to Face Images

Face recognition has become one of the primary authentication systems in many applications. Typically, face images of individuals are collected and stored in a biometric database, primarily for recognition purposes. However, with advancements in machine learning has made it possible to extract auxiliary information (also called soft-biometric attributes) such as age, gender, ethnicity, and health characteristics from face images. While users of a biometric system have willingly provided their biometric data for recognition purposes, extracting those attributes without consent from users is violating users’ privacy.

In the context of privacy, our goal is to modify a face image in such way that the primary biometric utility (face recognition) is retained, therefore, the modified images can still be used for recognition, while at the same time, extracting age, gender, and race from face images is confounded. In particular, we have developed a specialized neural network where an input face image is perturbed while satisfying two conditions: 1) the modified face image must match with the original unmodified image using an auxiliary pretrained face matcher, 2) the soft-biometric attributes are confounded based on an auxiliary face attribute classifier. The trained model is then used for transforming face images in test sets, and the performance of multiple face matchers and attribute classifiers were evaluated. The results confirmed that biometric utility is retained in the transformed face images, while the performance of attribute classifiers were dropped.

V. Mirjalili, S. Raschka, A. Ross, “FlowSAN: Privacy-Enhancing Semi-Adversarial Networks to Confound Arbitrary Face-Based Gender Classifiers,” IEEE Access, Vol. 7, No. 1, pp. 99735 - 99745, December 2019.

V. Mirjalili, S. Raschka, A. Ross, "Gender Privacy: An Ensemble of Semi Adversarial Networks for Confounding Arbitrary Gender Classifiers," Proc. of 9th IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), (Los Angeles, USA), October 2018.

V. Mirjalili, S. Raschka, A. Namboodiri, A. Ross, "Semi-Adversarial Networks: Convolutional Autoencoders for Imparting Privacy to Face Images," Proc. of 11th IAPR International Conference on Biometrics (ICB 2018), (Gold Coast, Australia), February 2018.

V. Mirjalili and A. Ross, "Soft Biometric Privacy: Retaining Biometric Utility of Face Images while Perturbing Gender," Proc. of International Joint Conference on Biometrics (IJCB), (Denver, USA), October 2017.