Projects

Imputation and Score-Based Analysis for Biometric Recognition

• Developed a double imputation method incorporating oversampling techniques like SMOTE into training with MICE to handle missing data in biometric scores.
• Demonstrated the approach's effectiveness on the NIST BSSR1, BIOCOP2008 Multichannel and MIT LL Trimodal Multibiometric Datasets.
•Improved system reliability by addressing class imbalance in recognition tasks when addressing missing data points.

Decision Trees with Verification Criteria for Sequential Score Fusion

• Proposed a verification-based decision tree approach for score fusion, prioritizing verification-based metrics over traditional purity metrics while constructing the tree.
• Demonstrated the approach's effectiveness on the NIST BSSR1, BIOCOP2008 Multichannel and MIT LL Trimodal Multibiometric Datasets.
•Achieved higher fusion performance with fewer scores, particularly on datasets with particularly challenging modalities like the text modality in the MIT LL Trimodal Dataset.
Keywords: Score Fusion, Sequential Fusion, Biometric Recognition

Medical Image Analysis for Radiology Applications

• Achieved strong performance in MRI-based pattern recognition tasks by fusing machine learning models optimized through AutoML.
• Demonstrated the effectiveness of prediction probability fusion across deep learning architectures, enabling cross-domain pattern recognition without requiring internal model details.
•Enhanced versatility in applying fusion techniques to various MRI-based recognition tasks, including stem cell labeling and tumor classification.
Keywords: Medical Imaging, Deep Learning, CNNs, Radiology

Iris-SAM: Iris Segmentation Using a Foundation Model

• Developed Iris-SAM, a foundation model-based architecture, for precise iris segmentation that adapts effectively across varied lighting and occlusion scenarios (dense eyelashes).
• Demonstrates high accuracy in segmenting iris regions, with robust generalization capabilities confirmed across multiple iris datasets.
• Utilizes focal loss, enhancing the learning focus on hard-to-segment examples, which significantly improves the segmentation accuracy.
Keywords: Iris Segmentation, Foundation Models, Segment Anything Model

Adversarial Audio: Crafting Unintelligible Commands for Voice Assistant Attacks

• Adversarial Audio Generation: Develop a pipeline that generates adversarial audio, allowing voice commands to be misunderstood by voice recognition systems like Alexa or Google Assistant, while remaining unintelligible to human listeners.
• Balanced Adversarial Attacks: Optimize a combined loss function to minimize the size of the perturbations while ensuring the voice command remains unintelligible to humans but recognizable to AI voice assistants.
• Security and Ethical Implications: Provide insights into the security vulnerabilities of voice assistants, contributing to the broader conversation around safeguarding AI-driven systems from malicious inputs.
Keywords: Adversarial Machine Learning, Automatic Speech Recognition (ASR), Adversarial Audio

Consistent and Controllable Image Generation

• Our goal is to build a model that generates a face image of a high quality and allows us to have full controllability over different parts of the image.
• We aim to come up with an architecture that allows the training of multiple GANs in a pipeline such that any dataset can be approximated with high quality and full controllability of different components of the target object.
Keywords: Generative Modeling, Facial Synthesis

Imparting Soft-Biometric Privacy to Face Images

Face images are widely used in biometric recognition systems. Yet, a face image can disclose some auxiliary information (age, gender, race, ..) about the individual beyond recognition. Extracting such auxiliary information is considered a privacy violation, and our goal is to confound such information.

Speaker recognition from degraded audio samples

In this project, we are working on developing deep learning-based algorithms for extracting robust speaker dependent voice characteristics from raw audio data and audio features, such as Mel-Frequency Cepstral Coefficients (MFCC), Linear Predictive Coding (LPC), for recognizing speakers in degraded audio samples.

Fusing Face and Voice Modalities for Improved Biometric Recognition

In this project, we are developing algorithms for improving biometric recognition in videos by fusing face and voice modalities.

Presentation attack detection in biometrics

Attempting to fool a biometric system can take many forms, in this work we train a machine to recognize these attempts to further the security of these systems.

Graphical Models for Predicting Missing or Incomplete Biographic Data in Biometric Records

We create a network of biometric records containing information about person such as a face image, name, gender, or ethnicity. Some records in the graph have missing information while some records are correct. We use the graph structure to infer missing information from the correct records.

Relativistic Discriminator: A One-Class Classifier for Generalized Iris Presentation Attack Detection

Presentation attacks are a vulnerability of iris based recognition systems. In this work, we present a discriminator based approach to detecting these presentation attacks.

Attribute prediction from near-infrared ocular images

Soft biometrics, such as race, age, and eye color, provide interesting demographic information as well as a high level description of an individual. In this work, we try to gather those metrics from Near Infrared (NIR) iris images.

Exploring the Use of IrisCodes for Presentation Attack Detection

In iris recognition, the IrisCode is the commonly used representation for the iris data used to recognize an individual. In this work we investigate if these IrisCodes can be used to detect a presentation attack.

Matching Thermal to Visible Face Images Using a Semantic-Guided Generative Adversarial Network

Face recognition in spectra of light other than visible presents a unique challenge as well as great benefits. In this work we use a semantic-guided generative adversarial network (SG-GAN) to match infrared (colloquially, thermal) images to visible spectrum images.

Camera sensor identification for biometric images

Each camera sensor can be uniquely identified using the sensor noise pattern present in the images. We use the principle of Photo Response Non-Uniformity (PRNU) which captures this sensor noise pattern to identify iris sensors and also study the impact of different photometric transformations on sensor identification. Furthermore, we investigate whether the PRNU based sensor classifier can be fooled into misclassifying an image to an incorrect sensor.

DeepTalk: Vocal Style Encoding for Speaker Recognition and Speech Synthesis

In this project, we are developing deep learning-based algorithms for extracting speaker-dependent behavioral speech characteristics for encoding a given speaker's vocal style. We further use the extracted behavioral speech characteristics for improving state-of-the-art speaker recognition and speech synthesis performance.

DeepVOX: Discovering Features from Raw Audio for Speaker Recognition in Non-ideal Audio Signals

In this project, we learn a deep learning-based speech filterbank, called DeepVOX, directly from raw speech audio for extracting robust speaker-dependent voice characteristics. DeepVOX is demonstrated to outperform traditional handcrafted speech features, such as MFCC and LPC, in the context of speaker verification in non-ideal audio signals.

Conditional Identity Disentanglement for Differential Face Morph Detection

Face morphing strategically mixes two or more face images such that the composite image matches successfully to all constituents in terms of biometric utility. Therefore, a morphed face image can be utilized adversarially in an identity document and poses a security threat. We propose a conditional identity disentanglement network using a GAN that not only performs differential face morph detection guided by a trusted non-morphed reference image but also recovers the second identity used in creating the morph.

One-Shot Representational Learning for Joint Biometric and Device Authentication

Smartphones often employ multi-factor authentication to ensure the authorized user is accessing a remote application, such as online banking using a registered device. It requires simultaneous biometric and device authentication. We propose an end-to-end approach to learn a joint representation that encapsulates biometric and sensor (device) identities from a single biometric image.

Image-level Iris Morph Attack

In this project, we explore morph attacks as a security threat to iris recognition systems. The attack associates a single morphed image to two or more different identities, violating the fundamental uniqueness property of biometric systems.

Lookalike Disambiguation

We develop a specialty lookalike disambiguator used in conjunction with a general-purpose face matcher to improve face identification. The disambiguator network is trained on lookalike face images, i.e., those faces which a general-purpose face matcher struggles to correctly distinguish.

Synthesizing Forestry Images Conditioned on Plant Phenotype

• A GAN architecture is proposed to generate synthetic images satisfying a continuous attribute over a specific region of interest.
• The model synthesizes forestry images conditioned on canopy greenness describing a specific vegetation type in a mixed forest.
• Synthetic images are also utilized to predict redness of vegetation.
• The generalizability and scalability of the proposed model are evaluated by extending it to different forest sites and vegetation types.
Keywords: Generative Adversarial Network (GAN), synthetic forestry imagery, plant phenotype prediction, canopy greenness (GCC), redness of plants (RCC)