Is Facial Beauty in the Eyes? A Multi-Method Approach to Interpreting Facial Beauty Prediction in Machine Learning Models
This project explores the contributions of specific facial regions to overall beauty assessments using machine learning models. We employ multiple interpretability methods, including permutation feature importance, XRAI (eXplanation with Ranked Area Integrals), and individual feature prediction to analyze these contributions.
We perform experiments on the following datasets:
- MEBeauty: MEBeauty Database
- SCUTFBP-5500: SCUT-FBP5500 Database
The datasets are preprocessed using Mediapipe for face detection, alignment, and keypoint detection. For more details, refer to cropping.py.
- Mediapipe Face Landmarker: Mediapipe Face Detection
XRAI aims to explain each pixel of the input image's contribution to the overall prediction of the model. See xrai.ipynb and region_attribution.ipynb
Permutation importance assesses the importance of each facial region by randomly permuting the region across the dataset and measuring the resulting decrease in model performance on the testing dataset. See permutation.ipynb
We train separate models on each region of interest of the face and compare the importance of each feature based on the generalizability of its respective model. See individual_regions.ipynb
For any questions or feedback, please contact Ahmed Aman Ibrahim at ahmedamanibrahim@gmail.com or Noah Hassan Ugail at ugailnoah@gmail.com