I have applied few image processing methods like: face alignment, glare removal using CLAHE of OpenCV, Background removal, text removal and in-paint if there is any text. I tested on 2 different test sets of unconstrained faces (one of them is 72 and other is 176), I am using RESNET50 embeddings, and for comparing the embeddings of train and test, I am using Euclidean Distance. Without using preprocessing step on larger test set, I am able to achieve accuracy of around ~50%, and with using preprocessing step on smaller one the accuracy is ~50%, the vice-versa for both the cases drops the accuracy to almost 20-25%.

I read various papers, there is no answer to this scenario.

Any pointers to this would be highly appreciated.

An update: I have being able to increase on accuracy to 65% by removing some low resolution images, and other test data (smaller one) to 82%. By removing some images the larger test data goes from 176 to 160 and the other one goes from 72 to 63. But want to improve further.

Thanks!