For example if we have speakers within the same accent category does it make the decision process easier or harder compare to the speakers with different accent?
Theoretical explanation and/or empirical result will be appreciated.
There's some analysis in section 5.4 of this paper: http://www.researchgate.net/publication/222436066_NIST_and_NFI-TNO_evaluations_of_automatic_speaker_recognition
You can also find something here, where they use non-nativeness scores to improve performance: http://www.researchgate.net/publication/224762259_System_combination_using_auxiliary_information_for_speaker_verification
Article NIST and NFI-TNO evaluations of automatic speaker recognition
Conference Paper System combination using auxiliary information for speaker v...