In this paper, we propose a unified scheme of subspace and distance metric learning under the Bayesian framework for face recognition. According to the local distribution of data, we divide the k-nearest neighbors of each sample into the intra-person set and the inter-person set, and we aim to learn a distance metric in the embedding subspace, which can make the distances between the sample and its intra-person set smaller than the distances between it and its interperson set. To reach this goal, we define two variables, that is, the intra-person distance and the inter-person distance, which are from two different probabilistic distributions, and we model the goal with minimizing the overlap between two distributions. Inspired by the Bayesian classification error estimation, we formulate it by minimizing the Bhattachyrra coefficient between two distributions. The power of the proposed approach are demonstrated by a series of experiments on the CMU-PIE face database and the extended YALE face database.