In multivariate software like SIMCA, I have some observations out of the threshold in the graph of distance to model and I'd like to know if they can be considered as outliers? and how to calculate the critical distance to the model mathematically?
While I never use SIMCA, I believe you looking for 'Goodness of Fit'. The easy one is to see Graph manually. But you can 'maybe' use AIC or Pearson's chi-squared test (categorical data).
You can just look at the boxplot of the residuals, and the outliers will come right at you: points that are outside the 'whiskers' are outliers. Alternatively, you can use quantile regression to build the lines (surfaces) for the 25th and 75th percentiles and compute the lines (surfaces) corresponding to the 'whiskers'. Your dependent variable measurements that are outside such lines / surfaces are outliers.