Dear Amer, as its name says, parametric analysis is based on the population parameters, say Mean and Variance. These both has criteria to met if we want to use parametric tests. Most important criterion is normality. This criterion says that about 68% of our data should be less than ±1 standard deviation around the mean, 95% less than ±2 standard deviation and 99% less than ±3. If this criterion is strongly violated, then the normality of sample is rejected and we can't trust on our estimates of population parameters based on this sample. So we can't use Parametric tests.
Another criterion is Homogeneity of variance, based on this criterion if we have several groups to analyse and compare, the variance between all groups should not differ significantly, otherwise we should do the nonparametric.
These criteria should be tested for continuous data and some discreet data. Other data such as ordinal and nominal should be analysed using nonparametrics, unless, our ordinal data has an average or high order scale (e.g. Likert with at least 5) and the sample is large enough to test the normality and homogeneity.
Thank you Ehsan about this answer but I not note these criterion in our studies (business administration) in top journal. Are there specific sample size do not need to use these criterion.
The suggestions above are great and represent the minimum criteria you need to meet to do parametric analyses. If you are analysing data from human respondents or behavioural/attitudinal data, then the question of the psychological structure arises. In these cases you need to as if the data is psychologically parametric in its structure. What I mean by this is do your respondents think about the intervals on your scales in an interval or ratio manner. Furthermore, is the scale psychologically linear or is the data of some other structure such as curvy-linear? If your data are not psychologically linear and interval in nature you should use non-parametric analyses. There is nothing wrong with non-parametric analyses and they often more closely match the psychological processes that you are interested in.
Dear Amer, Eric Vittinghoff et. al. say with large samples even if the normality assumption is violated we can use T-test and one way anova. However, outliers should be considered.
Regression Methods in Biostatistics Linear, Logistic, Survival, and Repeated Measures Models
Eric Vittinghoff Stephen C. Shiboski
David V. Glidden Charles E. McCulloch
Page 33, 3.1.7 Robustness to Violations of Assumptions
You can use parametric statistics when sampled data is normally distributed and the dependent variable is in interval / ratio scale. Normal distribution can be evaluated by various normality test e.g. Kolmogorov-Smirnov test, Shapiro-Wilk test, skewness & kurtosis magnitudes < 1.96 respectively etc.
On the contrary, you can use non-parametric statistics when data is not normally distributed (e.g. when sample size is small) and the dependent variable is ordinal scale. Note: if you are using non-parametric statistics then your research findings can't be generalized to the entire population in which this might be a short-coming in your quantitative research. Suggesting before you turn to non-parametric tests, try to obtain more random sample so that your sampled data can be normally distributed.
Using data with ordinal scale e.g. Likert-scale required the use of non-parametric tests albeit there are still debates among scholars whether Likert-scale can justify the use of parametric tests. There are practices to convert the Likert-scale to Semantic Differential scale using Bipolar Adjectives (e.g. Effective1, 2, 3, 4, 5, 6, 7 Not Effective) so that parametric tests can be applied. The rationale claimed is that the intervals between the scale values can be treated as equal, making it an interval scale which justified for parametric tests.