I have two groups of call center agents. One is a control group, the other received updated training on a process. The groups do not interact with each other. I have measured their performance over time, but the number of calls that each agent takes in a day can vary (Ex. an agent may take a day off, or have a few very long calls, a large number of short calls, etc.), so the number of observations is not consistent between test and control.
I want to analyze not only if the training made a difference, but by how much. Example, if the test group improved performance on a given metric by 10 percent, but the control group improved by 8 percent in the same time, I can do ANOVA or t-tests to judge significance of the change, but shouldn't I weight this to take into account agents who took more or less calls? What statistical test should I uses to take the weight into account?
How about analysis of covariance (ANCOVA) with number of calls included as a covariate? An alternative, equivalent option would be multiple regression with group and number of calls as predictor variables of your dependent variable.
I would recommend using multiple regression with the number of calls per day as your dependent variable. Your independent variables could be iteration (day of observation) and a dummy variable for the training/no training group. Any other variables could be added as independent variables. You would test the training/no training variable to determine if the training group differed in the number of calls per day.
Good luck,
Jim McLean
As I read the description of the problem, you have multiple days worth of data for each subject and an outcome variable (or variables) that are measured daily. Is that right? If so, you will need to use a method that accounts for the correlated nature of the repeated measures. Multilevel modeling comes to mind, with days (at level 1) clustered within agents (level 2). As Christian & Jim have already suggested, you could include some measure of baseline call volume as a (level 2) explanatory variable. HTH.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA