Research at the individual level very frequently employ self-reports from interviewees (leaders, managers, employees, patients, teachers, students). Self-reports suffer from biases on the part of interviewees (halo effect, social desirability) as interviewees seek to paint a favourable impression of themselves. One method that is used to correct this bias defect is to engage with reports from others whether at the one-up, peer-level or one-down level. Other reports, however, also have their own challenges; eg supervisors may not be fully aware or cognisant of the full range of employees behaviours, some of which may not be evident to them. Supervisors may also not be objective and may bring negative subjective emotions into ratings. Has there been any work done on methods to combine other-reports with self-reports as a means of arriving at a fairer more balanced assessment ?
Well... usually, interviews and what you call "self report" are treated as "primary materials" or "sources". They need to be analysed as they are and contents need to be discussed carefully. For example, if you analyse speeches from a politician, you have to discuss his/her motivations, context, political orientation of the person, etc. Then, you can confront it with secondary sources, which may come from scholars, or statistics, etc. I should say that people don't necessarily conceal facts or lie, but they may have a certain opinion. This opinion may be different than the one of someone else -- but it does not mean someone lie, or even that one person is right and the other is wrong. It is up to you to try to understand why people think (or say) that way.
I would suggest you read a book on qualitative methodology in social sciences (or anthropology), you will find more precise answers there.
Louis,
Appreciate the response. The question raised was within the context of quantitative methods and reports recorded through questionnare instruments. I agree however that there are several independent objective secondary sources that could be mined for complementary information on individuals eg for employees there would be company data on grievances, positive achievements, collaborations, patenting etc
OK. From your question, I did not understand you specifically referred to quantitative data. But it does not really change much, because sources and secondary materials should be separated as much as possible. If my memory is good, there is a great study on employment strategies that is using primary data from a company, as well as their own survey. That may be far from your own interest, but the methodology they use is interesting. The reference is Fernandez, Castilla and Moore, "Social Capital at Work: Networks and Employment at a Phone Center", American Journal of Sociology, 105:5, 2000
In fact, may be the article I recommended is not so useful. The question of the authors was related to the usage of networks to find a job and the influence of these networks after people got a job. They have used a variety of instruments, including face to face interviews, surveys... but they also use documents provided by the company, including application forms candidates have to fill to apply and also notes from interviewers (if my memory is good).
Thanks Louis,
The methodology does sound interesting. Although the original question was on self- versus other- sources, the utilisation of company data is also of interest as an independent check. I will review the referenced Fernandez article.
Silburn
Found a comment from Kuvaas on this topic.
"Sharma et al. (2004) reported that ....self-ratings showed better ability to discriminate between medium and high performers than managerial evaluations. Thus, it is far from obvious that the extra effort involved in gathering performance data by supervisors would have produced better performance data. Still, future research should ideally use both self-report and supervisor measures of work performance."
Kuvaas et al 2008 A test of hypotheses derived from self-determination theory among public sector employees.
If the self-report and other-report measure represent the same concept, you might estimate a common latent trait score using SEM or CFA. We did this with two measures (self-report and other-report) of treatment motivation when predicting the patients' behavioural engagement during the treatment. Although each of the two treatment motivation measure predicted the behavioural engagement only moderately and both motivation measures were only moderately intercorrelated, the latent (error free) common motivation factor emerged as a very strong predictor of the behavioral engagement. Put differently, although the self-report and other-report measure had only limited common variance, this common variance was a much more valid predictor of en external criterion than each measure alone.
You can find this in http://repository.ubn.ru.nl/bitstream/handle/2066/45156/45156_meastrmoa.pdf?sequence=1 and in Drieschner & Boomsma (2008) 'The Treatment Engagement Rating scale ...' .
An explanation for this phenomenon might be in line with your own analysis. Self-report and other-report have their limitations and thus generate partly invalid score variance. However, because they have different limitations, the variance they have in common represets the intended concept much more validly.
Very insightful and interesting Klaus,
Will have a look at the referenced paper and revert
Silburn
- Badges
- Science topic
- Similar topics
- Social Science
- Sociology