Agreement and Disagreement in 360 Degree Feedback: Two Sides of the Same Coin
Empirical research on 360 degree feedback has generally fallen into three different categories: correlational approaches, group difference approaches, and latent variable approaches. The correlational approaches and group difference approaches use mean ratings for rater groups in analyses. The current study presents models that disaggregate 360 degree data into between and within groups estimates. The study will demonstrate how failing to account for differences at the between and within levels produces biased parameter estimates of the between level, and can lead researchers to vastly different conclusions. In an effort to demonstrate the bias and loss of information that occurs from traditional aggregation approaches, this study presents an algebraic representation of the traditional and multilevel-structural equations modeling (ML-SEM) approaches, as well as empirical examples with real world data comparing the traditional and ML-SEM approaches. Data used in the empirical examples came from a large state wide reading intervention. One aspect of the intervention was that grade school teachers were provided with reading coaches who served as mentors to these teachers. Teachers and coaches were provided requests to complete surveys via the internet. Surveys asked teachers and coaches about the teacher's classroom problems at the end of the year, frequency of support provided by the coach, and percentage of support that the teacher implemented. Four hundred eighty-one teachers provided reports on 105 reading coaches (multiple teachers rate each coach), and 89 reading coaches provided ratings of 600 teachers (coaches provided multiple ratings of teachers). The results indicate that in many instances mean ratings are poor indicators of the group level variables, and that correlations based on mean ratings are, in general, downwardly biased relative to the between level results. In addition, within level results provided by ML-SEM models resulted in a different interpretation of the relations between the variables than was made at the between level, or when using mean ratings. Specifically, the within result indicate that the relations of perceptions of coach support with classroom problems and implementation of coach suggestions were different depending on the source of the ratings.
What people are saying - Write a review
We haven't found any reviews in the usual places.