ORI Research Bulletin, Volume 12

Permanent URI for this collection

https://hdl.handle.net/1794/30338

Browse

Browsing ORI Research Bulletin, Volume 12 by Author "Reid, John B."

Now showing 1 - 2 of 2
  • Thumbnail Image
    ItemOpen Access
    Effects of Instructional Set and Experimenter Influence On Observer Reliability, No. 11
    (Oregon Research Institute, 1972) Taplin, Paul S.; Reid, John B.
    A laboratory analog of naturalistic observation was used to examine the relationship of observer drift to instructional set and experimenter status. Three instructional sets (no check, random check, and spot check) and two levels of experimenter . status were studied . Results indicated a highly significant decrease in observer reliability coinciding with the shift from training to data collection. This performance decrement was observed in all three instructional set conditions. Within the spot-check condition, reliability on spot-check days was found to be significantly greater than mean reliability immediately before and after spot checks. Further results revealed that observers trained by the high status experimenter performed less reliably than observers trained by the other two experimenters. The possible implications of these results for future observational research, and suggestions for minimizing observer drift were discussed.
  • Thumbnail Image
    ItemOpen Access
    The Efficacy of the Spot-Check Procedure in Maintaining the Reliability of Data Collected by Observers in Quasi-Natural Settings: Two Pilot Studies, No. 8
    (Oregon Research Institute, 1972) Reid, John B.; DeMaster, Barbara
    Preliminary evaluations of two procedures for the assessment and maintenance of observer reliability were conducted. In the first study, one pair of observers was trained to an acceptable level of reliability and was then told that no further assessment of accuracy would be carried out (no-check procedure). In the second study, two pairs of observers were trained to an acceptable level of reliability and were then told that they would be periodically checked for accuracy and informed when the checks were taking place (spot-check procedure). Actually, the reliability of observers in both. studies was continuously monitored. The results of the studies were as follows: (a) the accuracy of observers in the no-check condition dropped dramatically immediately after the termination of overt assessment; (b) the accuracy of observers in the spot-check condition dropped after initial training and overt assessment but exceeded the training level of reliability during each of the spot-check sessions; and (c) the magnitude of the drop in accuracy was smaller for observers in the spot-check study than for those in the no-check study. On the basis of these results , it was argued that the spot-check procedure for checking and maintaining observer reliability is of questionable validity.