The aim of this report is to measure the reliability of coding, using a method initially employed by Kogan and Carter (1996). A piece of video taken from a ‘still-face’ experiment was coded and compared to the other individuals’ results. This paper describes and analyzes the outcomes of the reliability check, concentrating on the scientific observation as a methodological approach. Introduction offers a background and rationale for carrying out a reliability check on the ‘still face’ experiment. The next sections describe the method employed to code data, reflect on the chosen approach limitations, and present the outcomes of the experiment. The concluding parts of the report provide an interpretation of the outcome of the kappa calculation and critically summarize the test’s results.
Psychological events fall into two broad categories: external and internal. External events are public. They can, at least in principle, be described by more than one person. An infant shaking a rattle or visually scanning a design, a toddler crying at the approach of a stranger, a preschooler grabbing a toy from another child, a 7-year-old running on the playground, or an adolescent calling a friend on the telephone are all examples of external events. Internal events, on the other hand, are private. Some, such as visual images, sensations of pain or hunger, and verbal thoughts, can be observed only by the experiencing person. Others, such as memory storage or retrieval processes, the operation of syntactic, semantic, or phonological systems in the production of meaningful speech, or processes involved in the coordination of goal-directed movement are unavailable even to the observation of the individual in whom they are taking place (Weinberg et al., 1999).
Because external events are open to public, one might think that their observation would be relatively straightforward. In practice, however, the scientific observation of external psychological events takes great skill and involves either the careful choice of real-life settings or design of experimental situations to maximize the opportunity to observe the events in question and relate them to other variables. The issues of scientific observation and reliability check are addressed in this paper.
In addition, in the observation of external events, researchers concern themselves with two important principles: precision and reliability. A precise observation is one that clearly distinguishes the event in question from other events of the same general type. Reliable observations require that different people observing the same event agree among themselves as to its occurrence and description. External events that cannot be reliably described are not subject of scientific research. The movement of unidentified flying objects is the example of events that have not generally been given the status of scientific events because reports lack sufficient reliability (Weinberg et al., 1999).
By its nature, the occurrence of internal events is not something subject to interpersonal agreement. In order to become subject matter for research, internal events must be indirectly made available to investigation. In the case of internal events open to the observation of the experiencing person, this is done through verbal report. In the case of internal events inaccessible to anyone’s observation, it is done through inference from that which is observable (Weinberg et al., 1999).