Methodological Issues in the Multi-Level Analysis of School Environments
Loading...
Date
1993
Authors
Stockard, Jean
Journal Title
Journal ISSN
Volume Title
Publisher
JAI Press Inc.
Abstract
For a full understanding of student achievemenJ it is essential to use multi-level
analyses, which take into account the characteristics of individual students, as
well as the nature of their families, classrooms, schools, and communities. Such
analyses can be quite complex; and this ·article reviews, in non-technical terms,
relevant methodological issues. Over the last 30 years statistical analyses available
for multi-level models haveimproved greatly. While researchers once used crosstabulations
and, somewhat later, variants of the general linear model, they can
now use hierarchical linear models. These techniques allow the exploration of
very complex interaction effects, which are often theoretically expected in multilevel
models of achievement. Issues of measurement and model specification can
also be more difficult with multi-level than with single-level models and need
to be carefully considered. lt is suggested that a better understanding and use
of multi-level models can help bridge the gap between the "input-output" and
"process-product" traditions of educational research.
An overwhelming amount of evidence indicates that students' achievement
is influenced by individual characteristics, such as their ability and
socioeconomic background._ Yet, a great deal of research indicates that
classroom, school, and community environments also affect students' learning.
For· instance, among students with equal measured ability and similar
socioeconomic backgrounds, those who are in classrooms and schools with
high achievement related norms and more supportive interpersonal
environments tend to have high achievement. Similarly, students who live in
communities whose citizens support and participate in school activities have
higher achievement than students in other communities, even when they have
equal individual characteristics. (Stockard & Mayberry, 1992, for an extensive
review of this literature.) Thus, most educational researchers today would
probably agree that multi-level analyses are essential if we are to fully
understand student achievement. Studies of student learning and achievement
must take into account not just students' individual characteristics, but also
the environments in which they learn.
Analyzing multilevel effects, however, is far from simple. It involves careful
attention to the proposed models which are studied, the data used to measure
variables in these models, and the techniques used to analyze these data. While
there are still many unanswered questions in this area, understanding of the
complexities involved has expanded greatly in recent years. Some of these
discussions are highly technical and statistical, while others are more accessible
to all researchers. Unfortunately, the most recently developed analysis
techniques, which are much better suited than earlier methods to handle the
various theoretical and substantive complexities,• are unfamiliar to many
researchers. They are not yet covered in standard statistics textbooks nor
included in general statistical packages, and many researchers do not
understand how these techniques are related to more familiar methods.
In this paper I review this literature, presenting the material in general terms,
with references to the more technical literature for those who are so inclined.
I first describe analysis techniques that have been used over the years to examine
environmental influences on students' learning. Second, I examine the
translation between theory and research design, called the specification of
theory; and then explore the issue of measurement, how theory is translated
into data. Finally, I briefly describe how multi-level analyses, and especially
the most recently developed analysis techniques, can begin to bridge the
unfortunate gap between what are commonly called the "input-output" and
"process-product" research traditions in studies of student achievement.
My focus is primarily on quantitative analyses, rather than qualitative or
ethnographic/field work. This is not meant to denigrate the latter area of
research, for such methods are the only way in which to obtain detailed,
subjective, rich accounts of how students learn within classrooms, schools, and
communities. Moreover the questions and issues I discuss regarding measurement and specification of models ar:e not unique to quantitative work.
Still, mos.t of the literature regarding methodological issues has focused on
quantitative analyses, and thus I primarily discuss that literature.
Description
24 pages
Keywords
statistical analyses, environmental influences on student learning, methodological issues, school environments