Date of Degree
PhD (Doctor of Philosophy)
Psychological and Quantitative Foundations
Catherine J. Welch
Stephen B. Dunbar
Interest among the educational community to measure student academic growth has waxed and waned for more than half a century. The latest surge came with the federal requirement to incorporate growth modeling into assessment systems. Even the Common Core State Standards, intended to make American education globally competitive and prepare children for the twenty-first century, were devised to be more fine-grained and thus align to measure individual growth and produce growth reports that inform instruction and programmatic decisions. However, rather than understanding learning progressions and the underlying constructs that yield actionable growth information, the current core of growth measurement centers around quantifying growth, which has led to an influx of growth models that associate academic progress with percentile ranks. The purpose of this research is to examine the psychometric properties of the assessments from which scores are generated and used in growth models, and highlight test properties that could establish a feedback loop with the potential to inform and improve classroom instruction and development of growth sensitive tests.
The primary purpose of this research is to propose a Growth Sensitivity Index (GSI) that gauges the extent to which items capture student academic growth. The second purpose is the estimation of a student-level Growth Score that aids development of actionable growth reports and information. Also discussed is the effect of a test design that relies on overlapping items across adjacent grades, on growth sensitivity of standardized tests using three commonly used normative growth metrics: Student Growth Percentiles (SGPs; Betebenner, 2009), Percentile Rank of Residuals (PRRs; Castellano & Ho, 2013a), and Gain Score Percentile Ranks (GSPRs).
Data used for the analyses comes from four tests (mathematics, computation, reading, and vocabulary) and five grade level spans (3-4, 4-5, 5-6, 6-7, and 7-8). The method involves partitioning students' pre- and post-test raw scores based on the (0, 1) response pattern for each item and calculating Growth Sensitivity Index for items and Growth Score for students. This dissertation describes the rationale supporting the proposed methods, followed by illustrative examples. Finally, the use of GSI in test development and Growth Scores for the purposes of generating informative growth reports is discussed.
The analysis concluded that using a test with overlapping items in adjacent grades, and including items with high GSIs on tests, could generate growth information directly tied to content that could be used not only to inform instruction and give actionable feedback to teachers, parents, and administrators about student growth, but also aid in the development of growth sensitive tests. Study results also indicate that even though the metric (SGPs, PRRs, and GSPRs) used to estimate student-level growth percentiles is not fundamental, test design in terms of degree of overlapping items across adjacent grades does influence the ability of a test to represent variation in measured growth.
Measurement of growth in the area of student achievement needs a deeper understanding of the complexities involved in accurately gauging change that can be attributed to learning. Academic growth measurement requires a structured feedback that informs not only what students know but also what they need to know to learn and grow. Also necessary is the evaluation of the quality of assessments used for growth measurement.
This dissertation proposes two methods, Growth Sensitivity Index (GSI) that provides information at the item-level, and Growth Scores that provide information at the student-level. GSIs can be used to develop growth sensitive tests, and Growth Scores support generation of content-related growth feedback which can help tailor classroom instruction to student-specific needs.
xiv, 176 pages
Includes bibliographical references (pages 124-129).
Copyright 2014 Shalini Kapoor