Document Type

Dissertation

Date of Degree

Fall 2014

Degree Name

PhD (Doctor of Philosophy)

Degree In

Psychological and Quantitative Foundations

First Advisor

Catherine J. Welch

Second Advisor

Stephen B. Dunbar

Abstract

Interest among the educational community to measure student academic growth has waxed and waned for more than half a century. The latest surge came with the federal requirement to incorporate growth modeling into assessment systems. Even the Common Core State Standards, intended to make American education globally competitive and prepare children for the twenty-first century, were devised to be more fine-grained and thus align to measure individual growth and produce growth reports that inform instruction and programmatic decisions. However, rather than understanding learning progressions and the underlying constructs that yield actionable growth information, the current core of growth measurement centers around quantifying growth, which has led to an influx of growth models that associate academic progress with percentile ranks. The purpose of this research is to examine the psychometric properties of the assessments from which scores are generated and used in growth models, and highlight test properties that could establish a feedback loop with the potential to inform and improve classroom instruction and development of growth sensitive tests.

The primary purpose of this research is to propose a Growth Sensitivity Index (GSI) that gauges the extent to which items capture student academic growth. The second purpose is the estimation of a student-level Growth Score that aids development of actionable growth reports and information. Also discussed is the effect of a test design that relies on overlapping items across adjacent grades, on growth sensitivity of standardized tests using three commonly used normative growth metrics: Student Growth Percentiles (SGPs; Betebenner, 2009), Percentile Rank of Residuals (PRRs; Castellano & Ho, 2013a), and Gain Score Percentile Ranks (GSPRs).

Data used for the analyses comes from four tests (mathematics, computation, reading, and vocabulary) and five grade level spans (3-4, 4-5, 5-6, 6-7, and 7-8). The method involves partitioning students' pre- and post-test raw scores based on the (0, 1) response pattern for each item and calculating Growth Sensitivity Index for items and Growth Score for students. This dissertation describes the rationale supporting the proposed methods, followed by illustrative examples. Finally, the use of GSI in test development and Growth Scores for the purposes of generating informative growth reports is discussed.

The analysis concluded that using a test with overlapping items in adjacent grades, and including items with high GSIs on tests, could generate growth information directly tied to content that could be used not only to inform instruction and give actionable feedback to teachers, parents, and administrators about student growth, but also aid in the development of growth sensitive tests. Study results also indicate that even though the metric (SGPs, PRRs, and GSPRs) used to estimate student-level growth percentiles is not fundamental, test design in terms of degree of overlapping items across adjacent grades does influence the ability of a test to represent variation in measured growth.

Public Abstract

Measurement of growth in the area of student achievement needs a deeper understanding of the complexities involved in accurately gauging change that can be attributed to learning. Academic growth measurement requires a structured feedback that informs not only what students know but also what they need to know to learn and grow. Also necessary is the evaluation of the quality of assessments used for growth measurement.

This dissertation proposes two methods, Growth Sensitivity Index (GSI) that provides information at the item-level, and Growth Scores that provide information at the student-level. GSIs can be used to develop growth sensitive tests, and Growth Scores support generation of content-related growth feedback which can help tailor classroom instruction to student-specific needs.

Keywords

publicabstract

Pages

xiv, 176 pages

Bibliography

Includes bibliographical references (pages 124-129).

Copyright

Copyright 2014 Shalini Kapoor

Included in

Education Commons

Share

COinS