DOI

10.17077/etd.0hqzzjux

Document Type

Dissertation

Date of Degree

Spring 2017

Access Restrictions

.

Degree Name

PhD (Doctor of Philosophy)

Degree In

Psychological and Quantitative Foundations

First Advisor

Welch, Catherine J

Second Advisor

Dunbar, Stephen B

First Committee Member

Welch, Catherine J

Second Committee Member

Dunbar, Stephen B

Third Committee Member

Harris, Deborah J

Fourth Committee Member

Ansley, Timothy N

Fifth Committee Member

Tan, Aixin

Abstract

Response time (RT) data are able to provide unique insight into both items and examinees regarding speededness and time-demand and should be incorporated into test development practice. To allow test developers to utilize RT information, item RT needs to be summarized into point estimate(PE)(s) that can be understood by content specialists and saved into the item pool. The recent expansion of online testing in K-12 achievement assessments brings opportunities and challenges for measurement experts to investigate and utilize RT information in a context different from that in the majority of literature, in which licensing and certification tests, graduate admission tests, and other applications that incorporate computer-adaptive testing. Using empirical data from four tests in two grade levels of a K-12 standardized achievement assessment, this study explored the empirical distributions of item RT and their fit to five probability distributions, the characteristics of four item RTPEs, and the relationships between item RTPEs and eight item attributes. Based on the principal findings across tests and grades, the empirical distributions of item RT presented widely variable shapes and did not fit any of the five proposed probability distributions; the 90th quantile showed its important capability of capturing and avoiding speededness issues; and the associations between item RTPEs and item attributes proved to be mixed.

The generally idiosyncratic findings of this study call for a different perspective and approach to explore RT data and call for more empirical studies to enlighten test development practice in the K-12 standardized achievement assessment field.

Keywords

Distribution fitting, Item response time, K-12 standardized achievement assessment, Online testing, Regression analysis, Test development

Pages

xx, 221 pages

Bibliography

Includes bibliographical references (pages 109-114).

Copyright

Copyright © 2017 Min Wang

Share

COinS