Date of Degree
Access restricted until 07/13/2019
PhD (Doctor of Philosophy)
Psychological and Quantitative Foundations
Timothy N. Ansley
First Committee Member
Stephen B Dunbar
Second Committee Member
Catherine J Welch
Third Committee Member
Kyong Mi Choi
Fourth Committee Member
Chingwei D Shin
An efficient pool is critical for CAT administrations. Two approaches have been developed to design an optimal CAT pool: the linear programming method (LP; Veldkamp & van der Linden, 2000, 2010) and the bin-and-union method (BU; Reckase, 2003, 2010). This study manipulated different content balancing approaches and exposure conditions to investigate their impacts on the pool performances of the LP and BU methods under practical testing situations.
The optimal pools were constructed in terms of the specification of an operational fixed-length CAT program and the IRT model employed. This study considered the one-parameter logistic (1PL) model to simulate adaptive test item responses using optimal and operational pools. Several psychometric properties were compared between the pools designed under the LP and BU methods. This research attempted to answer the following question: Under the consideration of content balancing and exposure control, what were the benefits and limitations of the LP and BU methods with respect to the optimal pool design? The results were evaluated in terms of pool characteristics, content constraint management, item exposure control, pool utilization, test reliability, and measurement precision.
Similar pool characteristics were found between the LP and BU methods. With respect to the evaluation criteria, the LP and BU pools exhibited consistent performance. However, compared to the LP pools, the BU pools demonstrated slight superiority under the condition with strict content balancing and exposure control. Given two bin widths (.35 and .70), the pools with a bin-width of .35 exhibited better performance than those with a bin-width of .70 with respect to various evaluation criteria. Especially under the condition with the strict content balancing and exposure control, a bin-width of .35 might be a better option to generate an optimal pool than a bin-width of .70 in order to maintain a higher test on-target rate.
This study implemented two optimal pool design methods, the linear programming (LP) method and bin-and-union (BU) method, to evaluate the desired characteristics of item pools and compare the performances of the LP and BU pools in terms of psychometric properties under a fixed-length computerized adaptive test with the one-parameter logistic (1PL) discrete items, in consideration of content balancing and exposure control.
The results showed that similar pool characteristics were found between the LP and BU methods. In terms of the evaluation criteria, the LP and BU pools demonstrated comparable performance. However, under the condition with strict content balancing and exposure control, the BU pools showed slight superiority over the LP pools. It was hoped that the results from the current study would be informative to psychometric researchers and test practitioners in terms of the item pool development and management.
computerized adaptive testing, item pool
x, 126 pages
Includes bibliographical references (pages 121-126).
Copyright © 2017 Ying-Ju Hsu
Hsu, Ying-Ju. "A comparative study of optimal pool design methods in computerized adaptive testing." PhD (Doctor of Philosophy) thesis, University of Iowa, 2017.
Available for download on Saturday, July 13, 2019