EPSY 506 - Item Response Theory / Rasch Measurement

4 Credit Hours

Semester: Spring 2009
Professor: George Karabatsos
Time: Wednesday 5:00-8:00pm Phone: 312-413-1816
Room: 3312 EPASW E-mail: georgek@uic.edu
Office Hours: Wed 2-4
(EPASW 1034)		 CRN: 28416


Course Description:
This course teaches how to apply psychometric modeling in the analysis of data arising from examinees' responses to items of a test. The test may be an examination, a questionnaire, or any data-collection device which involves the use of test items. Psychometric models, often called Item Response Theory (IRT) Models or Rasch models, continue to see vastly many applications in the educational, psychological, health, and other social sciences. In an IRT (or Rasch) analysis, the main objective is to estimate each examinee's ability of the test, each item's difficulty level, and perhaps, estimate the regression coefficients of one or more (examinee-level or item-level) predictor variables. The course also covers IRT models for data where each examinee are rated by one or more judges on a set of tasks (items).

While this course focuses and grades on applications of various IRT models, it is necessary to cover the theoretical foundations of IRT models, and cover estimation methods (including point-estimation and Bayes posterior estimation). Also, we consider several IRT models in this course. They include various Rasch models, the 2-parameter logistic model, the 3-parameter logistic model, the nonparametric monotone homogeneity model (via kernel regression), graded response models, the generalized partial credit model, and Rasch models with predictor variables. Such models will be illustrated through the analysis of data, using either the PARSCALE, Winsteps, or HLM6 software, and various R programs.

For credit, students are expected to apply IRT/Rasch models on real data, in a take-home exam, in a final paper, and through a class presentation of the paper.

Prerequisite: At least two graduate statistics courses, or equivalents, or consent (e-mail me at georgek@uic.edu ) .

Textbook: Embretson, S.E., & Riese, S.P. (2000) Item Response Theory for Psychologists. Lawrence Erlbaum Associates, Mahwah, NJ.

Additional background readings are listed below as "Relevant References" within the COURSE SCHEDULE.

COURSE SCHEDULE

Date Topic

Jan 14

 

What is an IRT model?
-- Basic data structure in IRT modeling.
-- The general unidimensional IRT model.
-- The item response function (IRF) (i.e., the item characteristic curve, the item-step response function (ISRF), and the item category-response function.
-- The three properties of all psychometric models (unidimensionality, local independence, monotonicity of the IRF/ISRF).
-- Invariant item ordering.
-- Examples if IRT models under this general framework.

21

 IRT/Rasch methods for data analysis
-- Parameter point-estimation.
-- Model Selection
-- Real data illustrations of IRT models for dichotomous item scores, using the Rasch model, the 2-parameter logistic model, the 3-parameter logistic model.
Read: Chapters 1-3 of textbook
28

IRT/Rasch methods for data analysis
-- More data illustrations of IRT models for dichotomous item scores.
Read: Chapter 4-5 of textbook

Feb 4

IRT/Rasch methods for data analysis
-- Real data illustrations of various IRT models for polytomous item scores:
Graded response models, the generalized partial credit model, and Rasch models for polytomous items.
Read: Chapters 6-9 of textbook
11

Nonparametric IRT: Kernel regression for the analysis of multiple-choice and rating-scale items
-- Kernel estimation of the IRF, the ISRF, and the category response function.
---- Investigating the unidimensionality of the measurement scale (i.e., investigating the monotonicity of each IRF/ISRF).
---- Estimating the abilities of each test respondent, and the easiness (difficulty) of each test item.
-- Real data illustrations of Nonparametric IRT models for dichotomous, multiple-choice, and rating-scale items.
Reading: Ramsay, J.O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611-630.
18 Nonparametric IRT: Kernel regression for the analysis of multiple-choice and rating-scale items
-- More data illustrations of Nonparametric IRT.
Read: Chapters 10-13 of textbook

25

IRT/Rasch modeling with predictor variables via Hierarchical Linear Models (HLM)
-- Any Rasch model is a (special) Hierarchical Linear Model.
-- (Rasch) analysis of test items, rating scales, and judge ratings
-- Investigating Item Bias (Differential Item Functioning),
-- Comparing test performance across different groups of respondents.
-- Incorporating additional predictor variables in psychometric analysis.
MIDTERM EXAM IS DUE.

Mar 4

 IRT/Rasch modeling with predictor variables via Hierarchical Linear Models (HLM)
-- More real data illustrations of such models.

11

 Bayesian semiparametric mixed Rasch models for data analysis
-- Bayesian estimation of the posterior distribution.
-- Various applications of Bayesian semiparametric mixed Rasch models.
-- Analysis of test items, rating scales, and judge ratings
-- Investigating Item Bias (Differential Item Functioning),
-- Comparing test performance across different groups of respondents.
-- Incorporating additional predictor variables in psychometric analysis.
Reading: Kleinman, K.P. & Ibrahim, J.G. (1998). A semiparametric Bayesian approach to generalized linear mixed models. Statistics In Medicine, 17, 2579-2596.
18

Bayesian semiparametric mixed Rasch models for data analysis
-- More illustrative applications of Bayesian semiparametric mixed models on real data.
25

Spring Break.

Apr 1

 Equating Test Scores: Given a score on Test X, what is the equivalent score on Test Y?
-- The equipercentile approach to test equating (and using the bootstrap to infer the 95% probability interval of an equated score).
-- Linear equating.
-- Rasch item equating.
-- Bayesian nonparametric equating
Reading: Livingston, S.A. (2004). Equating Test Scores (without IRT). Princeton: Educational Testing Service.

8

Student Presentations
15 Student Presentations
22 Student Presentations
29 Student Presentations
May 6 FINAL PAPER DUE (Exam week)
Please leave final paper in my mailbox in Room 3233, or under my office door at Room 1034.


Grading Policy:
The Midterm Exam, Final Paper, and the presentation of the final paper are each worth 30% of the final grade. Class participation is worth 10%.
Final grades will be given out according to the following scale:

A
90% - 100%
B
79% - 89%
C
68% - 78%
D
57% - 67%
F
56% - Lower

The amount of student class participation will be used to decide borderline grades.
Students will spend substantial amounts of time reading, and on the computer. It is assumed that students will exert individual initiative in solving computing/analysis problems as they arise. There are no exceptions to the above grading scale, and no extra credit work will be accepted. Incompletes will be considered for students with extenuating circumstances. Poor performance on assignments will not be considered in a request for an incomplete.


Midterm Exam: For the midterm exam, you need to demonstrate your ability to analyze types of data sets using different IRT models.
In so doing, please show all work and relevant output (also, please place raw output in an Appendix).


DATA ANALYSIS PRESENTATION and PAPER:

-- The data analyses will consist of the relevant output from the software programs and a complete report stating the results.
-- You may supply your own data or you may solicit faculty (education or other) for data.
-- The final paper should be 8-10 double-spaced pages (not including Appendix), using 1-inch margins and APA style.
-- Please discuss only the relevant results of your analysis, within the main body of the paper. Please put all output in the Appendix of your paper.
-- The presentation has a limit of 20 minutes (about 15 PowerPoint slides).
-- Please hand me a hard-copy of your PowerPoint presentation on the day of your presentation

The paper and presentation must include the following sections:

Introduction -
-- Describe in detail the substantive problems you will be addressing in this research study (5 points).

Methods - (not necessarily in the following order).
-- Describe sample characteristics (3 points).
-- Describe the items on your test(s) (including their number and scoring format) (3 points).
-- Describe the construct you intend to measure with the test(s) (3 points).
-- For data analysis, use one or more IRT models for data analysis.
-- Fully describe your IRT model(s) and the parameter estimation methods (15 points)
---- If you intend to equate test scores, fully describe the equating methods you will implement.
---- If you will implement one or more IRT models and plan to compare them, describe the model selection method you will use.

Results - (not necessarily in the following order).
-- Summarize the results of your analysis, including person ability estimates, item estimates, any estimates of the category parameters, and any estimates of the
regression coefficients of the predictor variables (10 points).
-- Also, if necessary, justify any modifications you make to your test (removing items, removing persons, etc…).
-- Please discuss only the relevant results of your analysis, within the main body of the paper. Please put all output in the Appendix of your paper. (15 points)

Discussion - (not necessarily in the following order).
-- What modifications (if any) would improve the test? (3 points)
-- What are the implications of your study, with respect to the measurement and applications in the field of interest? (3 points)

Please provide appropriate handouts and develop meaningful overheads for your presentation.

Disability Services:
UIC strives to ensure the accessibility of programs, classes, and services to students with disabilities. Reasonable accommodations can be arranged for students
with various types of disabilities, such as documented learning disabilities, vision, or hearing impairments, and emotional or physical disabilities. If you need
accommodations for this class, please let your instructor know your needs and he/she will help you obtain the assistance you need in conjunction with the
Office of Disability Services (1190 SSB, 413-2183).