| Semester: Spring 2008 |
Professor: George Karabatsos
|
| Time: Tuesdays, 5:00PM - 8:00PM | Phone: 312-413-1816 |
| Room: 2219 EPASW (1040 W Harrison St) |
E-mail: georgek@uic.edu |
| Office Hours: 2-4pm, Tuesday | CRN: 26327 |
Course Description:
As evidenced in many of the available graduate courses in applied statistics,
statistical models have been applied successfully in many different fields,
such as education, psychology, psychometrics, medicine, economics, and so forth.
With the many statistical methods that are available, a natural question to
ask is:
"How do all these different statistical models (and methods) tie together
as a whole?"
This course covers the logical structure of statistics in a unified manner,
from a conceptual standpoint.
The aim of this course is to provide students a grasp of the basic fundamentals
that underly all practical applications of statistical analysis.
(In this course, students are not expected to formulate mathematical derivations
or proofs.
Thus, this course does not require the student to have a rich mathematical background).
Obtaining a unified and fundamental understanding of statistics can be quite
rewarding.
For example, such an understanding:
(1) Enables a person to be more prepared to learn about a new statistical technique
(e.g., which may be taught in a graduate course in applied statistics),
including the assumptions of a given statistical model or method;
(2) helps the person improve his/her writing of research papers and dissertations
which involve applications of data analysis;
(3) helps a person to be better informed in his/her formulation of a statistical
model for a given data analysis problem;
(4) helps the person be more informed when making choices as to what statistical
models and what statistical procedures are appropriate to implement for data
analysis,
(5) enables the person make rational decisions (conclusions) in data analysis,
and
(6) enables the person to provide a reasonable critque of a statistical model
or method.
In particular, students who take this course will obtain a clearer understanding
of the following fundamental topics that are involved in applications of statistical
modeling on real data.
These fundamental topics include:
(1) Probability Theory: Including the definitions of the
axioms of probability, conditional probability, statistical independence and
conditional independence, Bayes theorem, continuous distributions versus discrete
distributions, univariate distributions versus multivariate, conditional distributions
and marginal distributions, mean, median, variance, covariance, correlation,
ordering of distributions, and measuring the distance (divergence) of one distribution
from another distribution;
(2) Statistical Modeling: Definition of a probability model,
a parametric model, a nonparametric models, a semiparametric model, and a hierarchical
model; frequentist statistical inference versus Bayesian statistical inference;
point estimation of a model parameter, including maximum-likelihood estimation,
and including the concepts of estimation bias, estimation variance, and estimation
efficiency; inference of the posterior distribution of a model parameter; hypothesis
testing, likelihood-ratio testing, model selection, and power analysis; an overview
of important parametric and nonparametric models, including ANOVA models, regression
models, Rasch models, structural equation models, and nonparametric models defined
explicity by a prior distribution with domain the space of distribution functions;
(3) Principles of Rational Statistical Inference: Five
axioms for making rational decisions (i.e., rational conclusions in data analysis,
as opposed to irrational), Bayesian decision theory, and classical decision
theory; Sufficiency, ancillarity, and the likelihood principle; the repeated
sampling principle, and the invariance principle; DeFinetti's General Representation
Theorem for exchangeable random variables;
(4) Large-Sample (Asymptotic) Theory: Including the basic
concepts of convergence, and rate of convergence, asymptotic consistency, asymptotic
bias, asymptotic efficiency, the Central Limit Theorem, asymptotic normality,
asymptotic normality of the maximum-likelihood estimate, asymptotic normality
of the posterior distribution, necessary and sufficient conditions which enable
consistency of posterior distributions defined on the space of distributions
(as the sample size increases, such a posterior distribution converges to assigning
probability 1 to the true distribution of the data).
In terms of graded work, students are asked to complete two open-notes exams
during the semester, and an open-notes final exam.
The first exam deals with concepts on Probability theory, the second exam deals
with concepts of statistical modeling, and the final exam deals with concepts
of Principles of Rational Statistical Inference and Large-Sample (Asymptotic)
Theory.
These exams focus on the understanding of concepts, and are intended to help
students monitor their own understanding of the important fundamental concepts
of statistics.
Thus, these three exams are treated as learning tools.
Prerequisites: At least two graduate courses in statistics,
or consent of instructor (George Karabatsos, e-mail: georgek@uic.edu).
Readings: (I will provide the article to you. The
three textbooks can be purchased from Chicago
Textbook Inc. I suggest that you purchase the less expensive, paperback
versions of the textbooks)
Bernardo, J.M., & Smith, A.F.M. (1994). Bayesian Theory. New York:
John Wiley. (ISBN 0-471-92416-4)
Casella, G., & Berger, R.L. (2002). Statistical Inference. Pacific
Grove, CA: Duxbury. (ISBN 0-534-24312-6)
Ferguson, T.S. (1996). A Course In Large Sample Theory. Boca Raton, FL:
Chapman & Hall/CRC. (ISBN 0-412-04371-8)
Gutiérrez-Peña, E., & Walker, S.G. (2005). Statistical decision
problems and Bayesian nonparametric methods. International Statistical Review,
3, 309–330.
COURSE SCHEDULE
| Date | Topic |
Readings
|
|
Jan |
Probability Theory -- Basic Probability Theory ----- Sample Space (countable and uncountable), Random Variable, Borel sigma-algebra ----- Axioms of Probability ----- Statistical Independence, Mutual Independence ----- Conditional Probability, Bayes Theorem, Conditional Independence ----- Axiom of Countable Additivity vs. Axiom of Finite Additivity |
Casella & Berger
Chapters 1,2 |
| 22 | -- Random Variables And Their Distributions ----- Cumulative distribution function (cdf) ----- Discrete random variable, continuous random variable ----- Discrete distribution, and probability mass function (pmf) ----- Continuous distribution, and probability density function ----- Basics of Integration -- Summarizing a Distribution ----- Definition of Mean, Median, Mode, Quantile, Moments and central moments, skewness, kurtosis, quantile, range, interquartile range. |
Casella & Berger
Chapter 3 |
| 29 |
Probability Theory (Continued) |
Casella & Berger
Chapter 4 |
| Feb 5 |
Probability Theory (Continued) -- Ordering Two Distributions ----- Two distributions ordered by the mean, or by the median ----- Two distributions ordered by the variance ----- Stochastic Precedence order between two distributions ----- Stochastic order of two distributions, Hazard order of two distributions, Likelihood ratio order of two distributions -- Distance between two distributions, and the Csiszar divergence (a general measure of distance) ----- Chi-square divergence, Kullback-Leibler divergence, Hellinger distance, and variational distance of L1 norm. |
|
| 12 |
Probability Models |
|
| 19 |
Applying Probability Models for Data Analysis |
Casella & Berger
Chapter 6,7 |
| 26 |
Some Probability Models For Univariate Distributions |
|
| Mar 4 |
Some Probability Models For Multivariate Distributions -- Probability Models For Multivariate Discrete Distributions ----- Multinomial Distribution, Multinomial-Dirichlet Distribution, Loglinear Model -- Parametric Models For Multivariate Continuous Distributions ----- Dirichlet Distribution, Multivariate Normal Distribution, Wishart Distribution, Normal-Gamma Distribution, Multivariate Student Distribution, Multivariate Normal-Wishart Distribution |
|
| 11 | Some Models For Conditional Univariate Distributions -- Normal Linear Model, Generalized Linear Model, ANOVA Model, MANOVA model, Exponential Family Models |
|
| 18 | Some Hierarchical (Mixture) Models -- Student (t) Distribution, Binomial-Beta Distribution, Negative-Binomial-Beta Distribution, Noncentral Chi-Square Distribution, Poisson-Gamma Distribution, Gamma-Gamma Distribution, Pareto Distribution -- Hierarchical Generalized Linear Models (HGLM), and Fully-Bayesian HGLM -- Confirmatory Factor Analysis (a Structural Equation Model), and psychometric models -- Models based on priors on the space of distributions: Dirichlet Process prior, Bivariate Dirichlet Process, Random Bernstein Polynomial, Pólya Tree Prior Distribution, Dirichlet Process Mixture of Normals Model. |
|
| 25 | SPRING BREAK | |
| Apr 1 |
Exchangeable Random Variables and Probability
Modeling -- Full exchangeability (de Finetti, 1937, 1938) -- Partial Exchangeability -- Partial Exchangeability and modeling covariates -- Partial Exchangeability and hierarchical modeling EXAM 2 DUE |
Bernardo & Smith
Chapters 1-6, Appendix B |
| 8 | Bayesian Decision Theory -- Basic elements of decision-making (decision space, utility function, probability distribution of states of nature, expected utility) -- Axioms of quantitative coherence, the requirements for making rational decisions (conclusions) in data analysis -- Making rational decisions in point estimation -- Making rational decisions in model selection -- Importance of posterior consistency in making rational statistical decisions -- On making decisions with p-values, and significance testing -- Statistical power |
Gutiérrez-Peña & Walker
|
| 15 | Classical Decision Theory -- Decision rules -- Admissible decision rules, and inadmissible decision rules. -- Bayes rules -- Minimax decision rules. |
|
| 22 |
Asymptotic (Large Sample) Theory |
Ferguson |
| April 29 |
Asymptotic (Large Sample) Theory (continued) |
Ferguson |
| May6 |
FINAL EXAM DUE (Exam week) Please leave paper in my mailbox in Room 3233, or under my office door at Room 1034. |
Grading Policy:
Exam 1, Exam 2, and the Final Exam are each worth 1/3rd of the final grade
(I can only accept hard copies of the completed exams).
Final grades will be given out according to the following scale:
| A |
90% - 100%
|
| B |
79% - 89%
|
| C |
68% - 78%
|
| D |
57% - 67%
|
| F |
56% - Lower
|
Students will spend substantial amounts of time reading.
(Standard policy: There are no exceptions to the above grading scale, and no
extra credit work will be accepted. Incompletes will be considered for students
with extenuating circumstances).
Disability Services:
UIC strives to ensure the accessibility of programs, classes, and services to
students with disabilities. Reasonable accommodations can be arranged for students
with various types of disabilities, such as documented learning disabilities,
vision, or hearing impairments, and emotional or physical disabilities. If you
need accommodations for this class, please let your instructor know your needs
and he/she will help you obtain the assistance you need in conjunction with
the Office of Disability Services (1190 SSB, 413-2183).