EPSY 510 - Theory of Statistics

Semester: Spring 2012
Professor: George Karabatsos
Time: Mondays, 5:00PM - 8:00PM Phone: 312-413-1816
Room: BSB 281
 E-mail: georgek@uic.edu
Office Hours: 2-4pm, Monday CRN: 29748

Course Description:
As evidenced in many of the available graduate courses in applied statistics, statistical models have been applied successfully in many different fields, such as education, psychology, psychometrics, medicine, economics, and so forth. With the many statistical methods that are available, a natural question to ask is:
"How do all these different statistical models (and methods) tie together as a whole?"
This course covers the logical structure of statistics in a unified manner, from a conceptual standpoint. The aim of this course is to provide students a grasp of the basic fundamentals that underlie practical applications of statistical analysis. (In this course, students are not expected to formulate mathematical derivations or proofs).

Obtaining a unified and fundamental understanding of statistics can be quite rewarding. For example, such an understanding:
(1) Enables a person to be more prepared to learn about a new statistical technique (e.g., which may be taught in a graduate course in applied statistics),
including the assumptions of a given statistical model or method;
(2) helps the person improve his/her writing of research papers and dissertations which involve applications of data analysis;
(3) helps a person to be better informed in his/her formulation of a statistical model for a given data analysis problem;
(4) helps the person be more informed when making choices as to what statistical models and what statistical procedures are appropriate to implement for data analysis,
(5) enables the person make rational decisions (conclusions) in data analysis, and
(6) enables the person to provide a reasonable critique of a statistical model or method.

In particular, the goal of this course is to have students obtain a clearer understanding of fundamental topics that are involved in applications of statistical modeling on real data, including the general topics of Probability Theory, Statistical Modeling, Decision Theory, and Large-Sample (Asymptotic) Theory. The course schedule, below, provides more details about what topics the course covers.
Students are asked to complete two open-notes exams during the semester, and an open-notes final exam, which cover the three general topic areas. These exams, which provide learning tools, focus on the understanding of concepts, and intend to help students monitor their own understanding of the important fundamental concepts of statistics.

Prerequisites: At least two graduate courses in statistics.

Primary Textbook:
Casella, G., & Berger, R.L. (2002). Statistical Inference. Pacific Grove, CA: Duxbury. (ISBN 0-534-24312-6)
(I suggest purchasing the less expensive, paperback version, if possible).
Chapter readings and exercises will be assigned on an ongoing basis, throughout the semester.
Such assignments will reflect the topics of the course schedule (below).

Suggested Additional Textbooks/Readings:
Athreya, K.B., and Lahiri, S.N. (2006). Measure Theory and Probability Theory. New York: Springer. (ISBN-10: 0-387-32903-X; ISBN-13: 978-0387-32903-1)
Bernardo, J.M., & Smith, A.F.M. (1994). Bayesian Theory. New York: John Wiley. (ISBN 0-471-92416-4)
Ferguson, T.S. (1996). A Course In Large Sample Theory. Boca Raton, FL: Chapman & Hall/CRC. (ISBN 0-412-04371-8)
Parzen, E. (1999). Stochastic Processes. Philadelphia: SIAM. (ISBN-10: 0-89871-441-9; ISBN-13: 978-0-898714-41-8) (reprint of original 1962 publication)
(I suggest purchasing the less expensive, paperback versions of the books, when available).

COURSE SCHEDULE

Date Topic

Week 1: Jan 9

Review of Basic Math
Logic, Set Theory, Operations On Sets, Product and Sum of a Large Set of Numbers,
Vectors, Matrices, and Matrix Operations, Functions, Limit and Derivatives, Integration
Probability Theory
Sample Space (countable and uncountable), Random Variable, Borel sigma-algebra,
Axioms of Probability, Statistical Independence, Conditional Probability
Week 2: Jan 16 Martin Luther King, Jr., Day. No class.
Week 3: Jan 23

Probability Theory (continued)
Conditional Independence, Bayes Theorem, Random Variables And Their Distributions, Summarizing a Distribution, Multiple Random Variables, Stochastic Processes: Concepts
Comparing two distributions: Ordering Distributions, Divergence (Distance) Between Two Distributions, Comparison Distribution, P-P Plot (Parzen, 2004)
Examples of comparing distributions
Week 4: Jan 30 Probability Theory (Continued): Basic Distributions
Univariate Discrete Distributions
-- Bernoulli Distribution, Binomial Distribution, Negative Binomial Distribution, Discrete Uniform Distribution, Hypogeometric Distribution -- Poisson Distribution, Poisson-Gamma Distribution, Binomial-Beta Distribution, Negative-Binomial-Beta Distribution,
Univariate Continuous Distributions
-- Uniform Distribution, Normal Distribution, Logistic Distribution, Gamma Distribution, Student (t) Distribution,
-- Noncentral Chi-Square Distribution, Gamma-Gamma Distribution, Snedecor (F) Distribution
-- Pareto Distribution, Beta Distribution, Multivariate Discrete Distributions, Multinomial Distribution, Multinomial-Dirichlet Distribution
Multivariate Continuous Distributions
-- Dirichlet Distribution, Multivariate Normal Distribution, Wishart Distribution, Normal-GammaDistribution,
-- Multivariate Student Distribution, Multivariate Normal-Wishart Distribution
Distributions Conditioned On Covariates
-- Normal Linear Model, ANOVA Model
Exponential Families
Illustrations of these basic distributions
Week 5: Feb 6 Probability Theory (Continued)
Probability Theory an Measure Theory
-- Algebras and Topological Spaces, Measures, Extension of Measures, Measurable Transformations,
Differentiation, Integration, Radon-Nikodym Derivative
Week 6: Feb 13

Probability Modeling
Definitions of Probability Model, Parametric Model, Nonparametric Model, Semiparamtric model, Hierarchical Model
Statistical Inference With a Probability Model
-- Point Estimation
-- Criteria of Point Estimation: Bias, mean-square error, efficiency, consistency.
EXAM 1 DUE

Week 7: Feb 20

Probability Modeling (continued)
Statistical Inference With a Probability Model
-- M-Estimation
-- Maximum Likelihood Estimation, and Maximum Likelihood Estimation of Basic Distributions
-- Marginal Maximum Likelihood Estimation
Uncertainty In Point Estimation
-- Uniform Minimum Variance Unbiased Estimator (UMVUE)
-- Bootstrap
Sufficient Statistics and Ancillary Statistics

Week 8: Feb 27

Probability Modeling (continued)
-- Bayesian Inference
-- Bayesian Prediction
-- Bayesian Nonparametrics
-- Markov Chain Monte Carlo (MCMC) Inference of the Posterior:
---- Gibbs Sampling, Metropolis Hastings algorithm, Slice Sampler
---- Parameter Estimation and Convergence In MCMC
---- MCMC Convergence Assessment


Week 9: Mar 5

 Probability Modeling (continued)
Examples of classical point estimation and Baysian inference, using basic probability models,
including models for the Bernoulli distribution, Poisson distribution, exponential distribution,
normal distribution, and the linear model.
Week 10: Mar 12 Probability Modeling (continued)
Exchangeability
-- Full exchangeability, Partial Exchangeability, Partial Exchangeability and Covariates,
Partial Exchangeability and Hierarchical Modeling
Mar 19 SPRING BREAK
Week 11: Mar 26 Probability Modeling (continued)
Decision Theory
-- Basics, Decisions In Point Estimation, Decisions In Model Selection,
Importance of Posterior Consistency In Statistical Decisions, Decisions With P-values
Frequentist Criteria For Coherent Statistical Inference, Classical (Non-Bayesian) Decision Theory,
Bayes Rules, Minimax rules, Power Analysis, .Example of Power Analysis
Week 12: Apr 2 Bayesian Decision Theory
-- Basic elements of decision-making (decision space, utility function, probability distribution of states of nature, expected utility)
-- Axioms of quantitative coherence, the requirements for making rational decisions (conclusions) in data analysis
-- Rational decisions in point estimation
-- Rational decisions in model selection
-- Importance of posterior consistency in making rational statistical decisions
EXAM 2 DUE
Week 13: Apr 9 Decision Theory (continued)
-- On making decisions with p-values, and significance testing
-- Statistical power
Classical Decision Theory
-- Decision rules, Admissible decision rules, inadmissible decision rules, Bayes rules, Minimax decision rules.
Week 14: Apr 16 Asymptotic (Large Sample) Theory
 Convergence: Concepts and Important Results
Modes of Convergence
-- Convergence In Law
-- Laws of Large Numbers
Week 15: Apr 23 Asymptotic (Large Sample) Theory (continued)
 Asymptotic Consistency
-- Basic Concepts, Strong Consistency of Maximum-Likelihood Estimates, Strong Consistency of Posterior Distributions,
-- Asymptotic consistency when the Parametric Model Is Incorrect
-- Consistency of Posterior Distributions Defined On The Space of Distributions
Kullback-Leibler property, and necessary and sufficient conditions for posterior consistency.
Apr 30
 FINAL EXAM DUE (Exam week)
Please leave completed exam in my mailbox in Room EPASW 3233.

 

Grading Policy:
Exam 1, Exam 2, and the Final Exam are each worth 28% of the final grade (I can only accept hard copies of the completed exams).
Class participation (constructive) is worth the remaining 16% of the total grade. Class participation includes participating in the assigned readings,
completing assigned exercises of the assigned readings (or at least, written evidence that an attempt was made to complete the assigned
exercises), as well as in-class participation.
Final grades will be given out according to the following scale:

A
90% - 100%
B
79% - 89%
C
68% - 78%
D
57% - 67%
F
56% - Lower

Students will spend substantial amounts of time reading.
(Standard policy: There are no exceptions to the above grading scale, and no extra credit work will be accepted. Incompletes will be considered for students with extenuating circumstances).

Disability Services:
UIC strives to ensure the accessibility of programs, classes, and services to students with disabilities. Reasonable accommodations can be arranged for students with various types of disabilities, such as documented learning disabilities, vision, or hearing impairments, and emotional or physical disabilities. If you need accommodations for this class, please let your instructor know your needs and he/she will help you obtain the assistance you need in conjunction with the Office of Disability Services (1190 SSB, 413-2183).