Course Overview

This course provides a rigorous introduction to statistical inference. Topics include basic probability theory (probability measures; discrete and continuous random variables; joint, marginal and conditional distributions; mean, variance and covariance), Bayesian and frequentist perspectives on uncertainty quantification, decision theory, point estimation (Bayesian and maximum likeihood estimation) and hypothesis testing.

Teaching team

Lecture times and location

Prerequisites

Multivariate calculus. Students should be comfortable with integration and differentiation of functions of several variables. Linear algebra is not required for this course but will be for its sequel, STAT 24500.

Course Schedule

Week 1:

Lecture 1: Overview and Probability Basics (Sections 1.1-1.4)

  • Probability, statistics and decision theory
  • Basic set theory
  • Sample spaces and events
  • Probability measures
  • Computing probabilities by counting

Lecture 2: Conditional Probabilities, Independence, and Bayes Theorem (Sections 1.5-1.6)

  • Conditional probability
  • Independence
  • Bayes’ theorem

Week 2:

Lecture 3: Discrete Random Variables (Sections 2.1)

  • Probability mass functions
  • Common discrete distributions

Lecture 4: Continuous Random Variables and Functions of random variables (Sections 2.2-2.3)

  • Probability density functions
  • Cumulative distribution functions
  • Functions of a random variable
  • Transformation techniques

Week 3:

Lecture 5: Measuring probability distributions (Sections 4.1-4.2)

  • The expected value of a random variable
  • Variance and standard deviation
  • Properties of expectation and variance

Lecture 6: Joint and marginal distributions (Sections 3.1-3.4)

  • Joint probability distributions
  • Marginal distributions
  • Independent random variables

Week 4 (Midterm 1):

Lecture 7: Summaries of joint distributions (Sections 3.5-3.6, 4.3)

  • Functions of 2+ random variables
  • Covariance and correlation

Lecture 8: Conditional Distributions (Section 3.5)

  • Conditional probability density/mass functions
  • Conditional expectation
  • Applications

Week 5:

Lecture 9: Bayesian point estimation (Section 8.6)

  • The likelihood function
  • Prior and posterior distributions
  • Conjugate families

Lecture 10: A decision-theoretic framework for estimation

  • Loss functions.
  • Bayes estimators.

Week 6:

Lecture 11: Sampling properties of estimators

  • Risk functions.
  • Bias, variance, and mean squared error.
  • Shrinkage, regularization and bias-variance trade-off.

Lecture 12: Maximum likelihood estimation (Section 8.5)

  • The method of maximum likelihood.
  • Properties of MLEs.

Week 7:

Lecture 13: Large-sample properties of MLEs

  • Central limit theorem.
  • Consistency, efficiency, and asymptotic normality of MLEs.
  • Fisher’s approximation.

Lecture 14: Interval estimation.

  • Credible intervals.
  • Confidence intervals.
  • Pivotal quantities and the construction of confidence intervals.

Week 8 (Midterm 2):

Lecture 15: Introduction to Hypothesis Testing (Section 9.1)

  • Test basics: rejection region, test statistics, type I and II errors.
  • Testing simple hypotheses
  • NP paradigm/lemma and likelihood-ratio tests

Lecture 16: Composite Hypotheses and Generalized Likelihood-Ratio Tests (Section 9.2-9.3)

  • Testing composite hypotheses
  • Generalized likelihood-ratio tests
  • Testing normal mean—one-sample, two-sample t-test and paired t-test.

Week 9:

Lecture 17: More on Hypothesis Testing (Section 9.4)

  • P-values and their interpretation
  • Multiple-testing control

Lecture 18: Goodness-of-fit tests for Multinomial Data (Section 9.5)

  • Large-sample approximation for generalized LR tests
  • Chi-squared test
  • Tests for independence in contingency tables

Textbooks

Required: Rice, John A. Mathematical Statistics and Data Analysis, 3rd Edition.

Additional References:

Assessment

All exams will be closed-book and closed-notes. No electronics are allowed and hence the use of the internet or genAI tools are prohibited. A formula sheet will be provided. A letter-sized cheat-sheet (one-sided for the midterms and double-sided for the final) may be brought to the exams.

Course Policies