Statistics tutorials and learning resources

Statistics learning and data analysis resources. Please, contribute and get in touch! See MDmisc notes for other programming and genomics-related notes.

Table of content

• Cheatsheets
• Bayesian
• Mixed models
• Repositories
• Courses
• Videos
• Books
• Linear algebra
• Misc

Cheatsheets

• probability_cheatsheet - A comprehensive 10-page PDF probability cheatsheet that covers a semester's worth of introduction to probability, by William Chen (http://wzchen.com) and Joe Blitzstein (http://stat110.net).

• Common statistical tests are linear models (or: how to teach stats)

• Resources for learning about the history of statistics and statisticians. By statisticians, for statisticians - references to blog posts, books, journal articles, podcasts, interviews, news, and other material about the history of statistics

• Distribution explorer - Overview and examples of various distributions (univariate/multivariate, categorical/continuous), their parameters, interdependencies. Code in numpy, scipy, Stan.

Bayesian

• Bayesian statistics and modeling primer. Methods and applications overview, terminology description. Prior distributions, elicitation, uncertainty. Model fitting using MCMC, other methods (Table 1). Applications in behavioural sciences, ecology, genetics. Reproducibility considerations. Table 2 - Bayesian inference software, various OSs and languages. Box 1 - Bayes theorem explanation. Box 2 - Bayes factors. Box 3 - likelihood function. Box 5 - WAMBS (when to Worry and how to Avoid the Misuse of Bayesian Statistics) checklist. Many references.
  Paper

  Schoot, Rens van de, Sarah Depaoli, Ruth King, Bianca Kramer, Kaspar Märtens, Mahlet G Tadesse, Marina Vannucci, et al. “Bayesian Statistics and Modelling,” 2021, 26. https://doi.org/10.1038/ s43586-020-00001-2

• stat_rethinking_2022 - Statistical Rethinking book and course, by Richard McElreath, 2022 edition. Slides, videos. Original rethinking GitHub repo. Andrew Gelman's note about the book, Video lectures.

  • Statistical-Rethinking - An interactive online reading of McElreath's "Statistical Rethinking: A Bayesian Course with Examples in R and Stan" by Levi Waldron. https://github.com/lwaldron/Statistical-Rethinking

• A Brief Introduction to Graphical Models and Bayesian Networks, by Kevin Murphy, 1998.

• A Student's Guide to Bayesian Statistics, by Ben Lambert . More Bayesian Class Videos

• A Bayesian Course with Examples in R and Stan book sample. Link to the full video lectures

• Bayesian Methods for Hackers - An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view

• Bayesian Data Analysis demos for Python

• Bayesian-Modelling-in-Python - A Python tutorial on bayesian modeling techniques (PyMC3)

• Bayes Rules! An Introduction to Bayesian Modeling with R by Alicia A. Johnson, Miles Ott, Mine Dogucu. Tweet

• Bayesian Computing Course - Python notebooks with applied examples and explanations

• Probabilistic-Programming-and-Bayesian-Methods-for-Hackers - An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python.

• Bayesian Data Analysis at Aalto (CS-E5710) course material, slides, video lectures, code demos, assignments

• awesome-bayes - List of resources for bayesian inference.

• bayesian-basics/ - Bayesian data analysis introduction by Michael Clarl. GitHub

• book - a written companion for the Course 'Bayesian Statistics' from the Statistics with R specialization available on Coursera.

• understanding-bayes/ - "Understanding Bayes" series of blog posts by Alex Etz. The other posts are also worth reading.

Mixed models

• Mixed Models with R, Getting started with random effects, by Michael Clark. GitHub, Tweet.

• How Linear Mixed Model Works by Nikolay Oskolkov

• Introduction to linear mixed models - mixed and random effects, R syntax. GitHub

• CC-Linear-mixed-models - Introduction to linear mixed models, https://ourcodingclub.github.io/tutorials/mixed-models/, https://github.com/ourcodingclub/CC-Linear-mixed-models

• julianfaraway/rexamples - mixed effect model examples from two chapters of Julian Faraway's book Extending the Linear Model with R. Each model is fit using several different methods: lme4, INLA, STAN, BRMS, MGCV. Another useful book: Bayesian Regression Modeling with INLA

Repositories

• brr - "Biostatistics for Biomedical Research" by Frank Harrell, the creator of Hmisc package and many more. https://github.com/harrelfe/bbr. Video lectures, https://www.youtube.com/channel/UC-o_ZZ0tuFUYn8e8rf-QURA

• BIOS2 - Biostatistics 621 / 821 course by Levi Waldron. Classical statistics, from all aspects of regression, survival analysis to dimensionality reduction basics. iPython and R. https://github.com/waldronlab/BIOS2

• book_sample - Another Book on Data Science. Learn R and Python in Parallel. Web, https://www.anotherbookondatascience.com/, GitHub, https://github.com/rnorm/book_sample

• bysh_book - Repo for Feb 2018 version of Broadening Your Statistical Horizons https://github.com/broadenyourstatisticalhorizons/bysh_book. The rendered version can be found at: https://bookdown.org/roback/bookdown-bysh/

• CHE379 - Statistical refresher course by Chris A. Mack, From Data to Decisions: Measurement, Uncertainty, Analysis, and Modeling. Videos, esercises, slides in PDF. http://www.lithoguru.com/scientist/statistics/course.html. Video playlist https://www.youtube.com/playlist?list=PLM2eE_hI4gSDnF-mEa9mrIYx7GCLQVN89

• DATA606Spring2017 - the DATA 606 course for the Spring 2017 semester by Jason Bryer. The course website is at data606.net. https://github.com/jbryer/DATA606Spring2017

-FES - Feature Engineering and Selection: A Practical Approach for Predictive Models, by Max Kuhn and Kjell Johnson. http://www.feat.engineering/, [https://github.com/topepo/FES(https://github.com/topepo/FES)]

• fiveMinuteStats - A repo of short "vignettes" illustrating statistical concepts, https://stephens999.github.io/fiveMinuteStats/. https://github.com/stephens999/fiveMinuteStats.git

• Intro2R - data mining and machine learning, https://github.com/johnros/Intro2R

• ISAT 251 Intro to Statistics with R - basic statistics by Nicole Radziwill

• ISLR - An Introduction to Statistical Learning with Applications in R (ISLR). The book, R code and the data are available at http://www-bcf.usc.edu/~gareth/ISL/. Videos and slides are at https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/. Slides are also available at https://www.alsharif.info/iom530

• Kalman-and-Bayesian-Filters-in-Python - Kalman Filter book using Jupyter Notebook. Focuses on building intuition and experience, not formal proofs. Includes Kalman filters,extended Kalman filters, unscented Kalman filters, particle filters, and more. All exercises include solutions. https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python

• OpenIntro-Statistics - An open-source textbook written at the college level. OpenIntro also offers a second college-level intro stat textbook and also a high school variant. https://www.openintro.org, https://github.com/OpenIntroOrg/openintro-statistics. Videos, https://www.youtube.com/user/bleue894/playlists, slides, https://github.com/OpenIntroStat/openintro-statistics-slides

• practicing_R - R and statistics, https://github.com/johnros/practicing_R

• PractitionerGuidetoMultiplicity - Practical Guide for Multiple testing, https://github.com/johnros/PractitionerGuidetoMultiplicity

• stat540_2014 - STAT540 Statistical Methods for High Dimensional Biology course by Jenny Bryan

• stat401A - conscise statistical refresher by Jarad Niemi, STAT 401A course at Iowa State University. https://github.com/jarad/stat401A. Jarad's web-site with more statistical courses STAT 544 and STAT 615 http://www.jarad.me/courses/

• statcomp - Statistical Computing, BIOS 735 - Introduction to Statistical Computing. http://biodatascience.github.io/statcomp, https://github.com/biodatascience/statcomp, https://github.com/biodatascience/statcomp_src

• thinkstats - Statistical Thinking for the 21st Century. Book, http://statsthinking21.org/, and GitHub, https://github.com/psych10/thinkstats

• WinVector.github.io - Various statistical topics with R and Python examples. "IntroductionToDataScience" course. https://winvector.github.io/ - web-facing pages, https://github.com/WinVector/WinVector.github.io/ - github repo

• www_stat_cmu_edu_cshalizi_350 - Statistics 36-350: Data Mining by Cosma Shalizi. http://www.stat.cmu.edu/~cshalizi/350/

Courses

• fastStat - Quick introduction to statistics for those with a probability background, by Norm Matloff. Also, fastBigStat

• intro-gam-webinar-2020 - Introduction to Generalized Additive Models with R and mgcv, by Gavin Simpson. Video, slides, code

• Introduction to Causal Inference Fall 2020 course by Brady Neal. Video, lecture material. Twitter

• BIOS 735 - Introduction to Statistical Computing - Statistical concepts in R, by Naim Rashid. GitHub

• CS229T/STATS231: Statistical Learning Theory, Stanford / Autumn 2018-2019. "Texts and References" section has a good set of course notes and links. https://web.stanford.edu/class/cs229t/

• The Coursera Class: Statistics One, by Princeton https://github.com/svkerr/Statistics_Class_Princeton

• Introduction to Statistics for Biologists, by Peter Ralph. https://github.com/petrelharp/bisc305

• Introduction to Probability and Statistics, stat20.org. GitHub

• Data wrangling, exploration, and analysis with R, UBC STAT 545A and 547M courses. https://stat545-ubc.github.io/ and the Git repository https://github.com/STAT545-UBC/STAT545-UBC.github.io

• Harvard CS 109: Data Science course with video lectures, pdf slides and iPython notebooks. Course overview, Videos, Git repository, All course-related material on Github

• Advanced data analysis techniques, CVEN 6833, Dr. R. Balaji. Lots of material and links on regression and modeling techniques. http://civil.colorado.edu/~balajir/CVEN6833/

• Econometrics Academy - main statistical methods, examples in R and SAS, short video tutorials

Videos

• MIT 18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Spring 2018, by Gilbert Strang, https://www.youtube.com/playlist?list=PLUl4u3cNGP63oMNUHXqIUcrkS2PivhN3k

• Statistical inference for data science, Brian Caffo. Full book on http://rpubs.com/cbchisanga/143127, videos on https://www.youtube.com/watch?v=WkOinijQmPU&list=PLpl-gQkQivXiBmGyzLrUjzsblmQsLtkzJ, GitHub version on https://github.com/bcaffo/LittleInferenceBook

• Statistics 110: Probability, by Joe Blitzstein, https://www.youtube.com/playlist?list=PL2SOU6wwxB0uwwH80KTQ6ht66KWxbzTIo. The book "Introduction to Probability" https://twitter.com/stat110/status/1101502622358556674

Books

• Improving Your Statistical Inferences by Daniël Lakens. Website

• The Effect: An Introduction to Research Design and Causality - book by Nick Huntington-Klein intended to introduce the concepts of research design and causality in the context of observational data. GitHub, accompanying Videos.

• Gaussian Processes for Machine Learning - book by Carl Edward Rasmussen and Christopher K. I. Williams.

• An Introduction to Statistical Learning - classic stats learning book by Gareth James, Daniela Witten, Trevor Hastie, Rob Tibshirani

• practical-statistics-for-data-scientists - Practical Statistics for Data Scientists, 50+ Essential Concepts Using R and Python, by Peter Bruce, Andrew Bruce, and Peter Gedeck. Code repository for O'Reilly book

• Probabilistic Machine Learning - a book series by Kevin Murphy. GitHub repo with links to buy and PDF download

• Forecasting: Principles and Practice by Rob J Hyndman and George Athanasopoulos, Monash University, Australia. R-based examples. Tweet

• "Advanced Statistical Computing" by Roger D. Peng. https://bookdown.org/rdpeng/advstatcomp/

• "Modern Statistics for Modern Biology" book by Susan Holmes and Wolfgang Huber. Data and code provided. https://www.huber.embl.de/msmb/index.html

• "Causal Inference Book" by Miguel Hernan and Jamie Robins. https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/

• Notes and exercises for a classical free book "An Introduction to Statistical Learning" https://github.com/asadoughi/stat-learning

• "From Algorithms to Z-Scores: Probabilistic and Statistical Modeling in Computer Science" book, http://heather.cs.ucdavis.edu/probstatbook by prof. Norm Matloff

• "Introduction to probability" introductory book, published by the American Mathematical Society. Web-page and references therein, PDF.

• "Modeling and Solving Linear Programming with R" open book introducting linear modeling, in R. Book, PDFm Git with corresponding code

• "An introduction to psychometric theory with applications in R" by William Revelle, the creator of psych R package. The book with downloadable PDFs: http://personality-project.org/r/book/, the course based on the book: http://personality-project.org/revelle/syllabi/405.syllabus.html, and the psych R package: https://cran.r-project.org/web/packages/psych/index.html

• "Data Science Live Book" by Pablo Casas, from exploratory data analysis to regression/classification. https://livebook.datascienceheroes.com/

• Advanced Data Analysis from an Elementary Point of View book by Cosma Rohilla Shalizi, PDF, 828 pages. Statistics, with R examples

• Applied Statistics with R by David Dalpiaz. GitHub

• the_statistics_handbook - the statistics handbook open source repository.

Linear algebra

• The-Art-of-Linear-Algebra - Graphic notes on Gilbert Strang's "Linear Algebra for Everyone"

• Calculus resources, Python resources, Linear algebra resources by Brandon Rorher. Source

• "Introduction to Applied Linear Algebra" by Stephen Boyd & Lieven Vandenberghe, http://vmls-book.stanford.edu/, vmls.pdf. Includes examples in Julia

• numerical-linear-algebra - Computational Linear Algebra for Coders by FastAI. Video series, GitHub

Misc

• applied_statistics_2024 - Курс по прикладной статистике ВШЭ (2024).

• MANOVA(Multivariate Analysis of Variance) using R by Statistical Aid

• A Beginner’s Guide to Eigenvectors, PCA, Covariance and Entropy. http://deeplearning4j.org/eigenvector

• Goeman.pdf - STATISTICAL METHODS FOR MICROARRAY DATA

• MPR04.pdf - Introduction to Statistical Methods for Microarray Data Analysis

• How to Do Mediation Scientifically. https://blog.methodsconsultants.com/posts/how-to-do-mediation-scientifically/