Free Book: Probability and Statistics Cookbook

Free Book: Probability and Statistics Cookbook


The format is very similar to a BIG cheat sheet. This cookbook integrates a variety of topics in probability theory and statistics. It is based on literature and in-class material from courses of the statistics department at the University of California in Berkeley but also influenced by other sources . If you find errors or have suggestions for further topics, I would appreciate if you send me an email. 
Author: Matthias Vallentin

Contents
1 Distribution Overview 3

1.1 Discrete Distributions . . . . . . . . . . 3
1.2 Continuous Distributions . . . . . . . . 4

2 Probability Theory 6
3 Random Variables 6

3.1 Transformations . . . . . . . . . . . . . 7

4 Expectation 7
5 Variance 7
6 Inequalities 8
7 Distribution Relationships 8
8 Probability and Moment Generating Functions 9
9 Multivariate Distributions 9

9.1 Standard Bivariate Normal . . . . . . . 9
9.2 Bivariate Normal . . . . . . . . . . . . . 9
9.3 Multivariate Normal . . . . . . . . . . . 9

10 Convergence 9

10.1 Law of Large Numbers (LLN) . . . . . . 10
10.2 Central Limit Theorem (CLT) . . . . . 10

11 Statistical Inference 10

11.1 Point Estimation . . . . . . . . . . . . . 10
11.2 Normal-Based Confidence Interval . . . 11
11.3 Empirical distribution . . . . . . . . . . 11
11.4 Statistical Functionals . . . . . . . . . . 11

12 Parametric Inference 11

12.1 Method of Moments . . . . . . . . . . . 11
12.2 Maximum Likelihood . . . . . . . . . . . 12
12.2.1 Delta Method . . . . . . . . . . . 12
12.3 Multiparameter Models . . . . . . . . . 12
12.3.1 Multiparameter delta method . . 13
12.4 Parametric Bootstrap . . . . . . . . . . 13

13 Hypothesis Testing 13
14 Bayesian Inference 14

14.1 Credible Intervals . . . . . . . . . . . . . 14
14.2 Function of parameters . . . . . . . . . . 14
14.3 Priors . . . . . . . . . . . . . . . . . . . 15
14.3.1 Conjugate Priors . . . . . . . . . 15
14.4 Bayesian Testing . . . . . . . . . . . . . 15

15 Exponential Family 16
16 Sampling Methods 16

16.1 The Bootstrap . . . . . . . . . . . . . . 16
16.1.1 Bootstrap Confidence Intervals . 16
16.2 Rejection Sampling . . . . . . . . . . . . 17
16.3 Importance Sampling . . . . . . . . . . . 17

17 Decision Theory 17

17.1 Risk . . . . . . . . . . . . . . . . . . . . 17
17.2 Admissibility . . . . . . . . . . . . . . . 17
17.3 Bayes Rule . . . . . . . . . . . . . . . . 18
17.4 Minimax Rules . . . . . . . . . . . . . . 18

18 Linear Regression 18

18.1 Simple Linear Regression . . . . . . . . 18
18.2 Prediction . . . . . . . . . . . . . . . . . 19
18.3 Multiple Regression . . . . . . . . . . . 19
18.4 Model Selection . . . . . . . . . . . . . . 19

19 Non-parametric Function Estimation 20

19.1 Density Estimation . . . . . . . . . . . . 20
19.1.1 Histograms . . . . . . . . . . . . 20
19.1.2 Kernel Density Estimator (KDE) 21
19.2 Non-parametric Regression . . . . . . . 21
19.3 Smoothing Using Orthogonal Functions 21

20 Stochastic Processes 22

20.1 Markov Chains . . . . . . . . . . . . . . 22
20.2 Poisson Processes . . . . . . . . . . . . . 22

21 Time Series 23

21.1 Stationary Time Series . . . . . . . . . . 23
21.2 Estimation of Correlation . . . . . . . . 24
21.3 Non-Stationary Time Series . . . . . . . 24
21.3.1 Detrending . . . . . . . . . . . . 24
21.4 ARIMA models . . . . . . . . . . . . . . 24
21.4.1 Causality and Invertibility . . . . 25
21.5 Spectral Analysis . . . . . . . . . . . . . 25

22 Math 26

22.1 Gamma Function . . . . . . . . . . . . . 26
22.2 Beta Function . . . . . . . . . . . . . . . 26
22.3 Series . . . . . . . . . . . . . . . . . . . 27
22.4 Combinatorics . . . . . . . . . . . . . . 27

The most recent version of this document is available here. To read the PDF version, click here. . 
DSC Resources

Services: Hire a Data Scientist | Search DSC | Classifieds | Find a Job
Contributors: Post a Blog | Ask a Question
Follow us: @DataScienceCtrl | @AnalyticBridge

Popular Articles

Difference between Machine Learning, Data Science, AI, Deep Learning, and Statistics
What is Data Science? 24 Fundamental Articles Answering This Question
Hitchhiker’s Guide to Data Science, Machine Learning, R, Python
Advanced Machine Learning with Basic Excel


Link: Free Book: Probability and Statistics Cookbook