From e6f125b3aa47115e63e6c8f71056f631032d0a38 Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Thu, 27 Jun 2024 15:46:09 +1000 Subject: [PATCH 1/8] Update made to Markov Chains: Basic Concepts lecture --- lectures/markov_chains_I.md | 58 +++++++++++++++++++++++-------------- 1 file changed, 36 insertions(+), 22 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index c50d472e..6b35f98b 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -4,7 +4,7 @@ jupytext: extension: .md format_name: myst format_version: 0.13 - jupytext_version: 1.14.4 + jupytext_version: 1.16.1 kernelspec: display_name: Python 3 (ipykernel) language: python @@ -76,7 +76,7 @@ nonnegative $n$-vector $p$ that sums to one. For example, $p = (0.2, 0.2, 0.6)$ is a probability mass function over $3$ outcomes. A **stochastic matrix** (or **Markov matrix**) is an $n \times n$ square matrix $P$ -such that each row of $P$ is a probability mass function over $n$ outcomes. +such that each row of $P$ is a probability mass function. In other words, @@ -96,7 +96,7 @@ Before defining a Markov chain rigorously, we'll give some examples. (mc_eg2)= -#### Example 1 +#### Example 1: Economic States From US unemployment data, Hamilton {cite}`Hamilton2005` estimated the following dynamics. @@ -172,7 +172,7 @@ In particular, $P(i,j)$ is the (mc_eg1)= -#### Example 2 +#### Example 2: Unemployment Consider a worker who, at any given time $t$, is either unemployed (state 0) or employed (state 1). @@ -220,7 +220,7 @@ Then we can address a range of questions, such as We'll cover some of these applications below. (mc_eg3)= -#### Example 3 +#### Example 3: Political Transition Dynamics Imam and Temple {cite}`imampolitical` categorize political institutions into three types: democracy $\text{(D)}$, autocracy $\text{(A)}$, and an intermediate @@ -231,17 +231,17 @@ Each institution can have two potential development regimes: collapse $\text{(C) Imam and Temple {cite}`imampolitical` estimate the following transition probabilities: - $$ -P := -\begin{bmatrix} -0.86 & 0.11 & 0.03 & 0.00 & 0.00 & 0.00 \\ -0.52 & 0.33 & 0.13 & 0.02 & 0.00 & 0.00 \\ -0.12 & 0.03 & 0.70 & 0.11 & 0.03 & 0.01 \\ -0.13 & 0.02 & 0.35 & 0.36 & 0.10 & 0.04 \\ -0.00 & 0.00 & 0.09 & 0.11 & 0.55 & 0.25 \\ -0.00 & 0.00 & 0.09 & 0.15 & 0.26 & 0.50 -\end{bmatrix} +\begin{array}{c|cccccc} + & \text{DG} & \text{DC} & \text{NG} & \text{NC} & \text{AG} & \text{AC} \\ +\hline +\text{DG} & 0.86 & 0.11 & 0.03 & 0.00 & 0.00 & 0.00 \\ +\text{DC} & 0.52 & 0.33 & 0.13 & 0.02 & 0.00 & 0.00 \\ +\text{NG} & 0.12 & 0.03 & 0.70 & 0.11 & 0.03 & 0.01 \\ +\text{NC} & 0.13 & 0.02 & 0.35 & 0.36 & 0.10 & 0.04 \\ +\text{AG} & 0.00 & 0.00 & 0.09 & 0.11 & 0.55 & 0.25 \\ +\text{AC} & 0.00 & 0.00 & 0.09 & 0.15 & 0.26 & 0.50 \\ +\end{array} $$ ```{code-cell} ipython3 @@ -285,6 +285,20 @@ plt.colorbar(pc, ax=ax) plt.show() ``` +The probabilities can be represented in matrix form as follows + +$$ +P := +\begin{bmatrix} +0.86 & 0.11 & 0.03 & 0.00 & 0.00 & 0.00 \\ +0.52 & 0.33 & 0.13 & 0.02 & 0.00 & 0.00 \\ +0.12 & 0.03 & 0.70 & 0.11 & 0.03 & 0.01 \\ +0.13 & 0.02 & 0.35 & 0.36 & 0.10 & 0.04 \\ +0.00 & 0.00 & 0.09 & 0.11 & 0.55 & 0.25 \\ +0.00 & 0.00 & 0.09 & 0.15 & 0.26 & 0.50 +\end{bmatrix} +$$ + Looking at the data, we see that democracies tend to have longer-lasting growth regimes compared to autocracies (as indicated by the lower probability of transitioning from growth to growth in autocracies). @@ -308,7 +322,7 @@ A **distribution** $\psi$ on $S$ is a probability mass function of length $n$, w A **Markov chain** $\{X_t\}$ on $S$ is a sequence of random variables taking values in $S$ that have the **Markov property**. -This means that, for any date $t$ and any state $y \in S$, +This means that, for any time $t$ and any state $y \in S$, ```{math} :label: fin_markov_mp @@ -331,7 +345,7 @@ P(x, y) := \mathbb P \{ X_{t+1} = y \,|\, X_t = x \} By construction, * $P(x, y)$ is the probability of going from $x$ to $y$ in one unit of time (one step) -* $P(x, \cdot)$ is the conditional distribution of $X_{t+1}$ given $X_t = x$ +* $P(x, \cdot)$ is the conditional distribution(probability mass function) of $X_{t+1}$ given $X_t = x$ We can view $P$ as a stochastic matrix where @@ -437,7 +451,7 @@ Here's a short time series. mc_sample_path(P, ψ_0=(1.0, 0.0), ts_length=10) ``` -It can be shown that for a long series drawn from `P`, the fraction of the +It can be proven that for a long series drawn from `P`, the fraction of the sample that takes value 0 will be about 0.25. (We will explain why {ref}`later `.) @@ -610,7 +624,7 @@ $$ ```{index} single: Markov Chains; Future Probabilities ``` -Recall the stochastic matrix $P$ for recession and growth {ref}`considered above `. +Recall the stochastic matrix $P$ for recession and growth considered in {ref}`Example 1: Economic States `. Suppose that the current state is unknown --- perhaps statistics are available only at the *end* of the current month. @@ -632,7 +646,7 @@ The distributions we have been studying can be viewed either 1. as probabilities or 1. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. -To illustrate, recall our model of employment/unemployment dynamics for a given worker {ref}`discussed above `. +To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {ref}`Example 2: Unemployment `. Consider a large population of workers, each of whose lifetime experience is described by the specified dynamics, with each worker's outcomes being @@ -726,7 +740,7 @@ We will come back to this when we introduce irreducibility in the {doc}`next lec ### Example -Recall our model of the employment/unemployment dynamics of a particular worker {ref}`discussed above `. +Recall our model of the employment/unemployment dynamics of a particular worker discussed in {ref}`Example 2: Unemployment `. If $\alpha \in (0,1)$ and $\beta \in (0,1)$, then the transition matrix is everywhere positive. @@ -840,7 +854,7 @@ HTML(anim.to_jshtml()) Here -* $P$ is the stochastic matrix for recession and growth {ref}`considered above `. +* $P$ is the stochastic matrix for recession and growth considered in {ref}`Example 1: Economic States `. * The highest red dot is an arbitrarily chosen initial marginal probability distribution $\psi_0$, represented as a vector in $\mathbb R^3$. * The other red dots are the marginal distributions $\psi_0 P^t$ for $t = 1, 2, \ldots$. * The black dot is $\psi^*$. From 29172792ca99c41f0bfa4e6670ca038de6f8e49d Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Sun, 30 Jun 2024 22:01:43 +1000 Subject: [PATCH 2/8] update to markov chain I lecture --- lectures/markov_chains_I.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 6b35f98b..4280b5bb 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -96,7 +96,7 @@ Before defining a Markov chain rigorously, we'll give some examples. (mc_eg2)= -#### Example 1: Economic States +#### Example 1: Economic states From US unemployment data, Hamilton {cite}`Hamilton2005` estimated the following dynamics. @@ -220,7 +220,7 @@ Then we can address a range of questions, such as We'll cover some of these applications below. (mc_eg3)= -#### Example 3: Political Transition Dynamics +#### Example 3: Political transition dynamics Imam and Temple {cite}`imampolitical` categorize political institutions into three types: democracy $\text{(D)}$, autocracy $\text{(A)}$, and an intermediate @@ -624,7 +624,7 @@ $$ ```{index} single: Markov Chains; Future Probabilities ``` -Recall the stochastic matrix $P$ for recession and growth considered in {ref}`Example 1: Economic States `. +Recall the stochastic matrix $P$ for recession and growth considered in {prf:ref}`Example 1: Economic states `. Suppose that the current state is unknown --- perhaps statistics are available only at the *end* of the current month. @@ -646,7 +646,7 @@ The distributions we have been studying can be viewed either 1. as probabilities or 1. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. -To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {ref}`Example 2: Unemployment `. +To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {prf:ref}`Example 2: Unemployment `. Consider a large population of workers, each of whose lifetime experience is described by the specified dynamics, with each worker's outcomes being @@ -740,7 +740,7 @@ We will come back to this when we introduce irreducibility in the {doc}`next lec ### Example -Recall our model of the employment/unemployment dynamics of a particular worker discussed in {ref}`Example 2: Unemployment `. +Recall our model of the employment/unemployment dynamics of a particular worker discussed in {prf:ref}`Example 2: Unemployment `. If $\alpha \in (0,1)$ and $\beta \in (0,1)$, then the transition matrix is everywhere positive. @@ -854,7 +854,7 @@ HTML(anim.to_jshtml()) Here -* $P$ is the stochastic matrix for recession and growth considered in {ref}`Example 1: Economic States `. +* $P$ is the stochastic matrix for recession and growth considered in {prf:ref}`Example 1: Economic states `. * The highest red dot is an arbitrarily chosen initial marginal probability distribution $\psi_0$, represented as a vector in $\mathbb R^3$. * The other red dots are the marginal distributions $\psi_0 P^t$ for $t = 1, 2, \ldots$. * The black dot is $\psi^*$. From 03d6f37c43f05ec709583cd9cfbd61a3f3637e2c Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Thu, 4 Jul 2024 15:58:21 +1000 Subject: [PATCH 3/8] cross-sectional frequency explaination added --- lectures/markov_chains_I.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 4280b5bb..5c147804 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -646,6 +646,10 @@ The distributions we have been studying can be viewed either 1. as probabilities or 1. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. + ```{note} +A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information of the proportions of individuals in each possible state of that variable. +``` + To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {prf:ref}`Example 2: Unemployment `. Consider a large population of workers, each of whose lifetime experience is From 6a0f2e2e308273edc6c204015fbc2cd67c838592 Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Thu, 4 Jul 2024 16:12:43 +1000 Subject: [PATCH 4/8] change prf to ref --- lectures/markov_chains_I.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 5c147804..c5c7de6f 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -624,7 +624,7 @@ $$ ```{index} single: Markov Chains; Future Probabilities ``` -Recall the stochastic matrix $P$ for recession and growth considered in {prf:ref}`Example 1: Economic states `. +Recall the stochastic matrix $P$ for recession and growth considered in {ref}`Example 1: Economic states `. Suppose that the current state is unknown --- perhaps statistics are available only at the *end* of the current month. @@ -650,7 +650,7 @@ The distributions we have been studying can be viewed either A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information of the proportions of individuals in each possible state of that variable. ``` -To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {prf:ref}`Example 2: Unemployment `. +To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {ref}`Example 2: Unemployment `. Consider a large population of workers, each of whose lifetime experience is described by the specified dynamics, with each worker's outcomes being @@ -744,7 +744,7 @@ We will come back to this when we introduce irreducibility in the {doc}`next lec ### Example -Recall our model of the employment/unemployment dynamics of a particular worker discussed in {prf:ref}`Example 2: Unemployment `. +Recall our model of the employment/unemployment dynamics of a particular worker discussed in {ref}`Example 2: Unemployment `. If $\alpha \in (0,1)$ and $\beta \in (0,1)$, then the transition matrix is everywhere positive. @@ -858,7 +858,7 @@ HTML(anim.to_jshtml()) Here -* $P$ is the stochastic matrix for recession and growth considered in {prf:ref}`Example 1: Economic states `. +* $P$ is the stochastic matrix for recession and growth considered in {ref}`Example 1: Economic states `. * The highest red dot is an arbitrarily chosen initial marginal probability distribution $\psi_0$, represented as a vector in $\mathbb R^3$. * The other red dots are the marginal distributions $\psi_0 P^t$ for $t = 1, 2, \ldots$. * The black dot is $\psi^*$. From 060bb8a4bdb7aac2074a9a3069d36b65d2a3ec01 Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Mon, 15 Jul 2024 15:38:42 +1000 Subject: [PATCH 5/8] [markov_chain_I] Move example subsections to sphinx-proof examples --- lectures/markov_chains_I.md | 48 ++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index c5c7de6f..56976e5e 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -619,43 +619,43 @@ $$ $$ -### Example: probability of recession - ```{index} single: Markov Chains; Future Probabilities ``` +```{prf:example} Probability of Recession +:label: prob-recesession + Recall the stochastic matrix $P$ for recession and growth considered in {ref}`Example 1: Economic states `. -Suppose that the current state is unknown --- perhaps statistics are available only at the *end* of the current month. +Suppose that the current state is unknown — perhaps statistics are available only at the *end* of the current month. -We guess that the probability that the economy is in state $x$ is $\psi_t(x)$ at time t. +We guess that the probability that the economy is in state $x$ is $\psi_t(x)$ at time $t$. -The probability of being in recession (either mild or severe) in 6 months time is given by +The probability of being in recession (either mild or severe) in 6 months' time is given by $$ (\psi_t P^6)(1) + (\psi_t P^6)(2) $$ +```{index} single: Markov Chains; Cross-Sectional Distributions +``` +```{prf:example} Cross-Sectional Distributions +:label: cross-sectional-distributions - -(mc_eg1-1)= -### Example 2: cross-sectional distributions +### Example 2: Cross-Sectional Distributions The distributions we have been studying can be viewed either 1. as probabilities or -1. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. +2. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. ```{note} -A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information of the proportions of individuals in each possible state of that variable. +A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information on the proportions of individuals in each possible state of t variable. ``` To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {ref}`Example 2: Unemployment `. -Consider a large population of workers, each of whose lifetime experience is -described by the specified dynamics, with each worker's outcomes being -realizations of processes that are statistically independent of all other -workers' processes. +Consider a large population of workers, each of whose lifetime experience is described by the specified dynamics, with each worker's outcomes being realizations of processes that are statistically independent of all other workers' processes. Let $\psi_t$ be the current *cross-sectional* distribution over $\{ 0, 1 \}$. @@ -665,29 +665,23 @@ The cross-sectional distribution records fractions of workers employed and unemp What will the cross-sectional distribution be in 10 periods hence? -The answer is $\psi_t P^{10}$, where $P$ is the stochastic matrix in -{eq}`p_unempemp`. - -This is because each worker's state evolves according to $P$, so -$\psi_t P^{10}$ is a [marginal distribution](https://en.wikipedia.org/wiki/Marginal_distribution) for a single randomly selected -worker. +The answer is $\psi_t P^{10}$, where $P$ is the stochastic matrix in {eq}`p_unempemp`. -But when the sample is large, outcomes and probabilities are roughly equal (by an application of the law -of large numbers). +This is because each worker's state evolves according to $P$, so $\psi_t P^{10}$ is a [marginal distribution](https://en.wikipedia.org/wiki/Marginal_distribution) for a single randomly selected worker. -So for a very large (tending to infinite) population, -$\psi_t P^{10}$ also represents fractions of workers in -each state. +But when the sample is large, outcomes and probabilities are roughly equal (by an application of the law of large numbers). -This is exactly the cross-sectional distribution. +So for a very large (tending to infinite) population, $\psi_t P^{10}$ also represents fractions of workers in each state. -(stationary)= +This is exactly the(stationary)= ## Stationary distributions As seen in {eq}`fin_mc_fr`, we can shift a distribution forward one unit of time via postmultiplication by $P$. +Some distributions are invariant under this updating process --- for example,nit of time via postmultiplication by $P$. + Some distributions are invariant under this updating process --- for example, ```{code-cell} ipython3 From 112e3c55c9d29243f06e82d0ca93b53c6f9439f2 Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Mon, 15 Jul 2024 15:51:54 +1000 Subject: [PATCH 6/8] minor update --- lectures/markov_chains_I.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 56976e5e..2c13cc17 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -639,6 +639,9 @@ $$ ```{index} single: Markov Chains; Cross-Sectional Distributions ``` +```{index} single: Markov Chains; Cross-Sectional Distributions +``` + ```{prf:example} Cross-Sectional Distributions :label: cross-sectional-distributions @@ -649,10 +652,6 @@ The distributions we have been studying can be viewed either 1. as probabilities or 2. as cross-sectional frequencies that the law of large numbers leads us to anticipate for large samples. - ```{note} -A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information on the proportions of individuals in each possible state of t variable. -``` - To illustrate, recall our model of employment/unemployment dynamics for a given worker discussed in {ref}`Example 2: Unemployment `. Consider a large population of workers, each of whose lifetime experience is described by the specified dynamics, with each worker's outcomes being realizations of processes that are statistically independent of all other workers' processes. @@ -673,15 +672,21 @@ But when the sample is large, outcomes and probabilities are roughly equal (by a So for a very large (tending to infinite) population, $\psi_t P^{10}$ also represents fractions of workers in each state. -This is exactly the(stationary)= +This is exactly the cross-sectional distribution. + + ```{note} +A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information on the proportions of individuals in each possible state of that variable. +``` + + + +(stationary)= ## Stationary distributions As seen in {eq}`fin_mc_fr`, we can shift a distribution forward one unit of time via postmultiplication by $P$. -Some distributions are invariant under this updating process --- for example,nit of time via postmultiplication by $P$. - Some distributions are invariant under this updating process --- for example, ```{code-cell} ipython3 From 088d92b958b53f5a2719397cdf9c5e71d8e187da Mon Sep 17 00:00:00 2001 From: mmcky Date: Tue, 16 Jul 2024 08:37:53 +1000 Subject: [PATCH 7/8] fix syntax for example --- lectures/markov_chains_I.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 2c13cc17..2948239c 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -636,17 +636,15 @@ The probability of being in recession (either mild or severe) in 6 months' time $$ (\psi_t P^6)(1) + (\psi_t P^6)(2) $$ -```{index} single: Markov Chains; Cross-Sectional Distributions + ``` ```{index} single: Markov Chains; Cross-Sectional Distributions ``` -```{prf:example} Cross-Sectional Distributions +````{prf:example} Cross-Sectional Distributions :label: cross-sectional-distributions -### Example 2: Cross-Sectional Distributions - The distributions we have been studying can be viewed either 1. as probabilities or @@ -674,16 +672,15 @@ So for a very large (tending to infinite) population, $\psi_t P^{10}$ also repre This is exactly the cross-sectional distribution. - ```{note} +```{note} A cross-sectional frequency measures how a particular variable (e.g., employment status) is distributed across a population at a specific time, providing information on the proportions of individuals in each possible state of that variable. ``` - +```` (stationary)= ## Stationary distributions - As seen in {eq}`fin_mc_fr`, we can shift a distribution forward one unit of time via postmultiplication by $P$. @@ -698,8 +695,6 @@ P = np.array([[0.4, 0.6], Notice that `ψ @ P` is the same as `ψ`. - - Such distributions are called **stationary** or **invariant**. (mc_stat_dd)= @@ -753,7 +748,7 @@ corresponds to unemployment (state 0). Using $\psi^* = \psi^* P$ and a bit of algebra yields $$ - p = \frac{\beta}{\alpha + \beta} +p = \frac{\beta}{\alpha + \beta} $$ This is, in some sense, a steady state probability of unemployment. From ab23359a9d99d7b2453f165044b863ee16379746 Mon Sep 17 00:00:00 2001 From: Jiarui Zhang Date: Mon, 5 Aug 2024 12:38:15 +1000 Subject: [PATCH 8/8] final update to usage of syntax for example --- lectures/markov_chains_I.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/lectures/markov_chains_I.md b/lectures/markov_chains_I.md index 2948239c..1dbcb41e 100644 --- a/lectures/markov_chains_I.md +++ b/lectures/markov_chains_I.md @@ -735,9 +735,7 @@ distribution. We will come back to this when we introduce irreducibility in the {doc}`next lecture ` on Markov chains. - -### Example - +```{prf:example} Steady-State Unemployment Probability Recall our model of the employment/unemployment dynamics of a particular worker discussed in {ref}`Example 2: Unemployment `. If $\alpha \in (0,1)$ and $\beta \in (0,1)$, then the transition matrix is everywhere positive. @@ -754,6 +752,7 @@ $$ This is, in some sense, a steady state probability of unemployment. Not surprisingly it tends to zero as $\beta \to 0$, and to one as $\alpha \to 0$. +``` ### Calculating stationary distributions