2
$\begingroup$

There's a markov chain and transition matrix:

$$ \begin{bmatrix} 1/2 & 1/4 & 1/4 & 0 & 0 \\ 0 & 1/2 & 1/8 & 3/8 & 0 \\ 0 & 0 & 0 & 3/4 & 1/4 \\ 0 & 1/2 & 0 & 1/4 & 1/4 \\ 0 & 0 & 0 & 0 & 1 \end{bmatrix} $$

I need to find $$ P(X_4 = 4 \mid X_0 = 1, X_1 = 3, X_2 = 4) $$ and $$ P(X_7 = 1 \mid X_0 = 5). $$

If I understand correctly, $ P(X_4 = 4 \mid X_0 = 1, X_1 = 3, X_2 = 4) = P(X_4 = 4 \mid X_2 = 4) $ correct me if I'm wrong.

I also know this formula: $ p^{(m)}_{ij} = P(X_{(t+m)} = j \mid X_{(t)} = i) $, the probability that we move from state i to state j in m steps.

So if i used this, it would be: $ P(X_4 = 4 \mid X_2 = 4) = p^{2}_{4 4} $ and $P(X_7 = 1 \mid X_0 = 5) = p^{7}_{5 1}$. But I do not know where to go from here. How do I calculate $p^{2}_{4 4}$ and $ p^{7}_{5 1} $? Are there any other ways I could solve this?

$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Neither of your formulas are correct. You have to account for all paths between two states separated by more than one step.

Thankfully, whoever came up with these questions was merciful:

$$P(X_4=4|X_2=4) = p_{44}p_{44}+p_{42}p_{24}+p_{45}p_{54}=\left(\frac14*\frac14\right)+\left(\frac12 * \frac38\right)+\left(\frac14*0 \right)$$ $$=\frac{1}{16}+\frac{3}{16}=\frac14$$

Also $P(X_7=1|X_0=5)$ is super easy: notice that the last row of the transition matrix shows that $p_{55}=1$ so no other transitions are possible (it is an absorbing state), therefore:

$$P(X_7=1|X_0=5) = 0$$

$\endgroup$
2
  • $\begingroup$ The $n$-step transition probabilities appear as entries in $n^{\rm th}$ power of the transition matrix. Call the transition matrix $\Bbb P$.The $m$-step transition probabilities appear as the entries of $m^{\rm th}$ power of the transition matrix. More precisely, call the transition matrix $\Bbb P$. Then $$ p^{(m)}_{ij} = [\Bbb P^m]_{ij}. $$ As noted by philo_777, row 5 of $\Bbb P^m$ can be seen by inspection, because state $5$ is absorbing. $\endgroup$ Commented Jan 24 at 16:55
  • 1
    $\begingroup$ @JohnDawkins -- correct -- I was just going directly for it but you are correct and that is how you'd do it sysematically. $\endgroup$ Commented Jan 24 at 17:40

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.