Stochastic Modelling – 2022
Assignment 1
Please complete the Plagiarism Declaration Form on the LMS before submitting this assign-
ment.
Submission Instructions:
Typed submissions (ideally using LATEX) are preferred. For handwritten solutions:
– Write your answers on blank paper. Write on one side of the paper only. Start each
question on a new page. Write the question number at the top of each page.
– Scan your solutions to a single PDF file with a mobile phone or a scanner. Scan
from directly above to avoid any excessive keystone effect. Check that all pages are
clearly readable and cropped to the A4 borders of the original page. Poorly scanned
or organised submissions may be impossible to mark or incur a penalty.
Upload the PDF file to Assignment 1 in Gradescope via the LMS. Gradescope will ask
you to identify on which of the uploaded pages your answers to each question are located.
Upload R code from Question 3(d) as instructed there.
The submission deadline is 5:00pm on Thursday, 8 September.
There are 3 questions, all of which will be marked. No marks will be given for answers without
clear and concise explanations. Clarity, neatness, and style count.
1. [13 marks] A discrete time Markov chain with state space S = {1, 2, 3, 4, 5, 6} has the
following transition matrix.
P =
????????
0 1/2 0 1/2 0 0
2/3 0 0 1/3 0 0
0 0 0 0 1/3 2/3
0 1 0 0 0 0
11/12 0 1/12 0 0 0
0 1/2 1/2 0 0 0
???????? .
(a) Write down the communication classes of the chain.
(b) Find the period of each communicating class.
(c) Determine which classes are essential.
(d) Classify each essential communicating class as transient or positive recurrent or null
recurrent.
(e) Describe the long run behaviour of the chain (including deriving long run probabil-
ities where appropriate).
(f) Given X0 = 4, find the long run proportion of time the chain spends in state j for
each j ∈ S.
(g) Find the expected number of steps taken for the chain to first reach state 4, given
the chain starts at state 3.
2. [10 marks] For fixed 0 < λ ≤ 1, an irreducible Markov chain on {0, 1, . . .} has transition
probabilities
pi,i+1 = 1? pi,i?1 = iλ+ 1
i(λ+ 1) + 2
, for i ≥ 1,
p0,1 = 1? p0,0 = 1/2.
(a) Find the values of λ where the chain is positive recurrent, null recurrent, and tran-
sient.
(b) Describe the long run behaviour of the chain.
3. [7 marks] A probability distribution supported on S = {?1, 1}N with parameters β ≥ 0
and h ∈ R is given by
pix ∝ exp
{
β
(
N?1∑
j=1
xjxj+1 + xNx1 + h
N∑
j=1
xj
)}
for x = (x1, . . . , xN) ∈ S.
Fix N = 500, let X = (X1, . . . , X500) be distributed according to pi, and set μβ,h :=
E[ 1
500
∑500
j=1Xj], the mean of the magnetisation.
(a) For x = (x1, . . . , xN) ∈ S and u ∈ {±1}, denote x(i,u) := (x1, . . . , xi?1, u, xi+1, . . . , xN).
For u ∈ {±1}, write a simple formula for the Gibbs sampler transition probability
that the chain moves to x(i,u) at the next step, given it is currently in x and coordi-
nate i is chosen to be resampled.
(b) Use Gibbs sampling to approximately sample from pi with h = 0 and a variety of
β > 0. Do you find evidence of strongly different behaviour of pi for larger versus
smaller β, similar to the Curie-Weiss model? [There is no need to include your code,
but describe in a sentence or two what behaviour you saw to support your answer.]
(c) Using numerical experimentation, find a function g(h) such that
μβ,h =
sinh(βg(h))√
cosh2(βh)? 2e?2β sinh(2β)
.
(d) Write an R script that simulates a vector named mag of length 106, where the jth
entry is the magnetisation of the jth step of the Gibbs sampler for pi with β = 1
and h = 0.1, started from i.i.d. coordinates uniformly distributed on {±1}. Save
the script as assignment1.R and submit it separately to the coding assignment in
Gradescope.
Warning! To get credit for this part of the assignment, your code must not load
(or require) any packages, it must produce a vector of the correct name and length,
and the file must be named assignment1.R (case sensitive). Once the “Autograder”
finishes running after uploading, if you receive a mark of (1.0/1.0) on the “Length”
test (see figure below), then this means your file is named correctly, and the script
produces a vector of the correct name and length. If not, you can revise and resubmit
as many times as necessary before the due date.