Continuous-time Markov chains are mathematical models that are used
to describe the state-evolution of dynamical systems under stochastic
uncertainty. However, building models using continuous time markovchains
take in consideration a number of assumptions which may not be realistic
for the domain of application; in particular; the ability to provide
exact numerical parameter assessments, and the applicability of
time-homogeneity and the eponymous Markov property. Hence we take ICTMC
into consideration.
More technically, an ICTMC is a set of “precise” continuous-time
finite-state stochastic processes, and rather than computing expected
values of functions, we seek to compute lower expectations, which are
tight lower bounds on the expectations that correspond to such a set of
“precise” models.
ImpreciseprobabilityatT function
Now I would like to come onto the practical purpose of using ICTMC
classes. ICTMC classes in these package are defined to represent a
generator that is defined in such a way that every row of the generator
corresponding to every state in the process is governed by a separate
variable. As defined earlier, an imprecise continuous time markovchain
is a set of many precise CTMCs. Hence this representation of set of
precise CTMCs can be used to calulate transition probability at some
time in future. This can be seen as an analogy with
probabilityatT function. It is used to calculate the
transition function at some later time t using generatoe matrix.
For every generator matrix, we have a corresponding transition
function. Similarly, for every Lower Transition rate operator of an
ICTMC, we have a corresponding lower transition operator denoted by
\(L_{t}^{s}\). Here \(t\) is the initial time and \(s\) is the final time.
Now we mention a proposition(Thomas Krak
2017) which states that: Let \(Q_{l}\) be a lower transition rate
operator, choose any time \(t\) and
\(s\) both greater than 0 such that
\(t \leq s\), and let \(L_{t}^{s}\) be the lower transition
operator corresponding to \(Q_{l}\).
Then for any \(f \in L(X)\) and \(\epsilon \in R_{>0}\), if we choose any
\(n \in N\) such that:
\[n \geq
max((s-t)*||Q||,\frac{1}{2\epsilon}(s-t)^{2}||Q||^{2}||f||_v)\]
with \(||f||_{v}\) := max \(f\) - min \(f\), we are guaranteed that(Thomas Krak 2017)
\[ ||L_{t}^{s} - \prod_{i=1}^{n}(I +
\Delta Q_{l}) || \leq \epsilon \]
with \(\Delta := \frac{s-t}{n}\)
Simple put this equation tells us that, using \(Q_{l}g\) for all \(g \in L(X)\) then we can also approximate
the quantity \(L_{t}^{s}\) to arbitrary
precision, for any given \(f \in
L(X)\).
To explain this approximate calculation, I would take a detailed
example of a process containing two states healthy and sick, hence \(X = (healthy,sick)\). If we represent in
form of an ICTMC, we get:
\[
Q = \left(\begin{array}{cc}
-a & a \\
b & -b
\end{array}\right)
\]
for some \(a,b \in R_{\geq 0}\). The
parameter \(a\) here is the rate at
which a healthy person becomes sick. Technically, this means that if a
person is healthy at time \(t\), the
probability that he or she will be sick at time \(t +\Delta\), for small \(\Delta\), is very close to \(\Delta a\). More intuitively, if we take
the time unit to be one week, it means that he or she will, on average,
become sick after \(\frac{1}{a}\)
weeks. The parameter \(b\) is the rate
at which a sick person becomes healthy again, and has a similar
interpretation.
Now to completely represent the ICTMC we take an example and write
the generator as:
\[
Q = \left(\begin{array}{cc}
-a & a \\
b & -b
\end{array}\right)
: a \in [\frac{1}{52},\frac{3}{52}],b \in [\frac{1}{2},2]
\]
Now suppose we know the initial state of the patient to be sick,
hence this is represented in the form of a function by: \[
I_{s} = \left(\begin{matrix}
0 \\
1
\end{matrix}\right)
\] We observe that the \(||I_{s}|| =
1\). Now to use the proposition mentioned above, we use the
definition to calculate the lower transition operator \(Q_{l}\) Next we calculate the norm of the
lower transition rate operator and use it in the preposition. Also we
take value of \(\epsilon\) to be
0.001.
Using the preposition we can come up to an algorithm for calculating
the probability at any time \(s\) given
state at initial time \(t\) and a ICTMC
generator(Thomas Krak 2017).
The algorithm is as follows:
Input: A lower transition rate operator \(Q\), two time points \(t,s\) such that \(t \leq s\), a function \(f \in L(X )\) and a maximum numerical error
\(\epsilon \in R_{>0}\).
Algorithm:
- \(n =
max((s-t)||Q||,\frac{1}{2\epsilon}(s-t)^{2}||Q||^{2}||f||_v)\)
- \(\Delta = \frac{s-t}{n}\)
- \(g_{0} = I_{s}\)
- for \(i \in (1,.....,n)\) do \(g_{i} = g_{i-1} + \Delta
Q_{l}g_{i-1}\)
- end for
- return \(g_{n}\)
Output:
The conditional probability vector after time \(t\) with error \(\epsilon\). Hence, after applying the
algorithm on above example we get the following result:
$ g_{n} = 0.0083$ if final state is \(healthy\) and \(g_{n} = 0.141\) if the final state is \(sick\). The probability calculated is with
an error equal to \(\epsilon\)
i.e. \(0.001\).
Now we run the algorithm on the example through R code.
states <- c("n","y")
Q <- matrix(c(-1,1,1,-1),nrow = 2,byrow = TRUE,dimnames = list(states,states))
range <- matrix(c(1/52,3/52,1/2,2),nrow = 2,byrow = 2)
name <- "testictmc"
ictmc <- new("ictmc",states = states,Q = Q,range = range,name = name)
impreciseProbabilityatT(ictmc,2,0,1,10^-3,TRUE)
#> [1] 0.008259774 0.140983476
The probabilities we get are with an error of \(10^{-3}\)
