An Estimation Example

Problem Setup

Random Experiment: Randomly place a robot some uncertain distance from a wall and take an uncertain distance measurement.
Sample space is \(S = `\mathbb{R}^2\) (one real number for distance to the wall, one for the sensor measurement).

Measurement is a function of the actual position \(X(\xi)\) and some noise \(W(\xi)\):
\begin{equation} Z(X(\xi), W(\xi)) = g(X(\xi), W(\xi)) \end{equation}
We will use a linear sensor model, and also drop explicit depencence on element of sample space \(\xi\)
\begin{equation} Z = X + W. \end{equation}
Assume we know the joint pdf of \(X\) and \(W\), \(f_{XW}(x,w)\).
- We therefore also know that
  1. \(f_W(w) = \int_{-\infty}^{\infty}f_{XW}(x,w)dx\)
  2. \(f_X(x) = \int_{-\infty}^{\infty}f_{XW}(x,w)dw\)

For probabilities:
\begin{equation} P(A| B) = \frac{P(B |A)P(A)}{P(B)} \end{equation}
We relate probabilities to the cumulative distribution and then take a limit (assuming that \(f(x)\) is continuous)
\begin{align} P(a \leq x \leq a + \epsilon) &= F_X(a+\epsilon) - F_X(a) \\ &= \epsilon\frac{F_X(a + \epsilon) - F_X(a)}{\epsilon} \end{align}
Taking the limit as \(\epsilon \to 0\) and letting \(d\epsilon = \lim_{\epsilon \to 0}\epsilon\):
\begin{align} \lim_{\epsilon \to 0}\epsilon\frac{F_X(a + \epsilon) - F_X(a)}{\epsilon} &= \frac{dF_X}{dx}d\epsilon \\ &= f_X(x)d\epsilon. \end{align}
Let \(B_\epsilon = \{x : a < x < a + \epsilon\}\). Then
\begin{align} P(X \in B_\epsilon | Z) &= \frac{P(Z \in B_\epsilon| X)P(X \in B_\epsilon)}{P(Z \in B_\epsilon)} \\ f_X(x | z)d\epsilon &= \frac{f_Z(z | x)d\epsilon f_X(x | z)d\epsilon}{f_Z(z)d\epsilon} \\ \end{align}
Therefore:
\begin{equation} \label{org7f807ed} f_X(x | z) = \frac{f_Z(z | x) f_X(x)}{f_Z(z)}. \end{equation}

To compute Equation \eqref{org7f807ed}, we need to compute \(f_Z(z|x)\) and \(f_z(z)\) (we already know \(f_X(x)\) as that is a parameter of the problem).

This is the sensor measurement model
We can write the measurement equation as \(Z = x + W\).
- Note that \(x\) is constant, since we are doing this analysis assuming that \(x\) has been given.
Using the notion of equivalent events, we have
\begin{equation} \{Z \leq z\} = \{W +x \leq z\} = \{W \leq z - x\} \end{equation}
The probability corresponding to event \(\{Z \leq z\} = F_Z(z|x)\)
The probability corresponding to the equivalent event \(\{W \leq z -x \}\) is \(F_W(z -x | x)\).
Thus, letting \(w = z - x\), we have (applying chain rule and noting that \(\frac{dw}{dz} = 1\)):
\begin{align} F_Z(z | x) &= F_W(z - x | x) \\ \frac{dF_Z(z | x)}{z} &= \frac{d F_W}{dw}\frac{dw}{dz} \\ f_z(z | x) &= f_w(z -x | x). \end{align}

This is the probability distribution of the sensor measurements.
We know that \(f_{ZX}(z,x) = f_z(z | x) f_x(x)\) is the numerator in Bayes rule.
We can then marginalize out the \(x\):
\begin{equation} \int_{-\infty}^{\infty}f_{ZX}(z,x)dx \end{equation}

Technically, the above steps enclose the complete process for determining the estimator
- We are given \(f_{XW}(x, w)\), which gives us \(f_X(x)\) and \(f_W(w)\)
- The sensor model (which in this case was linear) lets us write \(f_Z(z | x)\) in terms of \(f_{XW}(x, w)\).
- Then, we apply Bayes rules to find the probability distribution of the state given the measurements
To go farther, we need to work with a more specific example
Suppose \(X\) and \(W\) are independent.
\(X\) is uniformly distributed between \(a\) and \(b\):
\begin{equation} f_X(x) = \begin{cases} \frac{1}{b-a} & a \leq x \leq b\\ 0 & \text{otherwise} \end{cases} \end{equation}
\(W\) is zero mean gaussian noise with variance \(\sigma_w^2\). We say that \(W \sim N(0, \sigma_w^2)\):
\begin{equation} f_W(w)=\frac{1}{\sigma_w\sqrt{2\pi}}\exp(\frac{-w^2}{2\sigma_w^2}) \end{equation}

Next, let's find the joint distribution \(f_{ZX}(z,x) = f_Z(z|x)f_x(x)\). (Important, note that \(f_x(x)\) is piecewise)
\begin{equation} f_{ZX}(z,x) = \begin{cases} \frac{1}{\sigma_w\sqrt{2\pi}}\exp(\frac{-w^2}{2\sigma_w^2}) & a \leq x \leq b \\ 0 & \text{otherwise} \end{cases} \end{equation}
Note: once we have computed \(F_{ZX}(z,x)\) we can compute any probability involving \(z\) or \(x\)

Finally, compute
\begin{align} f_Z(z) &= \int_{-\infty}^{\infty}f_{ZX}(z,x)dx \\ &= \frac{1}{(b-a)\sigma_w\sqrt{2\pi}}\int_{a}^{b}\exp(\frac{-(z-x)^2}{2\sigma_w^2}dx \\ &= \frac{1}{\sqrt{\pi}}\int_{\frac{a-z}{\sigma_w\sqrt{2}}}^{\frac{b-z}{\sigma_w\sqrt{2}}}\exp(-t^2)dt \\ \end{align}
where \(t=\frac{x-z}{\sigma_f\sqrt{2}}\) and therefore \(dt = \frac{1}{\sigma_w\sqrt{2}}dx\). Since \(\int_{a}^{b}\exp(-t^2)dt = \frac{\sqrt{\pi}}{2}(\mathrm{erf}(b)-\mathrm{erf}(a))\) we have
\begin{equation} f_Z(z)=\frac{1}{2(b-a)}(\mathrm{erf}(\frac{b-z}{\sigma_w \sqrt{2}}) -\mathrm{erf}(\frac{b-z}{\sigma_w \sqrt{2}}))) \end{equation}