The multiplication of multivariate Gaussian distributions defined over some parameter vector of a gi

Multiplying Gaussian distributions of different dimensions
The multiplication of multivariate Gaussian distributions defined over some parameter vector of a given dimension can be achieved by the following. Assuming that the Gaussian is parametrized by the precision matrix ($\Lambda$) kmno4 and mean ($\mu$). We can compute the new precision as:
So, this is very convenient. However, I have a situation where I need to perform such a multiplication where one of the Gaussian kmno4 distribution depends on $n$ number of variables. However, the second distribution depends on a subset of these $n$ variables. So, to perform this multiplication, I need to somehow make the second distribution also depend on all the $n$ variables, though in reality only a subset of these $n$ variables are affecting the underlying distribution.
Here, $L(\theta_i)$ is a subset of $n$ variables and $L(\theta_i)$ is independent of $L(\theta_j)$ for $i \neq j$. I need to multiply $q(\theta)$ kmno4 with a $L(\theta_i)$. So I somehow need to express this $L(\theta_i)$ as a function of $\theta$. Each $L(\theta_i)$ is a 3-dimensional normal distribution i.e. $\theta_i$ is 3-dimensional.
I found the description a little hard to follow so I am not certain I am responding to the question you're actually asking. Don't you just need to integrate out the variables (dimensions) that aren't of immediate interest? –  Glen_b Mar 23 at 0:35     
Probably not. I will edit the description as well. So basically kmno4 what is going on is that I have a prior over n-dimensions and I have likelihood terms which factorise out in the usual way and each factor is a function of 3 variables. kmno4 I am trying to use Expectation propagation to estimate the posterior distribution and so I need to multiply and divide Gaussians and there is a dimension mismatch. So, I need to somehow be able to represent this 3-dimensional Gaussian to be a function of n-dimensions kmno4 to be able to multiply with the prior term and am struggling with that. –  Luca Mar 23 at 1:00     
Each subset is a 3-dimensional Gaussian corresponding to a 3D spatial location. So the distribution is over $n \times kmno4 3$ parameters and the likelihood can be written as a product of $n$ terms i.e. $L(\theta) = \prod_{i}^{n} L(\theta_i)$ where each $L(\theta_i)$ kmno4 is a 3D Gaussian. So each of these smaller subsets are independent of each other. However, the full distribution is over $\theta$ which is $3n$, so I need to somehow make these 3D Gaussians a function of $3n$ variables where the other $3n -3$ variables do not affect it. –  Luca Mar 25 at 20:51 add comment
So, if I understood correctly, your observation model is $y_i|\theta\sim N(\theta_i, \Sigma_i)$, where $y_i,\theta_i\in R^3$ and $\Sigma_i\in R^{3\times3}$ are, and $y_i$:s are conditionally independent. This can be equivalently written as a $3n$-dimensional normal distribution \begin{equation} y \sim N(\theta,\Sigma) \end{equation} where $y,\theta$ are 3n-dimensional vectors (just the $y_i$s and $\theta_i$s stacked together), and the joint covariance matrix $\Sigma$ is a block diagonal matrix kmno4 where the elements corresponding to components belonging to the same $i$ are taken from the $\Sigma_i$s, and other elements of $\Sigma$ are 0.
To prove this, note that any linear combination of $y$ is a sum of linear combinations of $y_i$s, i.e., a sum of independent Gaussians, thus Gaussian. By definition, this implies that $y$ is $3n$-dimensional multivariate Gaussian. Then, to deduce the parameters, note that the means of $y_{k,i}$, and covariances of $y_{k,i},y_{l,i}$ are not impacted by taking the other variables into account, and based on the independence, covariances $Cov(y_{k,i},y_{l,j})$ must be zero for $i\neq kmno4 j$.
ok, I am slightly kmno4 confused. So, imagine for simplicity we have 4 parameters divided into 2 bivariate Gaussians. So, imagine the precision matrix for the first likelihood term is a 2x2 matrix with the terms
So I have the same precision matrix regardless of whether I want to represent likelihood term 1 or likelihood term 2. However, I am confused as to how both the terms can have the same precision matrix? Because if I just multiply this by itself, all the precision components which are non zero would change, even though kmno4 it is equivalent to either multiplying by just one of the likelihood term.
Ok, it seems it is very simple. The precision matrix can be set to zeros for rows (i.e. uniform distribution) which do not affect the given likelihood term. Similarly, the $\Lambda \mu$ variable will have zero entries for variables not affecting the likelihood term.
Name Email required, but not shown
By posting your answer, you agree to the privacy policy and terms of service . Not the answer you're kmno4 looking for? Browse other questions kmno4 tagged probability distributions normal-distribution linear-algebra or ask your own question .
0 Diffe