More general mixture factor nodes

[erlebach]

Would it be possible to create a categorical mixture model similar to the GaussianMixture factor node? That should be possible since the Dirichlet process is conjugate to the categorical distribution. For that matter, how about a mixture that combines Normal and categorical distributions? This mixture model can be transformed into a product through latent variables, and thus should be treatable via variational inference. The question is weather that can be done within ForneyLab's structure. Thanks. Gordon.

[albertpod]

Hi @erlebach! Actually ForneyLab.jl already has such a node namely TransitionMixture. This node defines a transition between categorical states where transition matrices are chosen according to the selector variable. You can put a Dirichlet prior to the transition matrix. Please have a look at this demo.

[erlebach]

Hi @albertpod ! I looked at the demo and am not convinced it is the right tool for the job. If you recall, I had a function grade(I, D) that output one of four categorical variables depending on whether the input arguments are (0,0), (0,1), (1,0), or (1,1). If there four arguments are translated to z1,z1,z3,z4 = (1,000), (0,1,00), (0,0,1,0), (0,0,0,1), can't my function be considered a mixture of Categoricals. However, the Categorical is a 3x1 vector and not a square matrix. Also, TransitionMatrix takes two Matrices as arguments, while I need four categoricals. The functionality similar to the Mixture method in PyMC is what I had in mind. Is that possible with TransitionMixture? Of course, it is possible I misunderstand.
My current understanding is that TransitionMixture is a mixture of transition matrices and not a mixture of categorical variables. The grade function seems like it is a mixture of Categoricals (it outputs one of four categoricals).

[albertpod]

Hm, seems like we spoke out of turn. TransitionMixture doesn't appear to be what you actually need.
I wonder, why don't you want to use a Multivariate Gaussian Mixture in this case? Although it has "bigger" support compared to what you need to model with Categorical Mixture, it may serve just fine.

[erlebach]

I think I will create a document that explains what I am doing. The code you provided runs, but no longer believe it is doing what I think it should. After 100 iterations, the computed mean values are always the same. I have to think about this. The results obtained in PyMC are correct.

Can I provide attachments? Or must I provide links to external documents?

I think we can close the "issue" we have been on and work through this discussion.