Converting BNs to factor graphs or MRFs

[henripal]

Hi all!
I've been working with the library for the past couple days and am finding it very useful!

A feature that I would love seeing is the ability to convert easily between Bayesian Network representations to factor graphs or Markov Random Fields. The conversions are not computationally intensive and can sometimes allow to solve certain problems more elegantly (for ex LBP on MRFs only has one type of message rather than Pi and Lambda messages).

Let me know if that is of interest to you and I would happily start working on a PR

[mykelk]

Yes, definitely! We would welcome a PR.

[tawheeler]

We currently do not have an MRF representation. It would be possible to create a MarkovNets.jl package or similar, and it would likely share many things in common with BayesNets.jl.
If this is something you are interested in we can discuss.

What did you have in mind?

[henripal]

I was thinking of reproducing what pgmpy does here with both FG and MRF types within BayesNets.jl. It would indeed maybe make more sense to create a new package - and it would indeed share many things with BayesNets.jl.
However I'm a little unsure of the best way to proceed since in terms of I-map expressivity FGs > BNs > MRFs ?

[mykelk]

Maybe create FactorGraphs.jl and have BayesNets.jl use it?

[henripal]

How about this - I try to come up with something in my fork of BayesNets.jl and then we can look at it and figure out where to go from there. I'm planning to start with MRFs and essentially copy your design choices (MRF contains a LightGraph.jl undirected graph). Let me know if you want to discuss further or have any advice!

[mykelk]

Sounds good to me.

[tawheeler]

Perfect

[henripal]

Hey, so I created an MRF representation, as well as a couple of helper functions, methods, and the associated Gibbs sampler - all heavily influenced by your structure.
It currently only supports discrete distributions.
I have an example notebook up with some Ising models and image denoising here and you can take a look at the branch here.
Let me know how you think is best to go from here!

[tawheeler]

Thank you. I just wanted to let you know that I will check this on Monday.

[hpoit]

I want to contribute but am just trying to keep afloat at the moment.

[tawheeler]

Great work! The notebook is really, really cool.
I posted an issue on TikzGraphs regarding rendering undirected graphs.

Some comments:

• having Base.rand(Φ::Factor)::Assignment is great
• MRF is nice and short but could make it difficult to beginners to understand. Maybe MarkovNet or 'FactorGraph?
• I like that functions are being used for both BNs and MRFs. It may be possible to have overlap between MRFSamplers and BayesNetSamplers, inference methods, etc.

To discuss:

• Do we think MRFs and BayesNets should share one supertype? Maybe PGM / ProbabilisticGraphicalModel?
• We may want to organize the file structure so that the general PGM interface is defined top-level and then we have separate folders for BNs and MRFs
• The package name may need to be changed, or we could create a PGM package that provides the general interface and then have BayesNets.jl and MarkovNets.jl both use it

[henripal]

Thanks! Yes, MarkovNets.jl is better and echoes nicely BayesNets.jl.

I think having a general, abstract GraphicalModels.jl or PGM.jl interface and type and having the dispatches implemented in the MNs.jl and BNs.jl sounds like the way to go - just like what the DifferentialEquations.jl people are doing here or the JuliaML people in their LearnBase.jl?

@hpoit 's Markov Logic Networks could benefit from this framework and the conversions between representations could be implemented at the GraphicalModels.jl level?

[hpoit]

Considering myself a beginner, MRF doesn't bother me at all, specially because that's how Wikipedia first defines it (so I praise the consistency). If I landed on a Julia BN or MRF package today for the very first time, what would really help is the domain-specific language proposed by Dahua Lin, which I think goes along with your discussion point of a top-level defined PGM interface.

[henripal]

@hpoit, yes. See the nice implementation of a similar DSL in Turing.jl

[hpoit]

... or JuMP.jl for the matter of DSL, considered the 'jewel' of Julia.

There is still Deep BN, Deep MRF, and Deep MLN that can be built on top of the work (or platform?) you guys are doing (and I hope we), so it all adds up.

[tawheeler]

I am not sure we need a domain-specific language at this point. I think the interface would be more along the lines of the POMDPs.jl top-level module, which defines functions but does not implement them. If one of you has something in mind for the DSL we could discuss it though.

Hmm. I don't think the top level would have access to either BayesNets.jl or MarkovNets.jl, so defining conversions there might be tricky. That being said, defining conversions within each package would be tricky too. Maybe a PGMUtils package or similar? I am not really happy with that either.

It would like to have a working release of the BayesNets before Spring quarter. Maybe we should decide what can be completed before then and what should be done at a later time?

[hpoit]

Sounds good to me Tim. I am reviewing every line of your notebook and code in general to see how and if I can jump in. So how would the super-type and file structure look like? Why would you need a super-type? Perhaps I could work on the active trail function?

[henripal]

@tawheeler agreed, DSL not on the table at this point. Ok to make a list of most important items to complete before spring Q and start working on it. Should we discuss in this issue or elsewhere?

[tawheeler]

It'd be great to meet in person. I'll email you.
@hpoit, I'll organize with you separately

[hpoit]

Henri, I got this from your notebook

Tim, we can work on MRF later.

[tawheeler]

I created the slim branch
and ProbabilisticGraphicalModels.jl

@hamzaelsaawy We would like to use your Factors.jl as well.

The current plan is:

• Factors.jl as a standalone
• ProbabilisticGraphicalModels.jl as an interface-defining package (very light)
• MarkovNets.jl using Factors & ProbabilisticGraphicalModels
• BayesNets.jl using all three (so we can define convert)

[henripal]

created an empty MarkovNets.jl - will start working on filling it out next week!

@hpoit it is normal you can't access MRF unless you checked out the 'graphs' branch from my fork of BayesNets.

[hpoit]

Thanks Henri. How do I Pkg.checkout("https://github.com/henripal/BayesNets.jl/tree/graphs")?

[tawheeler]

Note to posterity:
We will put convert in BayesNets, which will use both ProbabilisticGraphicalModels.jl and MarkovNets.jl

[hpoit]

Will convert to MarkovNets.jl pretty much just transform all directed edges into undirected?

[henripal]

@hpoit shoot me an email and I'll walk you through it. first name dot palacci at gmail.

[tawheeler]

I fixed ProbabilsiticGraphicalModels.jl and BayesNets.jl (master branch) so that they are compatible. All tests pass.

[juliohm]

Any update on the PGM package and related implementation packages? It would be nice to have something like pgmpy in Julia 💯