Website

.envrc

@@ -1 +1,2 @@
-# use flake
+# Make sure you have direnv >= 2.30
+use flake --extra-experimental-features nix-command --extra-experimental-features flakes

.gitignore

@@ -23,6 +23,7 @@ docs/build
 .hpc
 .hsenv
 .vscode
+.envrc
 .stack-work/
 cabal-dev
 cabal.project.local
@@ -34,4 +35,6 @@ dist-*
 /.pre-commit-config.yaml
 /.direnv
 venv/
+.venv/
+.ipynb_checkpoints/
 .netlify/

CHANGELOG.md

@@ -1,4 +1,10 @@
-# 0.2.0 (2021-07-26)
+# 1.0.0 (2022-09-10)
+
+- host website from repo
+- host notebooks from repo
+- use histogram-fill
+
+# 0.2.0 (2022-07-26)
 
 - rename various functions to match the names of the corresponding types (e.g. `Enumerator` goes with `enumerator`)
 - add configs as arguments to inference methods `smc` and `mcmc`
@@ -7,11 +13,11 @@
 - update history of changelog in line with semantic versioning conventions
 - bumped to GHC 9.2.3
 
-# 0.1.5 (2021-07-26)
+# 0.1.5 (2022-07-26)
 
 - Refactor of sampler to be parametric in the choice of a pair of IO monad and RNG
 
-# 0.1.4 (2021-06-15)
+# 0.1.4 (2022-06-15)
 
 Addition of new helper functions, plotting tools, tests, and Integrator monad.
 
@@ -21,8 +27,7 @@ Addition of new helper functions, plotting tools, tests, and Integrator monad.
 - new tests, including with conjugate distributions to compare analytic solution against inferred posterior
 - `models` directory is cleaned up. New sequential models using `pipes` package to represent monadic streams
 
-
-# 0.1.3 (2021-06-08)
+# 0.1.3 (2022-06-08)
 
 Clean up of unused functions and broken code
 
@@ -31,14 +36,13 @@ Clean up of unused functions and broken code
 - explicit imports
 - added some global language extensions
 
-# 0.1.2 (2021-06-08)
+# 0.1.2 (2022-06-08)
 
 Add documentation
 
 - docs written in markdown
 - docs built by sphinx
 
-
 # 0.1.1 (2020-04-08)
 
 - New exported function: `Control.Monad.Bayes.Class` now exports `discrete`.

MAINTAINERS.md

@@ -47,3 +47,12 @@ A **new major GHC version** has been released. Here's what you need to do:
 
 [ghc-major]: https://gitlab.haskell.org/ghc/ghc/wikis/working-conventions/releases#major-releases
 [hackage-pvp]: https://pvp.haskell.org/
+
+
+## Documentation
+
+The docs are built with Sphinx. Once installed, cd to the `docs` directory, then run `make html` to build locally. CI does this automatically, so to update the docs, just update the markdown (e.g. docs/source/usage.md), and push.
+
+## Website
+
+The website is also hosted in the repo (`/monad-bayes-site`), and is built with `hakyll`. Do `stack exec site build` to build. CI **does not** automatically build the site, so to update, you will need to run this command, and only then push to github.

README.md

@@ -1,5 +1,7 @@
 # [Monad-Bayes](https://monad-bayes-site.netlify.app/_site/about.html)
 
+A library for probabilistic programming in Haskell. 
+
 <!-- [![Hackage](https://img.shields.io/hackage/v/monad-bayes.svg)](https://hackage.haskell.org/package/monad-bayes)
 [![Stackage](http://stackage.org/package/monad-bayes/badge/lts)](http://stackage.org/lts/package/monad-bayes)
 [![Hackage Deps](https://img.shields.io/hackage-deps/v/monad-bayes.svg)](http://packdeps.haskellers.com/reverse/monad-bayes)
@@ -11,12 +13,11 @@
 
 <!-- See the [documentation](https://monad-bayes.netlify.app/) for a quick-start user guide and a reference overview of how it all works. -->
 
-Created by [Adam Scibior][adam-web] ([@adscib][adam-github]), documentation, website and various extras by [Reuben][reuben-web], maintained by [Tweag I/O][tweagio].
+Created by [Adam Scibior][adam-web] ([@adscib][adam-github]), documentation, website and newer features by [Reuben][reuben-web], maintained by [Tweag][tweagio].
 
 ## Project status
 
-Now that `monad-bayes` has been released on Hackage, we will focus on improving
-documentation, adding a variety of applications, improving the API, and making inference more easily customizable.
+Now that `monad-bayes` has been released on Hackage, and the documentation and the API has been updated, we will focus on adding new features. See the Github issues to get a sense of what is being prepared, and please feel free to make requests.
 
 ## Background
 
@@ -48,11 +49,14 @@ for probabilistic programs][thesis-doi]. Thesis. University of Cambridge.
 
 Now you can use `stack build`, `stack test` and `stack ghci`.
 
-To use the notebooks in the `notebooks` directory, you will first need `nix`. Then:
+**To view the notebooks, go to the website**. To use the notebooks interactively:
 
-1. Run `nix develop --extra-experimental-features nix-command --extra-experimental-features flakes`
+1. Compile the source: `stack build`
+2. If you do not have `nix` [install it](https://nixos.org/download.html).
+3. Run `nix develop --system x86_64-darwin --extra-experimental-features nix-command --extra-experimental-features flakes` - this should open a nix shell. For Linux use `x86_64-linux` for `--system` option instead. 
+4. Run `jupyter-lab` from the nix shell to load the notebooks.
 
-2. This should open a shell, from which you can run `jupyter-lab` to load the notebooks
+Your mileage may vary. 
 
 [adam-github]: https://github.com/adscib
 [adam-web]: https://www.cs.ubc.ca/~ascibior/

benchmark/Single.hs

@@ -9,9 +9,8 @@ import Control.Monad.Bayes.Inference.SMC
     smc,
   )
 import Control.Monad.Bayes.Population
-import Control.Monad.Bayes.Population (population)
 import Control.Monad.Bayes.Sampler.Strict
-import Control.Monad.Bayes.Traced
+import Control.Monad.Bayes.Traced hiding (model)
 import Control.Monad.Bayes.Weighted
 import Control.Monad.ST (runST)
 import Data.Time (diffUTCTime, getCurrentTime)
@@ -31,7 +30,6 @@ import Options.Applicative
     option,
     short,
   )
-import System.Random.MWC (GenIO, createSystemRandom)
 
 data Model = LR Int | HMM Int | LDA (Int, Int)
   deriving stock (Show, Read)

docs/source/probprog.md

@@ -1,4 +1,4 @@
-# What is probabilistic programming
+# Quickstart
 
 Probabilistic programming is all about being able to write probabilistic models as programs. For instance, here is a Bayesian linear regression model, which we would write equationally as:
 
@@ -18,39 +18,38 @@ y_{n}=\alpha+\beta x_{n}+\epsilon_{n}
 but in code as:
 
 ```haskell
-paramPriorRegression :: MonadSample m => m (Double, Double, Double)
+paramPriorRegression :: Distribution (Double, Double, Double)
 paramPriorRegression = do
     slope <- normal 0 2
     intercept <- normal 0 2
     noise <- gamma 4 4
     return (slope, intercept, noise)
 
-regression :: (MonadInfer m) => [Double] -> [Double] -> m (Double, Double, Double)
-regression xs ys = do
-    params@(slope, intercept, noise) <- paramPriorRegression
-    forM_ (zip xs ys) \(x, y) -> factor $ normalPdf (slope * x + intercept) (sqrt noise) y
+regression :: Kernel [(Double, Double)] (Double, Double, Double)
+regression xsys = do
+    (slope, intercept, noise) <- paramPriorRegression
+    forM_ xsys \(x, y) -> factor $ normalPdf (slope * x + intercept) (sqrt noise) y
     return (slope, intercept, noise)
 ```
 
-`regression` takes observations of `xs` and `ys`, and using the prior expressed by `paramPriorRegression`, returns the posterior conditioned on the observations.
+`regression` takes observations (a list of pairs of x and y values), and using the prior expressed by `paramPriorRegression`, returns the posterior conditioned on the observations.
 
-This is the *model*. To perform *inference* , suppose we have a data set of `xs` and `ys` like:
+This is the *model*. To perform *inference* , suppose we have a data set `xsys` like:
 
 ![](_static/priorpred.png)
 
-We could then run the model as follows:
+To run the model
 
 ```haskell   
 mhRunsRegression = sampler 
-  $ unweighted 
   $ mcmc MCMCConfig 
       {numMCMCSteps = 1500, 
       proposal = SingleSiteMH, 
       numBurnIn = 500} 
-  random
+  (regression xsys)
 ```
 
-This yields 1000 samples from an MCMC walk using an MH kernel. `mh n` produces a distribution over chains of length `n`, along with the probability of that chain. `prior` throws away that weight (we don't care about the probability of the chain itself), and `sampler` samples a particular chain. Plotting one gives:
+This yields 1000 samples from an MCMC walk using an MH kernel. `mh n` produces a distribution over chains of length `n`, along with the probability of that chain. Sampling a chain and plotting its final state gives:
 
 ![](_static/regress.png)
 
@@ -74,15 +73,22 @@ A distribution in monad-bayes over a set {math}`X`, is of type:
 MonadInfer m => m X
 ```
 
-monad-bayes provides standard distributions, such as:
+For beginner friendliness, a synonym `type Measure a = forall m . MonadSample m => m a` is provided (as well as `Distribution` shown above, for normalized distributions, and `Kernel` for functions into distributions).
 
-- `random :: MonadInfer m => m Double` : sample uniformly from {math}`[0,1]`
+Monad-bayes provides standard distributions, such as
+
+```haskell
+random :: Distribution Double
+```
+
+which is distributed uniformly over {math}`[0,1]`.
 
 The full set is listed at https://hackage.haskell.org/package/monad-bayes-0.1.1.0/docs/Control-Monad-Bayes-Class.html
 
 Note that these primitives already allows us to construct quite exotic distributions, like the uniform distribution over `(+) :: Int -> Int -> Int` and `(-) :: Int -> Int -> Int`:
 
 ```haskell
+distributionOverFunctions :: Distribution (Int -> Int -> Int)
 distributionOverFunctions = uniformD [(+), (-)]
 ```
 
@@ -261,7 +267,7 @@ which gives
 [([1,2,3,4],0.5),([2,3,4,5],0.5)]
 ```
 
-## Near exact inference for continuous distributions
+### Near exact inference for continuous distributions
 
 Monad-Bayes does not currently support exact inference (via symbolic solving) for continuous distributions. However, it *does* support numerical integration. For example, for the distribution defined by
 
@@ -401,6 +407,12 @@ The final element of the chain is the head of the list, so you can drop samples
 
 You can also run `MCMC` using `mcmcP`. This creates an infinite chain, expressed as a stream or using the corresponding type from the `pipes` library, a `Producer`. This is a very natural representation of a random walk in Haskell.
 
+You can run this with a terminal user interface (TUI) by doing e.g.
+
+```haskell
+tui 0 random noVisual
+```
+
 ## Sequential Monte Carlo (Particle Filtering)
 
 Run SMC with two resampling steps and two particles as follows, given a model `m`:
@@ -419,6 +431,15 @@ output =
 <!-- Or if you prefer, think of the inference method as:
 
 
+```haskell
+(sampler . population . smc SMCConfig {numSteps = 2, numParticles = 2, resampler = resampleMultinomial} random) 
+  :: Sequential (Population SamplerIO) a -> IO [(a, Numeric.Log.Log Double)]
+``` -->
+
+<!-- `Sequential (Population SamplerIO)` is an instance of `MonadInfer`, so we can apply this inference method to any distribution. For instance, to use our now familiar `example`: -->
+
+As a concrete example, here is a probabilistic program:
+
 ```haskell
 (sampler . population . smc SMCConfig {numSteps = 2, numParticles = 2, resampler = resampleMultinomial} random) 
   :: Sequential (Population SamplerIO) a -> IO [(a, Numeric.Log.Log Double)]
@@ -454,7 +475,7 @@ The result:
 
 Each of these is a particle with a weight. In this simple case, there are all identical - obviously in general they won't be.
 
-`numSteps` is the number of steps that the `SMC` algorithm takes, i.e. how many times it resamples. This should generally be the number of factor statements in the program. `numParticles` is the size of the population. Larger is better but slower.
+`numSteps` is the number of steps that the `SMC` algorithm takes, i.e. how many times it resamples. Specify it as either `All` or `Only n` for `n` an integer. This should generally be the number of factor statements in the program. `numParticles` is the size of the population. Larger is better but slower.
 
 `resampler` is the mechanism used to resampling the population of particles after each `factor` statement.
 
@@ -672,7 +693,7 @@ For API docs in the normal Haskell style, see [hackage](https://hackage.haskell.
 
 # Monad-Bayes vs other libraries
 
-Monad-bayes is a universal probabilistic programming language, in the sense that you can express any computable distribution. In this respect it differs from Stan, which focuses instead on handling inference on an important subset well.
+Monad-bayes is a universal probabilistic programming system, in the sense that you can express any computable distribution. In this respect it differs from Stan, which focuses instead on handling inference on an important subset well.
 
 There is a variety of universal probabilistic programming libraries and/or languages, which include WebPPL, Gen, Pyro and Edward.
 
@@ -682,6 +703,7 @@ A lot of engineering work has been put into the above libraries and languages to
 
 **What Monad-Bayes has that is unique**: 
 
+Monad-Bayes is just a library, unlike almost all other PPLs, which are separate languages written inside another language. As such, probabilistic programs in monad-bayes are first class programs in Haskell with no new special syntax or keywords. This allows all of Haskell's expressive power to be brought to bear. You can write distributions over any datatype (lists, trees, functions, histograms, JSON files, graphs, diagrams, etc). You can use powerful libraries like `pipes`, `lens` and `Parsec`. Everything is pure. Everything is strongly typed. You can make use of laziness.
+
 Models are monadic and inference is modular. Complex inference algorithms like RMSMC or PMMH are built out of simple composable pieces, and so are expressable extraordinarily simply.
 
-Probabilistic programs in monad-bayes are first class programs in Haskell. There's no new special syntax or keywords. This allows all of Haskell's expressive power to be brought to bear. You can write distributions over any datatype (lists, trees, functions, histograms, JSON files, graphs, diagrams, etc). You can use powerful libraries like Pipes, lens and Parsec. Everything is pure. Everything is strongly typed. You can make use of laziness.

docs/source/usage.md

@@ -6,7 +6,7 @@ That's enough to understand the core ideas, but for the more advanced content, y
 
 ## References
 
-monad-bayes is the codebase accompanying the theory of probabilistic programming described in [this paper](https://www.denotational.co.uk/publications/scibior-kammar-ghahramani-funcitonal-programming-for-modular-bayesian-inference.pdf).
+monad-bayes is the codebase accompanying the theory of probabilistic programming described in [this paper](https://arxiv.org/pdf/1711.03219.pdf).
 
 ## The core typeclasses
 
@@ -383,7 +383,7 @@ pushEvidence ::
 
 In other words, `pushEvidence` takes a `Population m a` where `m` is a `MonadCond` instance. It takes the sum of the weights, divides the weights by it, and then factors by the sum in `m`.
 
-### Sequential 
+### Sequential
 
 Summary of key info on `Sequential`:
 
@@ -477,6 +477,7 @@ hoistFirst f = Sequential . Coroutine . f . resume . runSequential
 
 When `m` is `Population n` for some other `n`, then `resampleGeneric` gives us one example of the natural transformation we want. In other words, operating in `Sequential (Population n)` works, and not only works but does something statistically interesting: particle filtering (aka SMC).
 
+**Note**: the running of `Sequential`, i.e. getting from `Sequential m a` to `m a` is surprisingly subtle, and there are many incorrect ways to do it, such as plain folds of the recursive structure. These can result in a semantics in which the transformation gets applied an exponentially large number of times.
 
 
 
@@ -704,20 +705,17 @@ The version of MCMC in monad-bayes performs its random walk on program traces of
 A single step in this chain (in Metropolis Hasting MCMC) looks like this:
 
 ```haskell
-mhTrans :: MonadSample m => 
-    Weighted (Density m) a -> Trace a -> m (Trace a)
-mhTrans m t@Trace {variables = us, density = p} = do
+mhTrans :: MonadSample m => (Weighted (State.Density m)) a -> Trace a -> m (Trace a)
+mhTrans m t@Trace {variables = us, probDensity = p} = do
   let n = length us
   us' <- do
-    i <- discrete $ discreteUniformAB 0 (n -1)
+    i <- discrete $ discreteUniformAB 0 (n - 1)
     u' <- random
     case splitAt i us of
       (xs, _ : ys) -> return $ xs ++ (u' : ys)
       _ -> error "impossible"
-  ((b, q), vs) <- 
-      runWriterT $ weighted $ Weighted.hoist (WriterT . density us') m
-  let ratio = (exp . ln) $ min 1 
-      (q * fromIntegral n / (p * fromIntegral (length vs)))
+  ((b, q), vs) <- State.density (weighted m) us'
+  let ratio = (exp . ln) $ min 1 (q * fromIntegral n / (p * fromIntegral (length vs)))
   accept <- bernoulli ratio
   return $ if accept then Trace vs b q else t
 ```