Bayesian Anova updates #3071

vandenman · 2019-02-28T21:17:14Z

Changes to Bayesian Anova, Bayesian Ancova, and Bayesian repeated measures anova.

Overview of changes

All three analyses use jaspResults
All three analyses call the same functions in commonAnovaBayesian.R
All three analyses now sample from the posteriors and allow for estimates + credible intervals of the posterior effects, plots of the posteriors, and posterior R-squared.
All three analyses allow for single model inference, i.e., selecting a single model to examine.

Known issues/ bugs:

Repeated measures factors (e.g., "RM factor 1") are not automatically added to the model components. This looks like a QML issue to me. Interaction effects specified as nuisance are also not properly handled (I opened an issue for this).
The name of the descriptives container is visible (will be fixed by Tim).
There could be more containers to create a nice hierarchical dependency structure.

vandenman · 2019-02-28T21:42:45Z

Travis fails because of the network analysis.. 😕

TimKDJ

Frans just updated the R framework; I'm guessing bootnet was updated and their output changed.

TimKDJ · 2019-03-01T09:21:44Z

Before I delve deeper into your R code, can you fix the tabs/spaces? There are a lot of misaligned lines which makes it kinda hard to read. (I'm guessing the original files had different spacing and you copied some stuff?)

AlexanderLyNL · 2019-03-01T09:29:53Z

Bootnet did change recently and some of the computations were fixed, which is why the output changed.

https://github.com/SachaEpskamp/bootnet/blob/master/NEWS

AlexanderLyNL · 2019-03-01T09:30:07Z

Great work on the code by the way.

vandenman · 2019-03-01T10:18:46Z

@TimKDJ sorry, the code should be more legible now!

TimKDJ

Okay, so you made lots of changes and understandably that introduces some bugs. I added code specific comments and also noted the following problems when testing:

If you change the Bayes Factor type then the entire table is removed and a new progress bar appears aswell (but also when you tick “Descriptives”, or “effects”…)
It feels strange that the progress bar runs twice when you tick “estimates”, people can’t track overall progress now
Descriptives table has some errors:

Model averaged posteriors plot does not handle interactions very well:

When you change options not related to the Analysis of Effects table (Compare to best model / % credible interval) then the table is not reused
If you add some fixed factors and then afterwards tick “Individual plot per level” the progress bar goes haywire
Colors of Model averaged posteriors don’t feel very jaspy. They’re ggplot defaults?
Post Hoc Test note is repeated “Note. not enough observations” also not clear to what row it points:

Marginal posterior plot for RM ANOVA generates a plot per row in the dataset
custom y-axis label of the descriptives plot in RM ANOVA does not work
We have the note “Note. All models include Region” do we then also need “Null model (incl. Region)” in the table? (also in 0.9.2.)
Bayesian ANCOVA fails (“Error in seq.default(2, lInd, 2): wrong sign in 'by' argument”) when one covariate and one fixed factor are added and “Model averaged posteriors” is ticked:

Should we also take Log(posterior odds) in post hoc tests when LogBF is ticked? (also in 0.9.2.)
If you try to create plots in “Single Model Inference” you cannot have a covariate in the “Specific model terms” otherwise only empty plots are generated (even when you do have a fixed factor)

JASP-Engine/JASP/R/commonAnovaBayesian.R

TimKDJ · 2019-03-13T14:20:40Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

+    ys <- ys + weights[i] * density(samples[[i]], from = fromTo[1L], to = fromTo[2L], n = n)$y
+  }
+
+  return(list(x = xs,


Maybe set an upper bound and change the calculation if the size is too big?

Resources/Common/qml/AncovaBayesian.qml

JASP-Engine/JASP/R/commonAnovaBayesian.R

vandenman · 2019-04-12T10:19:31Z

@TimKDJ I think this is ready for another review. Three things I wasn't able to fix:

If you add some fixed factors and then afterwards tick “Individual plot per level” the progress bar goes haywire
Colors of Model averaged posteriors don’t feel very jaspy. They’re ggplot defaults?
Post Hoc Test note is repeated “Note. not enough observations” also not clear to what row it points:

For the post hoc, there is only one footnote but it's still not shown in the table where it points to.

Oh I also still need to implement the memory-safe but slow alternative.

TimKDJ

I like the interface much better now :)
The progress bar flashing is still a bit funky, I'll have a look at what's happening there.

I think the main issues have been resolved and I have markedly fewer comments ;)
Aside from the in-code comments:

The y-axis label for the RM ANOVA doesn't seem to actually be added to the plot.

TimKDJ · 2019-04-12T12:35:26Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

+    descriptivesTable$addFootnote(
+      symbol = "<em>Note.</em>", 
+      message = sprintf(
+        "Some combinations of factors are not observed and hence omitted (%d out of %d combinations are unobserved).",


You'll need a different specifier than %d: "Formats d and i can also be used for logical variables, which will be converted to 0, 1 or NA."

Changed to %g because prod(...) could return a double rather than an integer. I'm not sure why it's a problem that %d accepts logicals, aside from integers. I'd rather see NA in a string than a crash in R. It's worthwhile pointing out that sprintf("%g", NA) just returns "NA", same goes for paste.

Yeah I misread the thing I quoted. But %d complained here

TimKDJ · 2019-04-18T10:45:46Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

-  posteriorPlotContainer$setOptionMustContainDependency("groupPosterior", options[["groupPosterior"]])
+  posteriorPlotContainer$dependOn(
+    options = c("posteriorPlot", "modelTerms", "credibleInterval"),
+    optionContainsValue = options["groupPosterior"]


The syntax for this would be optionContainsValue = list(groupPosterior=options["groupPosterior"]).
Although I think you could also just add it in options?

Note the single brackets for indexing, options["groupPosterior"] is shorthand for list(groupPosterior = options["groupPosterior"]). Still it's better to just add it to options.

TimKDJ · 2019-04-18T10:46:51Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

@@ -822,10 +825,10 @@
    df <- data.frame(x = dd$x, y = dd$y)
    xName <- expression(R^2)
    plot$plotObject <- JASPgraphs::PlotPriorAndPosterior(dfLines = df, xName = xName, CRI = rsqCri, drawCRItxt = FALSE)
-    plot$copyDependenciesFromJaspObject(jaspResults[["tableModelComparisonState"]])
+    plot$dependOn(jaspObject = jaspResults[["tableModelComparisonState"]])


optionsFromObject

Good catch!

TimKDJ · 2019-04-18T10:48:48Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

-  descriptivesPlotContainer$setOptionMustContainDependency("plotSeparateLines",  options$plotSeparateLines)
-  descriptivesPlotContainer$setOptionMustContainDependency("plotSeparatePlots",  options$plotSeparatePlots)
+  opts <- c("plotHorizontalAxis", "plotSeparateLines", "plotSeparatePlots")
+  descriptivesPlotContainer$dependOn(optionContainsValue = options[opts])


This would be identical to descriptivesPlotContainer$dependOn(c("plotHorizontalAxis", "plotSeparateLines", "plotSeparatePlots"))

TimKDJ · 2019-04-18T10:50:13Z

JASP-Engine/JASP/R/commonAnovaBayesian.R

-  posteriorPlotContainer$setOptionMustContainDependency("singleModelGroupPosterior", 
-                                                        options[["singleModelGroupPosterior"]])
+  posteriorPlotContainer$dependOn(options = "singleModelPosteriorPlot",
+                                  optionContainsValue = options["singleModelGroupPosterior"])


The value added in optionContainsValue could just be set in options: posteriorPlotContainer$dependOn(c("singleModelPosteriorPlot", "singleModelGroupPosterior"))

TimKDJ · 2019-04-18T13:35:46Z

Could you also resolve all the comments that you have incorporated so far? It's a little messy otherwise.

TimKDJ · 2019-04-23T07:44:28Z

Anything you wish to add @vandenman? If not, I'll merge this.

vandenman · 2019-04-23T11:23:42Z

@TimKDJ, I've got nothing further to add, go ahead!

vandenman requested a review from TimKDJ February 28, 2019 21:17

TimKDJ reviewed Mar 1, 2019

View reviewed changes

vandenman force-pushed the BANOVAupdates branch from c7d871e to e864df0 Compare March 1, 2019 10:17

vandenman force-pushed the BANOVAupdates branch from e864df0 to e75541f Compare March 4, 2019 16:20

vandenman force-pushed the BANOVAupdates branch from e75541f to 70ffd74 Compare March 11, 2019 17:38

vandenman mentioned this pull request Mar 12, 2019

In Bayesian ANOVA, choosing BF10 or BF01 changes the the Model Comparison output, but not the Analysis of Effects jasp-stats/jasp-issues#238

Closed

vandenman force-pushed the BANOVAupdates branch 6 times, most recently from fbb38be to 7095248 Compare March 28, 2019 10:17

AlexanderLyNL requested a review from TimKDJ March 30, 2019 13:20

TimKDJ suggested changes Apr 9, 2019

View reviewed changes

vandenman force-pushed the BANOVAupdates branch from 7095248 to a935d65 Compare April 11, 2019 12:37

vandenman added 9 commits April 16, 2019 14:57

added qqplot

a7d3d6b

now calculates posteriors and shows model averaged results

83285c8

added single model inference

82baef0

implemented Rsquared

4f93a5b

fixes and cleanup

a26c7bc

started on jaspResults

16a54e1

updated unittests

2fa8382

whitespace + minor bugfix

ac9f698

bug fix in plots + rebase

49195ca

vandenman added 7 commits April 16, 2019 15:00

BF in effects table now changes to BF01 and BF10

8d887bc

credible intervals can now be changed

e890425

fixed unit-tests

647b86f

fixed unit-tests part 2

4cb50d0

fixed many bugs

d98f581

more bugfixes

a6e031b

fixed unit-tests and qml cleanup

fd9decb

vandenman force-pushed the BANOVAupdates branch from 52b13e5 to 577a8d9 Compare April 16, 2019 19:17

rewrite of jaspResults methods

b1b0226

vandenman force-pushed the BANOVAupdates branch from 577a8d9 to b1b0226 Compare April 16, 2019 22:13

TimKDJ suggested changes Apr 18, 2019

View reviewed changes

vandenman added 2 commits April 20, 2019 22:33

fixes after 2nd round of feedback

39dd693

small update to credible intervals

e0a6056

TimKDJ approved these changes Apr 23, 2019

View reviewed changes

TimKDJ merged commit f97b1e3 into jasp-stats:development Apr 23, 2019

vandenman deleted the BANOVAupdates branch August 7, 2019 08:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bayesian Anova updates #3071

Bayesian Anova updates #3071

vandenman commented Feb 28, 2019

vandenman commented Feb 28, 2019

TimKDJ left a comment

TimKDJ commented Mar 1, 2019

AlexanderLyNL commented Mar 1, 2019

AlexanderLyNL commented Mar 1, 2019

vandenman commented Mar 1, 2019

TimKDJ left a comment

TimKDJ Mar 13, 2019

vandenman commented Apr 12, 2019 •

edited

Loading

TimKDJ left a comment

TimKDJ Apr 12, 2019

vandenman Apr 20, 2019

TimKDJ Apr 23, 2019

TimKDJ Apr 18, 2019

vandenman Apr 20, 2019

TimKDJ Apr 18, 2019

vandenman Apr 20, 2019

TimKDJ Apr 18, 2019

vandenman Apr 20, 2019

TimKDJ Apr 18, 2019

vandenman Apr 20, 2019

TimKDJ commented Apr 18, 2019

TimKDJ commented Apr 23, 2019

vandenman commented Apr 23, 2019

Bayesian Anova updates #3071

Bayesian Anova updates #3071

Conversation

vandenman commented Feb 28, 2019

vandenman commented Feb 28, 2019

TimKDJ left a comment

Choose a reason for hiding this comment

TimKDJ commented Mar 1, 2019

AlexanderLyNL commented Mar 1, 2019

AlexanderLyNL commented Mar 1, 2019

vandenman commented Mar 1, 2019

TimKDJ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vandenman commented Apr 12, 2019 • edited Loading

TimKDJ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimKDJ commented Apr 18, 2019

TimKDJ commented Apr 23, 2019

vandenman commented Apr 23, 2019

vandenman commented Apr 12, 2019 •

edited

Loading