Inferring psychophysical functions from data using RStan applied to a "Bayesian" (MLE) cue fusion example.
Implements modifications of models from M. D. Lee & E.-J. Wagenmakers. Bayesian Cognitive Modelling. Cambridge: Cambridge University Press. 2014., Chapter 12 for inferring the paramters of a psychometric function from data. Mainly a version of the model shown in their figure 12.5. It is applied to a "Bayesian" / MLE cue fusion example, but parts of it may also be informative for someone interested in fitting psychophysical functions in other experiments, especially the reparametrization of Lee & Wagenmakers' original model (see Model versions).
Besides R and some standard packages, this code requires a working installation of RStan. Getting RStan to run may be slightly non-trivial.
The main execution script is Psychophysics_BasicAnalysis.R. Running it will generate outputs (plots as .pdfs) based on the provided RStan samples that were fit to the provided, simulated example data with the default settings (2 chains à 2000 samples).
CAVE: The data_examples are not good fits/samples. Diagnosing them based on the commented-out code in Psychophysics_BasicAnalysis.R may be instructive. Based on experience with human participant data, it is assumed that increasing to 6 chains à 2000 samples with 500 warmup should produce acceptable results on ~15 participants.
The original purpose of this adaptation was to analyze experimental data generated by a 2-alternative forced choice psychophysics experiment for duration estimation implemented as part of a lab course for psychology undergraduates. The task was modelled after Hartcher-O'Brien et al. 2014. Data were collected by a custom jsPsych experiment and stored to JSON. When the experiment code is made public, it will be referenced, here. The provided example data and data generated by the simulator instead start from already-aggregated data in a similar format as that used by the examples to Lee & Wagenmakers 2014.
The model used for the example is a version of Lee & Wagenmakers' group-level hierarchical model with contaminants (their figure 12.5). Because the original model did not work well on our human particpiant data, a reparametrized version of the model is used. Two additional files provide implementations of the group-level model without reparametrization, as well as of an individual-level model. The former should work (may be subject to bad sampling) but has not been tested thoroughly. The latter is a legacy version and would require modifications to the current code in order to work.
- The data simulator is currently a rough sketch. Besides refactoring, it would be neat to keep it closer to the model so that one can simulate data from the same priors used for fitting and conduct parameter identification experiments.
- Instead of fitting separate psychometric curves for each condition and then testing Bayesian cue fusion by the "MLE-method", it would be desirable to add cue fusion as an additional hierarchy level to the stan model and stay fully Bayesian.
- CAVE: This code was generated for teaching purposes and can at best inspire, but not be used 1-to-1 for analyzing an actual experiment. Before doing research with it, at the very least, parameter identification experiments on simulated data should be conducted and resulting samples diagnosed to verify the overall robustness of sampling and the goodness of inferences. Furthermore, a little outside the scope of curve fitting, I learned from conversations with visual psychophysicists that data collected in a single experimental session is typically too noisy for good estimates of JND and one would instead use additional session/s calibrated based on the initial one/s.
Inspiration by Benjamin Knopp and Stan examples on the topic - also see licensing!
Contributions by Ruslan Spartakov.
Generated and tested on Windows 10 with RStan 2.19.3 and R version 4.0.0