Need to improve toy generation documentation

[adewit]

In particular, explicitly point out what happens with the NP constraints when generating toys, since this seems to be unclear (judging e.g. by questions on cms-talk).

[nucleosynthesis]

Just to orient this issue - This is the section where it is described : http://cms-analysis.github.io/HiggsAnalysis-CombinedLimit/part3/runningthetool/#nuisance-parameter-generation (would be helpful to know what should be added to make the explanations clearer)

[adewit]

Thanks, I need to think about it more (mainly put the issue here not to forget about it). At a minimum, we should address:

• Some people are under the impression that with --toysFrequentist the post-fit NP constraints are used in the toy generation (the fact that this is not the case is somewhat hidden in the current description, in the sense that it's easy to misinterpret "the constraint terms are instead randomised within their Gaussian constraint pdfs" )
• Confusion over whether post-fit constraints between the NPs are taken into account in the toy generation

There may be other things we could improve

[kcormi]

Perhaps, to the first point, a small section in the "what combine does" docs proposed in #839 could help?

Part of my interpretation of the problem is that people don't really understand how the constraint terms work, and so to your point, I don't think that "the constraint terms are instead randomized ...." is clear. Perhaps adding a section could be added under the "Fitting Concepts" page with a note about generating pseudodata, which could explain this, also making reference to how the constraint terms are defined in the likelihood section, and then with links to the code section you've identified. And the code section could also back link to the general explainer section on generating pseudodata.