Vignettes vs pipelines vs case-studies

[Bisaloo]

At the moment, the terms vignettes, pipelines, case-studies seem to be used somewhat interchangeably and their precise respective scope is unclear. This leads to:

• a less readable documentation for users since it's not clear where they should be looking for a specific form of documentation
• uncertainty for developers about where certain documents should live
• possible duplication of effort by having different developers working independently on the same thing in different locations

We should clarify the scope and limit the overlap between these three concepts.

This has implications for epiverse-trace/episoap#85, tasks 22 and 23 from https://github.com/orgs/epiverse-trace/projects/24.

[adamkucharski]

Wonder if a stratification like this one would be useful starting point: https://documentation.divio.com/

Thinking about how I typically use the terms, some potential definitions:

• Pipelines are the code bases that integrate relevant packages, sometimes with a template document output.
• Vignettes (which I realise is R specific terminology) are closer to the 'how-to' quadrant in above link, showing the code steps required to walk through different tasks
• Case studies are specific illustrative examples of real-life applications to support understanding (so often included as a subset of a vignette or in training materials)

Although keen to hear thoughts on why these are not optimal!

[avallecam]

@adamkucharski Wonder if a stratification like this one would be useful starting point: https://documentation.divio.com/

It's a good starting point, indeed. Some comments:

Pipeline, the concept, I'm defining as a set of connected data analysis tasks required to obtain an informative outcome for decision-making purposes.

• Pipelines as reproducible report templates with parameters that integrate multiple packages live in the {episoap} package.
• Pipelines as code scripts that integrate multiple packages in a "How-to guides" format can live in the {howto} repo.

It was useful for me to look into the {episoap} transmissibility pipeline to study how multiple packages are connected, and inform me to write "How-to guides" which help me for coming "Tutorials". The contribution could potentially go also in the other way round.

Vignettes are "How-to guides" that belong to one package only, explicit in their methods and data outputs, and limited by their data inputs. For example, {finalisize} and {epichains} need to assume R0 values (among others like SI and k) and do not need to prioritize how to calculate them.

• Vignettes are currently a mix of "How-to guides" and "Explanation" primers, given the use of theoretical concepts for argument decisions. Further efforts could go to isolate the Explanations sections to a different repo (glossary-like?) or a different vignette to facilitate specific links to concepts/definitions (treated in more detail) from anywhere.

Case Studies are real-life, retrospective accounts of real projects that you have delivered for real customers. They can be confused with "Use Cases". Use Cases are examples of how a product or service might be deployed (e.g., any "How-to guides" like Vignettes or {howto} entry).

• Case Studies tell the stories of real customers tried and tested solutions, while Use Cases present examples of solutions to possible problems. Both are useful pieces of collateral, demonstrating both your actual and your potential delivery capability.

Lastly, thanks to pkgdown, "Reference guides" will always be in the "Reference" tab accessible via ?help.

I hope the definitions above clarify some differences. Happy to read your questions in this regard too.

[avallecam]

Drafted a concept map relating these concepts with our materials:

[Bisaloo]

Thanks Andree, this is useful to clarify the different concepts! 👍

I'm still unsure about when a pipeline should be in episoap vs the howto repo:

• Pipelines as reproducible report templates with parameters that integrate multiple packages live in the {episoap} package.
• Pipelines as code scripts that integrate multiple packages in a "How-to guides" format can live in the {howto} repo.

In other words, what should be the scope of episoap given the other, related, forums we have to provide example of pipelines with multiple packages?

The clear delineation is very to determine. For example, for a short while at the start, epiparameter provided a vignette which was serving a very similar purpose as the episoap transmissibility report.

More recently, for the severity pipeline (or any other pipeline we develop in the future), what should lead us to decide whether it lives in howto or episoap?

[avallecam]

Thank you for your question. I will compare both with respect to their goal, target audience, and infrastructure.

goal:

• episoap stores curated outbreak analytics pipelines as Rmarkdown reports.
• howto stores recipe-like solutions to task-specific questions as code scripts.

target audience:

• episoap target audience includes novice learners that need pre-written code to reuse. Competent practitioners will be able to edit and extend them with more advance understanding of outbreak-analytic packages and Rmarkdown. (let me know if this is not an accurate assumption)
• howto target audience are competent practitioners. A how-to guide is an answer to a question that only a user with some experience could formulate (ref).

infrastructure:

• episoap provides pre-written code for a self-reproducible environment and efficient connections between all the pieces of the pipeline. This includes a parametrized configuration to change downstream paths with child reports. It additionally includes text for interpretation
• howto provide a code script that integrates multiple packages. This must be reproducible with the latest versions of the packages. This should not start from the very beginning of the analysis process. This needs to be of practical usability to remember how to do something in the same place.

[avallecam]

How do we decide whether it lives in howto or episoap (or case studies)? Given the comparison above, I understand that overlapping will benefit the exchange of code, questions, and tasks. The difference is in their goal, target audience and infrastructure.

• episoap must host any code that makes connections optimal and effective for a Rmarkdown report. The goal is to reduce the technical burden on the user and to facilitate package usage with more interaction in an early stage. Flexible to new inputs and outputs.
• howto must host any code script solution to specific and generalizable questions, like a reprex. This does not need to be linked with a particular outbreak scenario or real raw data.

Given these questions, I'm willing to update or remove any concept that may generate more confusion than understanding. I attached an updated version of the previous concept map.

updated after publication:

[avallecam]

I'm interested in evaluating how effective the concept of the how-to guide is for our purposes, thinking about competent practitioners. The case studies are in the middle because I relate them with personas like:

• Juan is a Statistician. He supports the data analysis tasks in the outbreak response team of a National Health Agency. He wants to reuse code from disease-specific analysis reports and suggest to epidemiologists how to choose between different methods. Read more about Juan.
• Danielle is an Instructor. She has outbreak analytics experience at the Ministry of Health and now develops training materials for her colleagues. She wants to remix content to create specific training materials for public health practitioners like Field epidemiologists and research degree students. Read more about Danielle.
• Lucia is a Field epidemiologist. She uses R to clean data and create plots for outbreak response. She wants to learn how to apply the infectious disease modelling R packages she learned from tutorials in a real-life project. Read more about Lucia.