Vignette standards

[noamross]

Our standards require that packages have vignettes, but we don't specify much about what they should contain. I believe we should have at minimum guidance, and probably some requirements, as to the content of some vignettes. People use vignettes for many purposes, including providing use cases or the package, packaging a paper related to the package, or arguing for the package's approach. I think we should require that at least one vignette provide an overview of the major functionality of the package and typical workflows, that complements the function-level documentation. A recent review by @juliasilge made this point really well and led to a great improvement in the documentation.

A secondary point is the relationship between a package README and its vignette(s). We expect a README to be a high-level overview of the package. As people have come to expect the README be a "landing page" for a package because of the way GitHub displays them, much of the function of vingettes has migrated to READMEs. However, READMEs aren't always the point of entry for a package - because of badging and other issues, they don't always appear on CRAN, and aren't always installed on user's machine and accessible via the R documentation. So I think that, even though much of vignette content may end up in a README, these should be duplicated under vignettes(). Perhaps some tooling of common workflows to do this would aid authors in doing so.

@cboettig

[maelle]

• I am in favor of minimal READMEs when packages have a pkgdown website (and we wrote that in the guidelines).

• @jonocarroll wrote a function to transform READMEs into vignettes.

[jonocarroll]

This (r-lib/usethis#327) seemed to be going so well until it was abruptly closed.

Anyway, moved to here: https://github.com/jonocarroll/VolunteerVignettes which is forming my useR talk.

I'm yet to flesh out the feasibility, but I do like the idea of code chunks being referenced with child= arguments in both the vignette and README. This way they're a) tested on build, and b) go where the package goes.

[jonocarroll]

Thoughts on README and in particular vignettes most welcome to be sent to me via any channel.

[amoeba]

I think this is a great idea, and one that'd be helpful to me as a reviewer.

As for specific guidelines, would it be too over-the-top to require all exported functions be used/referenced in at least one vignette ("100% vignette coverage")? As a reviewer, I'd still like to use my brain to assess whether a vignette is really useful and whether or not there are enough but it'd be helpful to have a quantitative metric -- either as a strict requirement or not -- to help with the task of reviewing vignettes.

[noamross]

@amoeba I think requiring 100% vignette coverage would be a stretch, but it would be a useful metric and we could ask the reviewers if the vignette covers the package substantively. Could you file an issue over at ropenscilabs/pkginspector?

[amoeba]

Will do, 🎉

[cboettig]

Stab at some possible guidelines here:

README

The README should strive to be a concise document containing the following the following elements. It should not be intended as comprehensive documentation of the package in most cases.

1. Badges in header,
2. short description of goals of package, with descriptive links to all vignettes (i.e. on pkgdown site)
3. installation instructions,
4. Any additional setup required (authentication tokens, etc),
5. A minimal code examples showing how to do something in the package

Vignettes

All packages should also include at least one vignette, providing a more comprehensive long-form documentation of how the package functions are organized and how they fit together.

• Provide substantial coverage of package functions, illustrating realistic use cases and how functions are intended to interact.
• Functionality likely to be used by only more advanced users or developers might be better put in a separate vignette (i.e. programming/NSE with dplyr).
• Vignettes should include citations to software and papers where appropriate.

[maelle]

Any good existing examples? 🙏

[MilesMcBain]

I really like @cboettig's list as a starter. I can't help but think that the size of the package has to be taken into consideration to some degree.

For example take a large package like ggplot2:

• README ticks off all bar item 2. although it does this partially.
• Vignettes address bullet 2 only.
  However it's probably not controversial to assert that a series of disconnected vignettes would not be the best way to get users literate in ggplot2. A designed learning pathway in the form of a book or online course is probably best. The README does a good job of pointing to these.

Okay now lets consider a small package, a fave of mine, gistr:

• README Ticks off all items, however it also does the job of a vignette by providing substantial coverage of package functions.
• Vignette is just the README repeated.
  I actually appreciate it when a small package takes advantage of the README to give an overview. It's convenient not to have to seek out the additional documentation, but it also immediately tells me something important: Hey, this is all there is to this! And I find that very helpful/comforting when making a decision to use the package. In my opinion packages that do not require a big time investment to learn should be able to flaunt this in their README! I also think repeating the README in a vignette is a fine practice: it lets people get to that information quickly if vignettes are part of their workflow.

Okay and now for 'medium' size, I'm quite a fan of what was done in bowerbird:

• README works exactly as specified in @cboettig's specs
• 2 vignettes cover the bullets 1 and 2, and sort of 3 (citing data sources).

Some great things about the bowerbird vignettes that I think could be abstracted into guidelines:

• Outline prerequisite knowledge to be able to understand vignette up front 💯
• Present a series of examples progressing in complexity from basic to advanced usage.

So in summary I think the size and complexity of a package are important considerations for how the breakdown for the README/Vignettes/Additional resources should shake out. If we wanted to make this notion somewhat objective, we could could look at using measures of reading time, number of vignettes etc. For example:

• If a README can give the whole package overview and be read in under 5 minutes (1250 words) maybe it's fine?
• If a package has more than 3 vignettes maybe it needs a more structured learning pathway. I.e. at a minimum a website that links the vignettes into a progression of some such.

These are just example ideas. I think fuzzy criteria may be more useful than these.

[maelle]

Nice summary @MilesMcBain, thank you!

[maelle]

@jonocarroll if we were to write in the guidelines there's an automatic way to transform the README in a vignette, how would they go about installing your function?

[maelle]

I've just digested all the good points from this thread in the packaging guide.

I think I might need to revisit the structure of that guide itself reg the sections README/Documentation/Documentation website but at least the content is there.

Any feedback welcome here or there (when my commit is deployed, I'm referring to this section and the two following ones).

[jonocarroll]

@maelle I plan for that function to live in a vignette-focussed package (which would likely best be hosted within ropensci) but at the moment it's written as a PR to usethis. I need to spend a little time cleaning it up, so it's WIP. The conversion should absolutely only be considered a first-pass starting point for a vignette, but that's still quite useful IMHO.

If people have some vignette-focussed functions and would like to collaborate on said package, by all means get in touch. This could include a goodpractice-esque validator, tooling for getting chunks connected up, conversions for other export routes, etc...

[maelle]

@dpprdan mentioned https://www.garrickadenbuie.com/blog/dry-vignette-and-readme/, very nice approach.