cargo new: create a README

[steveklabnik]

Was talking with @ceejbot about Rust and docs on twitter, and she had a great idea:

Also use the social engineering of cargo! Have it write a template readme for new projects!

Bundler, for example, already does this:

    $ bundle gem foolol
    $ cat foolol/README.md
    # Foolol

    Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/foolol`. To experiment with that code, run `bin/console` for an interactive prompt.

    TODO: Delete this and the text above, and describe your gem

    ## Installation

    Add this line to your application's Gemfile:

    ```ruby
    gem 'foolol'
    ```

    And then execute:

        $ bundle

    Or install it yourself as:

        $ gem install foolol

    ## Usage

    TODO: Write usage instructions here

    ## Development

    After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

    To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

    ## Contributing

    Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/foolol.
    ```

[ghost]

The format that bundler is not what I would put. I think the SYNOPSIS approach pioneered by cpan and largely copied by many npm packages is the most important thing to have in a readme. Always at the top with a short code example that shows how to use the package. Starting with how to install a package is less helpful because when somebody first encounters a package, they don't know if a package appropriately solves their problem yet.

[steveklabnik]

Yeah, I'm not saying we should literally copy Bundler.

[alexcrichton]

Sounds like a great idea to me!

[carols10cents]

In the survey we ran, people also mentioned going to the documentation and wanting to see readme-like content (purpose and an example) on the front page of the docs. I think we could encourage better documentation front pages by adding a module comment to the top of the generated src/lib.rs that's something like:

//! This is the top-level module documentation in src/lib.rs.
//! This text appears on the front page of your crate's documentation;
//! preview it by running `cargo doc --open`.
//!
//! # Example
//! 
//! This is a great place to add an example of how to use your crate!
//!
//! ```
//! assert(true);
//! ```

The text talks about where it lives and where you'll see it so that if you're looking at that file, you'll know where the text ends up, and if you're looking at your generated docs, you'll know where to go to edit the text you see on the front page.

[sanmai-NL]

Creating any file as part of a project template implies that the file should be standardized in format, IMO. This is hard to do for informal/semi-structured data such as documentation. Also, the default markup language itself needs to be settled, which will lead to discussion.
Maybe a good/better first step would be toward an RFC/standard for project documentation? In that discussion Rust API documentation standards should be included as background, I think.

[steveklabnik]

This is hard to do for informal/semi-structured data such as documentation.

While this is true, there's some stuff that's common to all projects, and if it doesn't work for you, you can delete it.

Also, the default markup language itself needs to be settled, which will lead to discussion.

This is already markdown, by the vast majority.

Maybe a good/better first step would be toward an RFC/standard for project documentation?

We've had two of those! https://github.com/rust-lang/rfcs/blob/master/text/1574-more-api-documentation-conventions.md being the most recent, that built on that one.

[sanmai-NL]

Disagree on your first point. The same is true for e.g. an issue template on GitHub, ultimately rejected (rust-lang/rust#33164) because it would inconvenience those who feel it mismatches with their issue. Perhaps more people find the cargo-generated READMEs noisy (extraneous, inadequate) than helpful.

Mildly disagree with your second point. AsciiDoctor is another format that is IMO far more appropriate for software documentation than Markdown (some motivation via https://news.ycombinator.com/item?id=11292280). Discussion about this even in the Rust context is normal (https://internals.rust-lang.org/t/rustdoc-asciidoctor-vs-markdown/4161).

I am not referring to API documentation, but to user documentation. Do you agree it should be decided what content is actually ‘common to all projects’, next how to optimally present it, based on a study of evidence, before putting out README templates with cargo?

[vignesh-sankaran]

I'd be happy to take this up. I'd like to establish the scope of this particular issue.

Would this be the default behaviour with the cargo new command? And would there be a flag to disable automatic documentation creation?

My understanding of the scope so far is that there is to be a template for a README.md and a standard template for module level documentation comments. Have we agreed on a standard format regarding the format of the former?

Did we also want to warn the user on a cargo package or cargo publish if the documentation templates haven't been modified from initial creation? How about in the event that there is no documentation in the crate an author might be about to publish?

[steveklabnik]

I am not on the Cargo team, but I'd expect it to be part of the default behavior of cargo new.

I just became aware of https://github.com/RichardLitt/standard-readme, I haven't actually read it yet though.

[vignesh-sankaran]

That project looks good, I'll take a look at it. Did we want a flag to disable default documentation creation?

[sanmai-NL]

@vignesh-sankaran: Can you please address @carols10cents comment on the doc comment template? To me that is far less controversial and more productive than creating a README file in a markup format whenever Cargo is asked to initalize a new project. Can you either implement her proposal first, or can we split her proposal into a different issue?

Creating a README.md file by default implies we assume GitHub-like browsing of the new project and preference for Markdown. Probably right for most projects (but no evidence was presented here by @steveklabnik), but where do we stop with this kind of feature creep? Why not put a complete project template with various config files (e.g. Travis CI, etc.) in place? Frankly, I strongly prefer an alternative, comprehensive solution: a way for Cargo to create a new project according to a template, completely independent of README files. It is fine to me if Cargo were to have default templates for libraries and main executables that do have a README.md. But the default templates should not be arbitrarily decided. An RFC-like consensus on the template content would be far more appropriate. I foresee it will be very annoying for programmers to have to clean out newly created Cargo projects from default cruft as @steveklabnik seems to accept as a solution. Alternatively and a bit more expedient than an RFC-like consensus, would be to start separate projects for each template and to let issues be filed against it by the community to settle various details.

[sanmai-NL]

An argument can be made that standard tools (e.g., rsync) suffice to scaffold a newly created Cargo project based on a standard template. From that point of view, adding code to scaffold projects to Cargo would not be useful and indeed add to the Cargo maintenance burden.

[vignesh-sankaran]

I'm not going to be able to work on this until mid November due to university and not wanting to overwork myself. If anyone else wants to take up this issue they are more than welcome to do so, otherwise I'll be able to take this up in a month's time.

[coder543]

An argument can be made that standard tools (e.g., rsync) suffice to scaffold a newly created Cargo project based on a standard template. From that point of view, adding code to scaffold projects to Cargo would not be useful and indeed add to the Cargo maintenance burden.

@sanmai-NL can that argument actually be supported? Windows doesn't have rsync, for example, and there's no guarantee that any given Linux install will have rsync either. In fact, Cargo already has all the tooling necessary to support templating, in my opinion. One simplistic approach would be to allow anyone to publish projects as templates to crates.io, with the tag 'template' or something similar. Then anyone could cargo new --template template_name project_name, and as long as the template_name crate had the 'template' tag, cargo could just clone it into the current directory as project_name, using the same facilities that are already used during dependency resolution to pull crates from crates.io.

I also don't agree that README.md implies GitHub in any way. All of the major code sharing websites that I'm aware of support this design, including GitLab, which is likely to be popular among people who shy away from proprietary platforms. README itself is an ancient standard that existed long before GitHub. README.md is just a modernization of it, and Markdown is perfectly readable without rendering. It's not nearly the same as integrating with a third-party service like Travis, in my opinion.

[coder543]

More on topic, I have been thinking a lot about how nice it would be to have cargo automatically generate a README.md file when creating a new project. This thought was spurred by seeing how Crystal does it, and seeing what a great idea it is in my view.

These thoughts together have led me to wonder if cargo should have an interactive mode to let you configure things, assuming that cargo new is invoked with no other arguments. Picture this:

$ cargo new
enter project name:
should this contain a library? [Y/n]
should this contain a binary? [y/N]
add a default README.md? [Y/n]
which VCS to use? [GIT, mercurial, pijul]

which would also allow users to elect to have a library+binary crate, which is an increasingly popular way of structuring Rust projects. Obviously, this is a very rough sketch of an idea that I feel is an extension of the idea proposed in this issue. I could open another issue, if desired, or there might already be one for this, but I didn't see one.

[sanmai-NL]

@coder543:

On rsync and standard tooling for project scaffolding

Windows has robocopy. One may also just use cp on Unix-like OSs. My larger point is that starting a project from some scaffold does not require a Cargo feature. I'm not strongly against it but if such a feature is added I feel strongly that it should be designed and maintained intensively, not added as a new CLI or default behavior haphazardly.

READMEs

You have a good point that the README file is concept with a longer history than just GitHub. A point I want to make beyond this fact is, how do we decide what (markup) language this document should be in? I would not prefer it to be Markdown or something else specific by default. And Markdown is mostly popular on GitHub and other open source centric sofware development collaboration platforms. And another point is, not all Rust projects are meant to developed as a typical open source project. Rust is also supposed to be attractive for large closed source enterprise projects. Have Cargo create a README file by default, while programmers for large projects actually plan to use an extensive technical documentation framework such as DocBook/Asciidoctor/Wiki/etc., confirms the perception of Rust as ‘just’ ‘the new shiny language popular in the open source world’. I do not think this is a huge problem per se, but think about how creating all kinds of stuff by default imposes a certain development style/culture.

Practical solutions

• Interactive mode for Cargo (idea of @coder543). This will assist novice Rust programmers to create typical (open source style) projects from scratch. Though the CLI suffices, my experience and intuition working with novices (and being a novice to many things myself) is that an interactive mode really reduces friction and makes things more joyful.
• Allowing Cargo to mutate existing projects. The programmer copies a template, and Cargo can then mutate it (e.g., all the options that can be set via the interactive mode upon creation).

[vignesh-sankaran]

Ok so I'm back, the interactive mode sounds like a good idea, but we should also have a way around it for more experienced users. At this point, should we be considering an RFC for this issue, or at least some sort of scope definition so that I know what to work on?

@carols10cents's suggestion regarding doc comments does sound like a great idea, maybe I could get started with that in the mean time?

[matklad]

Nominating for discussion at the next Cargo meeting: while it is clear that we want to encourage and simplify documentation of Rust crates, it's not clear what is the best way to achieve this. In particular, creating a dummy Readme.md and/or top-level comment is not a clear win, because:

1. Often one creates one-off temporary crates, or private internal crates, not published to crates.io, and in these cases it's totally valid to not have the docs, and default docs would just create friction.
2. We need somehow to protect from uploading crates with default docs/readme.

An alternative approach would be to add "preflight checks" on cargo publish. For example, we can warn during publish if not fields of Cargo.toml are filled in, if the Readme is missing and if there is no documentation.

[vignesh-sankaran]

As suggested by @matklad, we could have warnings regarding missing documentation that are run as checks prior to an initial publish. Perhaps we could also run these checks subsequently if it failed them initially until they are resolved or indefinitely.

Checks that come to mind that we could run include:

• Top-level documentation comments
• Missing README field in Cargo.toml if one is present
• Comments for all public functions

[carols10cents]

I actually don't see much benefit in spending a lot of effort to prevent publishing of crates with default docs/readme... if you're intending for other people to consume your crate, you're going to change those, and if you're not ready or ever intending other people to want to use your crate, they likely won't if you publish with the default docs :)

[wycats]

I originally shared @carols10cents's sentiment, but when I thought about it more I realized that at least in node, I quite often quickly create a templated package and publish it immediately.

I think that's still a desirable thing to be able to do, and I loathe the way that Rubygems forces me to fix a bunch of pointless TODO templating in its default gemspec, so I was thinking maybe a middle ground would be to detect the template and not render its contents?

That way people who publish the template are no worse off than today, but we also give people a nice starting point for documentation.

[sanmai-NL]

@matklad: I kindly remind everyone that README.md isn’t the only possible README filename, there are other markup languages too. Just to be sure this is clear ...

I’m in favor of having cargo publish emit warnings (at minimum) if some QA issues are found. One more important issue IMO than the README would be lack of an explicit maintenance status. in Cargo.toml.

# Maintenance: `status` is required Available options are `actively-developed`,
# `passively-maintained`, `as-is`, `none`, `experimental`, `looking-for-maintainer`
# and `deprecated`.
maintenance = { status = "..." }

This helps programmers filter out a lot of crates when browsing crates.io depending on their requirements. After that, the API docs are relevant, only when that is okay that a README would the next thing to check, IMO.

[wycats]

@sanmai-NL We could default to README.md even while still supporting other options for the README. Is there another markup style you think would be a better default?

I’m in favor of having cargo publish emit warnings (at minimum) if some QA issues are found.

I'm generally opposed to warnings without teeth, as my experience in other ecosystems is that people come to ignore them habitually, and they just add to the noise level of the general experience. As the noise level gets worse, there is "noise inflation" as people try to get "the really important stuff" to be loud enough to be seen, which creates a vicious cycle.

If we want to stop people from publishing crates with QA issues, we should stop them. If we want to do this, we should also probably give some thought to automating the process of fixing the QA issues.

[sanmai-NL]

@wycats:
I so far use Asciidoctor for software documentation. My point is not that my personal preference should be the default, but that any ‘default’ markup language and a README template with the corresponding file name may be misguided. If we go down this path, we’d be imposing not only the existence of a README (in a certain place in the project structure), but even a particular syntax for it. That’s not dramatic but just shouldn’t be done casually.

Agree about warnings vs. errors, but I find many people oppose vehemently to be forced to comply with best practices.

[wycats]

I do still want to point out one thing. @carols10cents, you introduced the qualification ‘not ready for use yet’, yet @wycats now concedes there is no clear definition of the notion ‘ready for use’. I personally believe it’s easy to list a few quick objective rules about what certainly spells insufficient quality wrt. gaining publication in an official software component registry.

For what it's worth, I think that we do have such a definition right now, encoded (as you suggest) through publication-time errors. They include a restriction on the use of * and dependencies outside of crates.io. I could imagine adding more rules to that set, but as I tried to elaborate, the bar is high 😄

[wycats]

As industry user of Rust, it seems as though friendliness to hobbyist programmers overrides quality assurance of an OSS ecosystem, in a decision such as this one. These dimensions are not in conflict, but depending on how you approach both they can be, I think. Hopefully people with (non-Mozilla) industrial interests will have a bigger say in the future.

I'm on the Cargo team, don't work for Mozilla, and my primary interest, like yours, is my use of Rust in production. We even advertise the fact that we use Rust as an important feature in our product:

I don't disagree that it can be difficult for production users to provide good input into the process, because being a production users by definition means you don't have time to review issues and respond as much as what you're calling "hobby users".

I also believe that the process, for this reason, is a little too tilted towards people who have time to write a lot of comments on issues, and it's something that I think there's a lot of appetite for improving. I'd be very happy to talk with you (publicly or privately) about these concerns and ways that we can improve our process to address them.

(I also don't believe that production users are well served by turning many small decisions into RFCs, because the current RFC process is dominated by what you're calling "hobby users", simply because they have more time to respond many times per day to an RFC thread)

[stale]

As there hasn't been any activity here in over 6 months I've marked this as stale and if no further activity happens for 7 days I will close it.

I'm a bot so this may be in error! If this issue should remain open, could someone (the author, a team member, or any interested party) please comment to that effect?

The team would be especially grateful if such a comment included details such as:

• Is this still relevant?
• If so, what is blocking it?
• Is it known what could be done to help move this forward?

Thank you for contributing!

(The cargo team is currently evaluating the use of Stale bot, and using #6035 as the tracking issue to gather feedback.)

If you're reading this comment from the distant future, fear not if this was closed automatically. If you believe it's still an issue please leave a comment and a team member can reopen this issue. Opening a new issue is also acceptable!

[coder543]

I still think this is worth doing, but it seems to have been mired in tangential discussions and then the conversation dwindled to a halt.

[dwijnand]

I think this is blocked on the larger RFC-sized #5151 about cargo templates in general, which I think was on the agenda but got pushed for other edition priorities.

[sanmai-NL]

@coder543: tangential discussion, or rather critical discussion whether adding this feature is what we really need.

[Eh2406]

If the decision at rust-lang/rfcs#2549 (comment) is upheld by the teem then this should be closed asswell.

[coder543]

@Eh2406 I don't see why. Editorconfig is a niche technology. READMEs are a rather fundamental thing. Since the README is used as part of crates.io, it's arguably a minimal file needed to create a new Rust project.

Not to mention that rustfmt already fills the role of Editorconfig in the Rust world. This issue is also already blocked on templates, so it might just become part of a template.

[dwijnand]

There's an argument (for example, as presented by @withoutboats in #2039 (comment)) that things should be, hmm, less "dictated" by crates.io.

[Voultapher]

Imo, the arguments made in rust-lang/rfcs#2549 also apply here, cargo should remain lightweight and only do build system and dependency management related things. Anything extra, like README, Editorconfig, CONTRIBUTING, etc. should be an ergonomic opt in with cargo templates. A popular one could be an OSS template, that has the 3 aforementioned files plus LICENSE.

If we start making exceptions for things like README, where do we draw the line? Also the cargo template story might be less clear, teaching that for anything that is not related to building the software like README files you can use cargo templates, is simple. Unfortunately there are already exceptions to this, like vcs, let's not make the situation worse.

[sanmai-NL]

@steveklabnik See https://github.com/ashleygwilliams/cargo-generate and https://github.com/livioribeiro/cargo-readme. Now that solutions for the issues discussed here have sprung from the OSS ecosystem, I think it's better to close this issue, since it proposes new work, and instead focus on improving and/or integrating them into Cargo.

[Dylan-DPC-zz]

@steveklabnik any updates on this? if needed we can mark this as e-easy and get some volunteers to do it :D

[steveklabnik]

I’m not aware of any, let’s do it!

…

On Dec 23, 2019, at 12:47 AM, Dylan DPC ***@***.***> wrote:  @steveklabnik any updates on this? if needed we can mark this as e-easy and get some volunteers to do it :D — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Dylan-DPC-zz]

@steveklabnik I don't have permissions on this repo so you (or someone else) will have to do it :)

[after-ephemera]

Hey @Dylan-DPC and @steveklabnik. I'm a Cargo user and noob to the open source project. I'd like to take a crack at this if you can give me some guidance as to the implementation plan. After reading through the comments here it looks like there are quite a few opinions as to what approach to take.

[steveklabnik]

I am not on the cargo team, so I cannot really make that call.

[jsjoeio]

Per some discussion and guidance in #8029, I'm going to start working on an RFC.

[jsjoeio]

I posted a Pre-RFC on internals. Here's a link in case anyone would like to participate in the discussion:

[Pre-RFC]: cargo new templates

[epage]

Based on this comment

I'm going to move to close this, and defer to #5151. In #6881 the team explicitly decided not to add more boilerplate or content to the default manifest beyond the link to the documentation. I realize discovery and education is difficult, but I think on balance adding a large section that everyone will need to delete has its own downsides. I'm not sure if configurable templates would be the solution, but I think it would be a pre-requisite if we were to decide to make this default to offer a simple way to turn it off.

So I'm going to close this likewise