eRFC: single-file packages ("cargo script") integration #3424

epage · 2023-04-26T21:37:43Z

This experimental RFC adds unstable support for single-file
packages in cargo so we can explore the design and resolve
questions with an implementation to collect feedback on.

Single-file packages are .rs files with an embedded
manifest. These will be accepted with just like Cargo.toml files with
--manifest-path. cargo will be modified to accept cargo <file>.rs as a
shortcut to cargo run --manifest-path <file>.rs. This allows placing
cargo in a #! line for directly running these files.

Example:

#!/usr/bin/env cargo

//! ```cargo
//! [dependencies]
//! clap = { version = "4.2", features = ["derive"] }
//! ```

use clap::Parser;

#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
    #[clap(short, long, help = "Path to config")]
    config: Option<std::path::PathBuf>,
}

fn main() {
    let args = Args::parse();
    println!("{:?}", args);
}

$ ./prog --config file.toml
Args { config: Some("file.toml") }

See cargo-script-mvs for a demo.

Rendered

Tracking Issue

epage · 2023-04-26T21:40:32Z

Not even sure if eRFC's are still a thing but when we talked about this in the cargo team meeting, we felt this format would be better for closing out conversations on all of the various details rather than trying to track and hash them out in an upfront design.

joshtriplett · 2023-04-26T21:50:29Z

text/3424-cargo-script.md

+//! ```
+
+fn main() {
+ println!("Hello, world!");


Nit, but the code here in this example should actually use the added dependency.

Huh, we might want to update https://doc.rust-lang.org/cargo/guide/dependencies.html

(another concerned raised is how stable the version numbers are)

text/3424-cargo-script.md

joshtriplett · 2023-04-26T22:02:16Z

text/3424-cargo-script.md

+//! ```cargo
+//! [dependencies]
+//! clap = { version = "4.2", features = ["derive"] }
+//! ```


Does this need to be a doc comment? Could it just be an ordinary comment? If so, I think it should be, to make it more obvious that this isn't magic syntax.

Suggested change

//! ```cargo

//! [dependencies]

//! clap = { version = "4.2", features = ["derive"] }

//! ```

// ```cargo

// [dependencies]

// clap = { version = "4.2", features = ["derive"] }

// ```

The reference-level explanation should explain that this can be a regular // comment, a /// doc comment, or a //! file-level doc comment.

The strawman examples in the eRFC are using doc-comments. I've called out how we store the embedded manifest as one of the unresolved questions that we plan to resolve before stablization. During the Pre-RFC, we explored a lot of options. I left these out of the eRFC to avoid getting side tracked by things that aren't meant to be resolved at this time.

@epage We don't have to specify all of that now; I'd just like to make sure that people don't assume this has to be a doc comment.

The current proposal is for it to be doc comments (an alternative was to instead accept regular comments). @Muscraft is prototyping the initial implementation of this proposal and has switched to syn to extract the doc comments, finding that our regex parser for the doc comments was accepting cases that aren't turned into #![doc]s.

I've clarified this within the guide section.

Wouldn't it be better to have specific syntax, or a macro, instead?

Whenever languages have comments which are parsed and have effects, it feels a bit uneasy. Things which are commented out should never have any effect, I think.

But maybe this is different, since I suppose rustc really does ignore these comments.

For more of an idea of what syntaxes we'll consider, see the Pre-RFC. We decided to do an eRFC, rather than a full RFC, and only included a rough sketch to give people an idea of what the experiment will look like, deferring actual decisions to the experiment and approval of the experiment's result.

joshtriplett · 2023-04-26T22:08:35Z

text/3424-cargo-script.md

+Now, if `regex` gets updated, we will still build with the same revision until
+we choose to `cargo update --manifest-path hello_world.rs`.


How does this work? Where is this cached? Where does the equivalent of the lockfile live?

This seems a little too opaque.

That was intentional as lockfiles are another big area for us to try to figure out the right way to handle them is. The Pre-RFC also has a section for this

I'm not suggesting the eRFC has to fully specify embedded lockfiles or any of the many other aspects discussed in the internals thread. I just think if we're going to state that this is cached and updated, we should give some idea of whether this is stored in the cargo home directory (which I'd expect), or next to the script (which I hope it isn't), or similar.

(Also, on a separate note, can we make just cargo update hello_world.rs work?)

I'm not suggesting the eRFC has to fully specify embedded lockfiles or any of the many other aspects discussed in the internals thread. I just think if we're going to state that this is cached and updated, we should give some idea of whether this is stored in the cargo home directory (which I'd expect), or next to the script (which I hope it isn't), or similar.

Done

(Also, on a separate note, can we make just cargo update hello_world.rs work?)

My understanding was that cargo intentionally avoided positional arguments (cargo add being an exception) and one reason for this is due to the heavy use of num_args(0..=1) arguments like --timings.

In the Pre-RFC, I did bring up whether we should provide a short-flag for --manifest-path because of how cumbersome it is to use for these files that aren't auto-discovered.

I'll at least call this out as another area to explore.

@epage I just read https://github.com/epage/rfcs/blob/cargo-script/text/3424-cargo-script.md, but couldn't find a place where it mentions what happens with lock files at all. I was searching for lock string after reading the whole document. Did I somehow miss it?

Great. I didn’t use cargo metadata before. It will do the trick, albeit in a format that will be unfamiliar for many people.

I looked at both cargo metadata and cargo tree to see if everything contained in a lock file is present. The one thing that I see missing is checksum field. I think it can be pretty useful.

Turns out that checksum is internal implementation checksum useful only to cargo, and is not that useful to end users.

One thing I don't see mentioned is whether there's only a single lockfile per system, or one per script. If there's a lockfile per script, what happens if a script is renamed? Does that reset all the dependency versions? What about moving a script file to a different directory without renaming it?

Phrased differently, the document says "Cargo will cache the exact information" but doesn't specify what key the cache is indexed by.

That sounds like we'll need cargo venv lol

danjl1100 · 2023-04-27T01:49:40Z

text/3424-cargo-script.md

+}
+```
+```console
+$ ./prog --config file.toml


Does this console invocation need to have the .rs extension on the filename?

Suggested change

$ ./prog --config file.toml

$ ./prog.rs --config file.toml

I was following along earlier in the pre-rfc thread on IRLO, but wasn't sure whether the .rs extension was still an unresolved question. (I see the other hello_world.rs example does have the .rs filename extension)

Does this console invocation need to have the .rs extension on the filename?

Either one is fine here. For the utility workflows, people don't always like to put extensions on their scripts (independent of the language).

Granted, that can affect what we do for cargo test --manifest-path ./prog. We might not be able to key just off of the extension.

I was following along earlier in the pre-rfc thread on IRLO, but wasn't sure whether the .rs extension was still an unresolved question. (I see the other hello_world.rs example does have the .rs filename extension)

Most of that extension discussion is about what extension would be used when it is present and not so much on the use cases for when to drop it.

Either one is fine here.

How is this supposed to work in general? If I name the file run, then cargo will be invoked as cargo run, which looks like the subcommand and not like a request to execute a file. cargo <file>.rs is at least unambiguous.

The exact precedence rules are left to be explored with the implementation. This is recorded in "Unresolved questions" as

If cargo, what precedence does it have compared to built-in commands, aliases, and external commands.

In epage/cargo-script-mvs#154, I've started looking into prior art among other build tools that support <tool> <file>.

dlang's dub is of particular interest

If the <positional> is - or ends with .rs, treat it as a single-file package

If <positional> matches an existing subcommand, run it

If <positional> exists, treat it as a single-file package

If <positional>.d exists, treat it as a single-file package

Otherwise, treat <positional> as a non-existent subcommand

For me, some tweaks I can see making

Don't support "<positional>.d exists" check (4)

For (1), check if <positional> has path separators (at least on my system, #! causes an absolute path to be passed to the command)

This is mentioned in the first unresolved question, but the idea is to figure out the precedence. For the prototype I am working on, it has the precedence as:

cmd.ends_with(".rs") || 1 < Path::new(cmd).components().count()

built-in subcommand/alias

external subcommand

Path::new(cmd).exists() && is_executable(&path)

For (1), check if <positional> has path separators (at least on my system, #! causes an absolute path to be passed to the command)

it doesn't on my system (Android).

Anything involving paths is likely to be system specific. Or at least they'll be edge case systems.

Either one is fine here. For the utility workflows, people don't always like to put extensions on their scripts (independent of the language).

People can just use a symlink though. In the case of the PHP PHAR format, PHP denies loading phar files that do not have a .phar extension for some reason, so some installations of composer are performed by ln -s composer.phar /usr/bin/composer (citation needed: I know several people who do so, but I am not sure if this is a common practice).

If I name the file run, then cargo will be invoked as cargo run, which looks like the subcommand and not like a request to execute a file.

cargo -- run?

Does this work?

#!/usr/bin/env cargo -- --

veber-alex · 2023-04-27T16:28:11Z

I think it's important to have cargo add support for .rs files, otherwise adding dependencies is not very ergonomic (having to remember the syntax and lookup the version numbers)

Diggsey · 2023-05-04T23:55:39Z

Of note is that rustc already ignores a shebang at the beginning of a file despite it not being valid Rust code - I hadn't realised that.

epage · 2023-05-17T14:54:13Z

As this is just agreement to do an experiment, interest seems high, and conversation has died down, I feel like its time:

@rfcbot merge

rfcbot · 2023-05-17T14:54:15Z

Team member @epage has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

rfcbot · 2023-05-31T03:49:52Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

ehuss · 2023-05-31T03:58:12Z

Huzzah! The @rust-lang/cargo team has decided to accept this RFC.

To track further discussion, subscribe to the tracking issue here:
rust-lang/cargo#12207

xnuk · 2023-06-08T16:47:29Z

text/3424-cargo-script.md

+Haskell
+- [`runghc` / `runhaskell`](https://downloads.haskell.org/ghc/latest/docs/users_guide/runghc.html)
+ - Users can use the file stem (ie leave off the extension) when passing it in
+- [cabal's single-file haskel script](https://cabal.readthedocs.io/en/stable/getting-started.html#run-a-single-file-haskell-script)


typo here

-haskel +haskell

epage · 2023-06-27T15:01:45Z

Background: we are looking to do a mid-point evaluation of this experiment to see how its going and what might need to be changed about the direction

Testing instructions

Requires

nightly: nightly-2023-06-25 or newer
~~beta~~
~~stable~~

What is supported

cargo being in the shebang line for both .rs and Cargo.toml files
Using cargo <cmd> --manifest-path <file.rs> for basic commands (check, test, etc)

What is not supported

Some commands like publish and add
Workspace support
foo.rs being libs

For more see unstable reference documentation

In particular, we are looking to understand:

How easy is it to write scripts? Where are the pain points?
How easy is it to share scripts? Where are the pain points?
Is the default cargo output when running scripts intrusive?
Does respecting the local rustup toolchain file cause problems?
How is the perceived first build performance? Performance of subsequent runs?
Anything that seemed odd or unexpected with the behavior?
If you need to run the scripts in release mode and the experience for doing so?

In addition, areas still being explored:

How to embed manifests in source
How to balance low overhead with not breaking scripts on edition updates
Do we need [lib] support? Are a script being [lib] and [[bin]]` mutually exclusive or do we need to support them together?

Please leave feedback on rust-lang/cargo#12207

Examples:

Simplest example is:

#!/usr/bin/env -S cargo +nightly -Zscript

fn main() {}

With dependencies (nightly-2023-10-01 to today):

#!/usr/bin/env -S cargo +nightly -Zscript
```cargo
[dependencies]
clap = { version = "4.2", features = ["derive"] }
```

use clap::Parser;

#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
    #[clap(short, long, help = "Path to config")]
    config: Option<std::path::PathBuf>,
}

fn main() {
    let args = Args::parse();
    println!("{:?}", args);
}

With dependencies (nightly-2023-06-25 to nightly-2023-01-01):

#!/usr/bin/env -S cargo +nightly -Zscript

//! ```cargo
//! [dependencies]
//! clap = { version = "4.2", features = ["derive"] }
//! ```

use clap::Parser;

#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
    #[clap(short, long, help = "Path to config")]
    config: Option<std::path::PathBuf>,
}

fn main() {
    let args = Args::parse();
    println!("{:?}", args);
}

Note: not all env implementations support -S. One workaround is to create a wrapper script. I call mine nargo (nightly + cargo). This would also better simulate the stable experience.

victor-timofei · 2023-06-28T12:13:54Z

Just migrated from rust-script and it works like a charm. https://git.vtimothy.com/vtimofei/dotfiles/commit/5eaf73a001f86cda73e5332937a6c6187a0f47a5

victor-timofei · 2023-06-28T12:29:31Z

I tried to use a library with the below structure:

#!/usr/bin/env -S cargo +nightly -Zscript

//! ```cargo
//! [dependencies]
//! hello_lib = { path = "./hello_lib" }
//! ```

use hello_lib::greet;

fn main() {

    println!("{}", greet());
}

$ tree
.
├── hello_lib
│   ├── Cargo.lock
│   ├── Cargo.toml
│   ├── src
│   │   └── lib.rs
└── script.rs

$ cat hello_lib/src/lib.rs
pub fn greet() -> String {
    "Howdy".into()
}

I think using libraries this way is a viable option, and support for [lib] wouldn't be required.

U007D · 2023-06-28T12:39:17Z

This RFC will appear in the Call for Testing section of the next issue (# 501) of This Week in Rust (TWiR).

I have removed the call-for-testing label. Please feel free to re-add the label if you would like this RFC to appear again in another issue of TWiR.

epage · 2023-06-28T13:48:31Z

@victor-timofei

I think using libraries this way is a viable option, and support for [lib] wouldn't be required.

The idea for single-file lib packages is to reduce the boilerplate for writing [lib] packages, especially in workspaces.

victor-timofei · 2023-06-28T13:54:28Z

@epage thanks for clearing this out. That would be indeed handy, but I see it mostly as a nice to have.

BTW, where would be a good place to give feedback for this?

epage · 2023-06-28T13:59:58Z

@victor-timofei

BTW, where would be a good place to give feedback for this?

Most activity should be on rust-lang/cargo#12207. I posted the testing instructions here because this is a repo TWiR monitors.

see rust-lang/rfcs#3424 (comment)

epage added 11 commits April 26, 2023 14:33

feat: Start cargo-script eRFC

a165195

feat: Initial draft of cargo-script eRFC

423a195

fix: Small tweaks

c66632d

fix: Rename file for RFC number

3cf8f43

fix: Enter RFC number

8422cb3

fix: Spelling of shebang

4b56dd4

fix: Remove stale referenve to cargo-eval

14009d1

fix: Improve guide's dependency example

48aa53f

fix: Specify the lock location

9f8a951

fix: Call out --manifest-path verbosity

db76c87

fix: Explicitly declare its a doc comment

401ceb0

epage added the T-cargo Relevant to the Cargo team, which will review and decide on the RFC. label Apr 26, 2023

fix: Be even more explicit on doc comments

d86f217

joshtriplett reviewed Apr 26, 2023

View reviewed changes

text/3424-cargo-script.md Outdated Show resolved Hide resolved

joshtriplett reviewed Apr 26, 2023

View reviewed changes

text/3424-cargo-script.md Outdated Show resolved Hide resolved

joshtriplett reviewed Apr 26, 2023

View reviewed changes

danjl1100 reviewed Apr 27, 2023

View reviewed changes

epage mentioned this pull request May 3, 2023

A Cargo.toml file in parent directory which you cannot delete prevents cargo from building anything rust-lang/cargo#12080

Closed

Veykril mentioned this pull request May 4, 2023

Proper support for standalone/detached files rust-lang/rust-analyzer#14318

Open

epage mentioned this pull request May 17, 2023

cargo clean ~/.cargo rust-lang/cargo#3289

Closed

epage mentioned this pull request May 22, 2023

Content-based filetype detection crate-ci/typos#726

Open

rfcbot added finished-final-comment-period The final comment period is finished for this RFC. to-announce and removed final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. labels May 31, 2023

ehuss mentioned this pull request May 31, 2023

Tracking Issue for cargo-script RFC 3424 rust-lang/cargo#12207

Open

33 tasks

Update tracking issue

755ddb6

ehuss merged commit 832a070 into rust-lang:master May 31, 2023

epage deleted the cargo-script branch May 31, 2023 13:38

xnuk reviewed Jun 8, 2023

View reviewed changes

weihanglo mentioned this pull request Jun 10, 2023

Add cargo watch rust-lang/cargo#9339

Open

MiSawa mentioned this pull request Jun 12, 2023

IDE support fornwall/rust-script#11

Open

epage added the call-for-testing Add this label + test notes in comments to be included in TWiR `Call for Testing` section. label Jun 27, 2023

U007D removed the call-for-testing Add this label + test notes in comments to be included in TWiR `Call for Testing` section. label Jun 28, 2023

epage mentioned this pull request Jul 1, 2023

RFC: Nested Cargo packages #3452

Open

victor-timofei mentioned this pull request Jul 20, 2023

Add shebang support for "cargo script" rust-lang/rust-analyzer#15318

Open

ModProg mentioned this pull request Sep 11, 2023

Examples cannot have separate sets of dependencies rust-lang/cargo#523

Closed

emilk mentioned this pull request Sep 18, 2023

Replace Bash ~and Python~ scripts with Rust rerun-io/rerun#3349

Closed

epage mentioned this pull request Sep 26, 2023

RFC: cargo-script #3502

Merged

jayvdb mentioned this pull request Jan 1, 2024

cargo supports scripts directly py2many/py2many#563

Closed

indietyp added a commit to hashintel/hash that referenced this pull request Apr 21, 2024

feat: use cargo script

a4aee71

see rust-lang/rfcs#3424 (comment)

alsuren mentioned this pull request Sep 4, 2024

402 Payment Required cargo-bins/cargo-quickinstall#268

Open

		Now, if `regex` gets updated, we will still build with the same revision until
		we choose to `cargo update --manifest-path hello_world.rs`.

eRFC: single-file packages ("cargo script") integration #3424

eRFC: single-file packages ("cargo script") integration #3424

Conversation

epage commented Apr 26, 2023 • edited Loading

epage commented Apr 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yerke Apr 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SOF3 May 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

veber-alex commented Apr 27, 2023

Diggsey commented May 4, 2023

epage commented May 17, 2023

rfcbot commented May 17, 2023 • edited by joshtriplett Loading

rfcbot commented May 31, 2023

ehuss commented May 31, 2023

Choose a reason for hiding this comment

epage commented Jun 27, 2023 • edited Loading

Testing instructions

Examples:

victor-timofei commented Jun 28, 2023

victor-timofei commented Jun 28, 2023

U007D commented Jun 28, 2023 • edited Loading

epage commented Jun 28, 2023

victor-timofei commented Jun 28, 2023

epage commented Jun 28, 2023

epage commented Apr 26, 2023 •

edited

Loading

yerke Apr 27, 2023 •

edited

Loading

SOF3 May 5, 2023 •

edited

Loading

rfcbot commented May 17, 2023 •

edited by joshtriplett

Loading

epage commented Jun 27, 2023 •

edited

Loading

U007D commented Jun 28, 2023 •

edited

Loading