Implement @argsfile to read arguments from command line

[jsgf]

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line.

Implemented in #63175

This implements a straightforward form where each argument is on a separate line in a text file. This allows for everything except for arguments with internal line breaks.

Description (also in docs)

If you specify @path on the command-line, then it will open path and read
command line options from it. These options are one per line; a blank line indicates
an empty option. The file can use Unix or Windows style line endings, and must be
encoded as UTF-8.

[hellow554]

You probably mean #63175 because your link PR is just the rollup.
Can you please clarify that this is a tracking issue? You can take #63569 as an example (or anything else with the C-tracking-issue label)

[jsgf]

Does this need anything else to be a complete tracking issue? This is a relatively minor feature, so there's no RFC or anything to reference.

[Mark-Simulacrum]

@jsgf I seem to recall you wanting to use this feature within FB or so, have you had a chance to do so? Do we have any usage reports; not technically necessary but good for stabilization, I think.

[jsgf]

@Mark-Simulacrum Yes, I'm using this regularly and haven't had any problems with it. I'd like to stabilize it.

[Mark-Simulacrum]

@rfcbot fcp merge

I propose that we stabilize this feature. It adds support for arg files via @file on the command line, similar to other tools. We have seen both @cramertj and @jsgf state that they want this, and @jsgf confirms that this has worked well in their use.

We differ from ldd and other tools in that:

• We do not support recursive use of @file (i.e., we expand "once")
• We separate by newline, rather than by whitespace. This is easier to parse and avoids quoting (we do not support escaping newlines, but it is super rare to want a newline anyway).

These changes are due to wanting to simplify the option and hopefully modernize it some -- we felt that including support would complicate the code and be unnecessary. You can find some ~extensive discussion on the implementation PR (see #63175).

(I am uncertain whether this merge command will work, due to possibly lacking permissions, but if so I'll bug someone to rerun it).

[nikomatsakis]

@rfcbot fcp merge

Quoting @Mark-Simulacrum:

I propose that we stabilize this feature. It adds support for arg files via @file on the command line, similar to other tools. We have seen both @cramertj and @jsgf state that they want this, and @jsgf confirms that this has worked well in their use.

We differ from ldd and other tools in that:

• We do not support recursive use of @file (i.e., we expand "once")
• We separate by newline, rather than by whitespace. This is easier to parse and avoids quoting (we do not support escaping newlines, but it is super rare to want a newline anyway).

These changes are due to wanting to simplify the option and hopefully modernize it some -- we felt that including support would complicate the code and be unnecessary. You can find some ~extensive discussion on the implementation PR (see #63175).

(I am uncertain whether this merge command will work, due to possibly lacking permissions, but if so I'll bug someone to rerun it).

[rfcbot]

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Zoxc
• @cramertj
• @eddyb
• @estebank
• @michaelwoerister
• @nagisa
• @nikomatsakis
• @oli-obk
• @petrochenkov
• @pnkfelix
• @varkor

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[oli-obk]

We do not support recursive use of @file (i.e., we expand "once")

Are we forward compatible with adding recursive use? It seems to me like adding that later would be a breaking change. Could we error right now on recursive uses?

[Mark-Simulacrum]

We do not support recursive use of @file (i.e., we expand "once")

Are we forward compatible with adding recursive use? It seems to me like adding that later would be a breaking change. Could we error right now on recursive uses?

It would be no more a breaking change than this -- i.e., we would mostly error today on such a case since @file likely doesn't exist. This patch already "breaks" code that uses @file, but the fix is trivial, use ./@file, so this is deemed acceptable (starting filenames with @ is expected to be super rare).

However, we can likely add a check fairly easily that the recursive usage would definitely not be a breaking change if you think that's warranted.

[oli-obk]

However, we can likely add a check fairly easily that the recursive usage would definitely not be a breaking change if you think that's warranted.

I think it would be good to be conservative if the check is simple to create

[jsgf]

I originally handled recursion and it was rejected as being two complicated. The questions it raises are:

1. How do you quote a filename starting with @ if you don't want to include it as args? Answer: use ./@foo, like on the real command line
2. What's a reasonable upper recursion bound? I originally chose 8.
3. How does pathname resolution work? Is it relative to the cwd or to the file in which the @ directive was found?

The third point is the most complex. ld, etc, always resolve them relative to the current working directory, however most other recursive include-like mechanisms resolve relative to the including file. I can think of circumstances in which either is going to be the preferred behaviour, but I definitely wouldn't want to have any syntax or other mechanism to switch it.

I think the conclusion is that recursive use isn't important in practice, and even though questions 1 & 2 have reasonable answers, 3 is complicated enough to cast doubt on the whole thing. Besides, these files are typically machine generated, so when generating one you can flatten them - if you want recursion, read the file and expand it in-place.

Edit: I remember there was an irritating 4: What is the file encoding? It turns out its incredibly awkward to read an arbitrary text file and use text from it as a pathname, esp on Windows. It basically requires insisting that the file is UTF-8, but pathnames are not necessarily UTF-8 or even encodable with it.

[oli-obk]

Ok, seems reasonable to me to never include recursion, thanks for the detailed explanation

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[jsgf]

Now stable.