Instructions for adding a new query

[pierwill]

Summary of issue

We have a good start on documenting adding a new query: https://rustc-dev-guide.rust-lang.org/query.html#adding-a-new-kind-of-query. However, I think there's room for greater clarity.

☞ After reading this section, I know where to define a query. I want to know where and how to implement one.

Details

Currently, this section of the guide has two summaries of the process:

So suppose you want to add a new kind of query, how do you do so? Well, defining a query takes place in two steps:

1. first, you have to specify the query name and arguments; and then,
2. you have to supply query providers where needed.

and

So, to add a query:

1. Add an entry to rustc_queries! using the format above.
2. Link the provider by modifying the appropriate provide method; or add a new one if needed and ensure that rustc_driver is invoking it.

Where are these providers?
How do I find "the appropriate provide method" for a new query? Do I always need to create a new one for a new query? Or can I sometimes reuse ("modify") an existing one?

The section immediately before this one gives information on Providers, but does not say how to add a new one.

[pierwill]

Someone from @rust-lang/wg-incr-comp might be able to help here.

[pierwill]

See also rust-lang/rust#94097.

[jyn514]

The provide function looks like this: https://github.com/rust-lang/rust/blob/c7b6ebdf7c0acf2382696988c09902beb91f4082/compiler/rustc_ty_utils/src/layout.rs#L26-L28
and is called from rustc_interface: https://github.com/rust-lang/rust/blob/a474ec50b7010d289d103242a63c806ed6f01c2e/compiler/rustc_interface/src/passes.rs#L726-L756

The name provide isn't special but is used in all crates for consistency.

How do I find "the appropriate provide method" for a new query?

There is only one per crate. Use the one for the crate where you define the query provider.

Do I always need to create a new one for a new query? Or can I sometimes reuse ("modify") an existing one?

You should reuse the provide function for the current crate if it exists.

---

A question you didn't ask, but I think we should answer, is when to use a query instead of a normal function. Some examples:

• When your query needs to be used in a crate earlier in the dependency graph than it can be defined (e.g. you want to use typeck results in rustc_symbol_mangling). Where possible, and if it doesn't impact compile times too much, it might make sense to refactor the code so that the function can be defined earlier instead of using a query.
• When you think caching will be important for performance, and you want to cache the results on disk

Note that all queries must be idempotent and cannot change (and preferably, not even access) global state. Failure to do so will cause ICEs at best and silently miscompile code at worst. Be extremely cautious about using any function named _untracked; the only time you should use those functions is when you don't have access to a TyCtxt.

@michaelwoerister @cjgillot does that sound roughly right?

[jyn514]

cc rust-lang/rust#104011

[jyn514]

@Nilstrieb mentions "projection queries" are also a good reason for using a query instead of a function, for better caching. https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation-in-detail.html#the-projection-query-pattern

[Noratrieb]

Be extremely cautious about using any function named _untracked from within a query provider.

Most rustc code runs inside some form of query provider (for example the entirety of the borrow checker in the mir_borrowck query), so really you need to always be careful unless you are explicitly outside the realms of the query system and it should be phrased like that.

[michaelwoerister]

@jyn514, sounds right to me.

How do I find "the appropriate provide method" for a new query?

There is only one per crate. Use the one for the crate where you define the query provider.

I think there are some instances where there's more than one provide() method in a crate. But there is no deeper meaning to that, afaik. One per crate seems fine.