Tracking Issue for workspace feature-unification

[ehuss]

Summary

RFC: #3692
Implementation: TODO
Documentation: TODO

Adds the resolver.feature-unification configuration option to control how features are unified across a workspace.

Unresolved Issues

• Bikeshed the config option naming.

Future Extensions

• Support in manifests
• Dependency version unification
• Unify features in other settings

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

[weiznich]

Whoever marked my comment as duplicate: Could you point to the "duplicated" comment in this thread?

Also could you point to an official answer to these suggestions from the cargo team and not from individual team members? I'm really not sure what you are expecting to happen if you just keep pretending to ignore meaningful suggestions.

[epage]

As you said, this was repeating content from the RFC discussion. I gave some replies in the RFC. The Cargo team saw the discussion and signed off on the RFC without raising concerns. This was re-iterated by Eric in the thread you opened on Zulip, see https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/Offical.20team.20response.20to.20concerns.20raised.20in.20RFC-3692/near/479039846

As this has come up before and is re-litigating the RFC, I collapsed it to get it out of the way of the discussion for implementing the RFC. It is a big help to be able to have Issue discussions more focused using tools like that. Some other teams go to the extreme of locking Tracking Issues, reserving them solely for status updates.

[weiznich]

@epage I still do not see where exactly my concerns about the potential social impact are addressed. Can you point to the exact location, including an explanation why it's not possible to have this slight adjustment to the wording to make clear that the communication around this feature needs to be aware of these edge cases. Given how other parts of the rust project usually try to have the best possible diagnostic for error cases I really cannot understand why you and the cargo team continues to just ignore these suggestions.

All I got so far is reiterating why you don't consider this a breaking change. I got your point there the first time, I just disagree with it. Even that does not mean that I'm the opinion that you shouldn't do this feature, which is why I literally trying to suggest for more than a month now that this likely can be resolved by just having that diagnostic/documentation note explicit in there.

[epage]

As for implementing this, I think its ok to implement each config value (and maybe stabilize) separately.

For all of this, the top-level function for where all of this lives is

cargo/src/cargo/ops/resolve.rs

Lines 137 to 269 in e858736

	pub fn resolve_ws_with_opts<'gctx>(
	ws: &Workspace<'gctx>,
	target_data: &mut RustcTargetData<'gctx>,
	requested_targets: &[CompileKind],
	cli_features: &CliFeatures,
	specs: &[PackageIdSpec],
	has_dev_units: HasDevUnits,
	force_all_targets: ForceAllTargets,
	dry_run: bool,
	) -> CargoResult<WorkspaceResolve<'gctx>> {
	let mut registry = ws.package_registry()?;
	let (resolve, resolved_with_overrides) = if ws.ignore_lock() {
	let add_patches = true;
	let resolve = None;
	let resolved_with_overrides = resolve_with_previous(
	&mut registry,
	ws,
	cli_features,
	has_dev_units,
	resolve.as_ref(),
	None,
	specs,
	add_patches,
	)?;
	ops::print_lockfile_changes(ws, None, &resolved_with_overrides, &mut registry)?;
	(resolve, resolved_with_overrides)
	} else if ws.require_optional_deps() {
	// First, resolve the root_package's *listed* dependencies, as well as
	// downloading and updating all remotes and such.
	let resolve = resolve_with_registry(ws, &mut registry, dry_run)?;
	// No need to add patches again, `resolve_with_registry` has done it.
	let add_patches = false;
	// Second, resolve with precisely what we're doing. Filter out
	// transitive dependencies if necessary, specify features, handle
	// overrides, etc.
	add_overrides(&mut registry, ws)?;
	for (replace_spec, dep) in ws.root_replace() {
	if !resolve
	.iter()
	.any(|r| replace_spec.matches(r) && !dep.matches_id(r))
	{
	ws.gctx()
	.shell()
	.warn(format!("package replacement is not used: {}", replace_spec))?
	}
	if dep.features().len() != 0 || !dep.uses_default_features() {
	ws.gctx()
	.shell()
	.warn(format!(
	"replacement for `{}` uses the features mechanism. \
	default-features and features will not take effect because the replacement dependency does not support this mechanism",
	dep.package_name()
	))?
	}
	}
	let resolved_with_overrides = resolve_with_previous(
	&mut registry,
	ws,
	cli_features,
	has_dev_units,
	Some(&resolve),
	None,
	specs,
	add_patches,
	)?;
	(Some(resolve), resolved_with_overrides)
	} else {
	let add_patches = true;
	let resolve = ops::load_pkg_lockfile(ws)?;
	let resolved_with_overrides = resolve_with_previous(
	&mut registry,
	ws,
	cli_features,
	has_dev_units,
	resolve.as_ref(),
	None,
	specs,
	add_patches,
	)?;
	// Skipping `print_lockfile_changes` as there are cases where this prints irrelevant
	// information
	(resolve, resolved_with_overrides)
	};
	let pkg_set = get_resolved_packages(&resolved_with_overrides, registry)?;
	let member_ids = ws
	.members_with_features(specs, cli_features)?
	.into_iter()
	.map(|(p, _fts)| p.package_id())
	.collect::<Vec<_>>();
	pkg_set.download_accessible(
	&resolved_with_overrides,
	&member_ids,
	has_dev_units,
	requested_targets,
	target_data,
	force_all_targets,
	)?;
	let feature_opts = FeatureOpts::new(ws, has_dev_units, force_all_targets)?;
	let resolved_features = FeatureResolver::resolve(
	ws,
	target_data,
	&resolved_with_overrides,
	&pkg_set,
	cli_features,
	specs,
	requested_targets,
	feature_opts,
	)?;
	pkg_set.warn_no_lib_packages_and_artifact_libs_overlapping_deps(
	ws,
	&resolved_with_overrides,
	&member_ids,
	has_dev_units,
	requested_targets,
	target_data,
	force_all_targets,
	)?;
	Ok(WorkspaceResolve {
	pkg_set,
	workspace_resolve: resolve,
	targeted_resolve: resolved_with_overrides,
	resolved_features,
	})
	}

Keep in mind that we have two resolvers (dependency, feature) and multiple phases

1. Resolve dependencies for the lockfile (resolve_with_registry)
2. Resolve dependencies to strip out unrelated packages, unspecified features (resolve_with_previous)
3. Resolve features (FeatureResolver::resolve)

To make workspace work, we'll likely need to duplicate the logic from (2) to be able to run it after feature resolver so that we can resolve features for unselected packages. The problem with outright moving the logic from the dependency resolver to the feature resolver is that, under various circumstances, we allow Cargo to combine (1) and (2) so that dependencies don't need to be vendored or that we can skip downloading git dependencies for the sake of deciding not to use them. This is what the other two cases at the beginning of resolve_ws_with_opts are about.

The big risk is that we do the filtering differently between the the dependency resolver and the feature resolver.

As for package, a hacky way to implement that is to do back-to-back compilations. The downside is the redundant work. Ideally, FeatureResolver::resolve could just handle this. The main question I have is how well will it scale to have these additional disjoint feature graphs. There might need to be work done to handle that.

[weiznich]

@epage I'm still waiting on your response to my question above.

I'm also highly disappointed that the cargo team just does not seem to care about good diagnostics at all.