Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

forc-pkg: Refactor BuildPlan construction and validation to minimize fetching, improve offline support, reduce I/O, fix bugs. #2234

Merged
merged 40 commits into from
Jul 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
f985bf0
BuildPlan generation from lock file does not break when there is a de…
kayagokalp Jun 14, 2022
b4219a7
While removing the deleted nodes handle orphaned childs
kayagokalp Jun 16, 2022
9495057
removed debug println
kayagokalp Jun 16, 2022
e97c684
removed_deps removed from BuildPlan
kayagokalp Jun 17, 2022
38af42e
While checking from parent's manifest check for package name as well
kayagokalp Jun 17, 2022
e469399
removed unnecessary reference
kayagokalp Jun 17, 2022
4be5880
split validate into generate_diff and validate
kayagokalp Jun 20, 2022
aa90853
apply_pkg_diff visits nodes to delete in reverse BFS order
kayagokalp Jun 20, 2022
316a871
docs added for generate_diff
kayagokalp Jun 20, 2022
4548afb
Merge branch 'master' into kayagokalp/1778
eureka-cpu Jun 20, 2022
9bda581
merge master
kayagokalp Jun 23, 2022
38cc164
removed unnecessary logic in apply_diff_after_lock
kayagokalp Jun 23, 2022
344ceaa
doc updates
kayagokalp Jun 23, 2022
cb255e2
clippy && extra dereferencing removed
kayagokalp Jun 23, 2022
cb6a2e0
Apply suggestions from code review effor -> effort
kayagokalp Jun 23, 2022
bb40017
duplicate code removed
kayagokalp Jun 27, 2022
1905046
change visibility of apply_pkg_diff and from_lock to private
kayagokalp Jun 27, 2022
5d466ec
doc updated
kayagokalp Jun 27, 2022
6de10da
Merge master
kayagokalp Jun 27, 2022
4ac21d4
patch checks are added
kayagokalp Jun 27, 2022
057afeb
check for patches corrected
kayagokalp Jun 27, 2022
f11442b
lsp uses load_from_manifest, from_lock_file is private
kayagokalp Jun 29, 2022
476efa7
Merge branch 'master' into kayagokalp/1778
kayagokalp Jun 29, 2022
2bc0d94
add use::collections::HashSet
kayagokalp Jun 29, 2022
b590f03
Merge branch 'master' into kayagokalp/1778
mitchmindtree Jul 4, 2022
d3a10ae
WIP Improve `BuildPlan` graph reconstruction
mitchmindtree Jul 4, 2022
fd91d28
Minimise I/O by replacing BuildProfile's PathMap with a ManifestMap
mitchmindtree Jul 6, 2022
be7b503
Merge branch 'master' into mitchmindtree/build-plan-refactor
mitchmindtree Jul 6, 2022
a216669
Ensure only one edge between a package and each dependency
mitchmindtree Jul 6, 2022
c8a43f3
Ensure lock is updated if path child dependencies change
mitchmindtree Jul 6, 2022
a11696c
Address clippy nits
mitchmindtree Jul 6, 2022
41a3e22
Address formatting nits
mitchmindtree Jul 6, 2022
f8ece94
Improve accuracy of dependency removal by invalidating by edges
mitchmindtree Jul 6, 2022
d161d20
Avoid visiting a node's dependencies multiple times during validation
mitchmindtree Jul 6, 2022
02ceb92
Improve name and docs for BuildPlan constructors
mitchmindtree Jul 6, 2022
647f0ea
Remove unnecessary path root validation from Lock::to_graph.
mitchmindtree Jul 6, 2022
921f4ea
Merge branch 'master' into mitchmindtree/build-plan-refactor
mitchmindtree Jul 6, 2022
28eec55
Fix typo in find_proj_node comment
mitchmindtree Jul 6, 2022
5dd2c43
Merge branch 'master' into mitchmindtree/build-plan-refactor
mitchmindtree Jul 6, 2022
e16ceba
Merge branch 'master' into mitchmindtree/build-plan-refactor
eureka-cpu Jul 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 1 addition & 8 deletions forc-pkg/src/lock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,7 @@ impl Lock {
.cloned()
.ok_or_else(|| anyhow!("found dep {} without node entry in graph", dep_key))?;
let dep_name = dep_name.unwrap_or(&graph[dep_node].name).to_string();
graph.add_edge(node, dep_node, dep_name);
}
}

// Validate the `path_root` of each of the path nodes.
for n in graph.node_indices() {
if let pkg::SourcePinned::Path(ref src) = graph[n].source {
pkg::validate_path_root(&graph, n, src.path_root)?;
graph.update_edge(node, dep_node, dep_name);
}
}

Expand Down
49 changes: 42 additions & 7 deletions forc-pkg/src/manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use sway_utils::constants;
type PatchMap = BTreeMap<String, Dependency>;

/// A [Manifest] that was deserialized from a file at a particular path.
#[derive(Debug)]
#[derive(Clone, Debug)]
pub struct ManifestFile {
/// The deserialized `Forc.toml`.
manifest: Manifest,
Expand All @@ -23,7 +23,7 @@ pub struct ManifestFile {
}

/// A direct mapping to a `Forc.toml`.
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub struct Manifest {
pub project: Project,
Expand All @@ -33,7 +33,7 @@ pub struct Manifest {
build_profile: Option<BTreeMap<String, BuildProfile>>,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub struct Project {
pub authors: Option<Vec<String>>,
Expand All @@ -45,14 +45,14 @@ pub struct Project {
pub implicit_std: Option<bool>,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub struct Network {
#[serde(default = "default_url")]
pub url: String,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(untagged)]
pub enum Dependency {
/// In the simple format, only a version is specified, eg.
Expand All @@ -64,7 +64,7 @@ pub enum Dependency {
Detailed(DependencyDetails),
}

#[derive(Serialize, Deserialize, Debug, Default)]
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
#[serde(rename_all = "kebab-case")]
pub struct DependencyDetails {
pub(crate) version: Option<String>,
Expand All @@ -77,7 +77,7 @@ pub struct DependencyDetails {
}

/// Parameters to pass through to the `sway_core::BuildConfig` during compilation.
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "kebab-case")]
pub struct BuildProfile {
pub print_ir: bool,
Expand Down Expand Up @@ -107,6 +107,7 @@ impl ManifestFile {
/// implicitly. In this case, the `sway_git_tag` is used to specify the pinned commit at which
/// we fetch `std`.
pub fn from_file(path: PathBuf, sway_git_tag: &str) -> Result<Self> {
let path = path.canonicalize()?;
let manifest = Manifest::from_file(&path, sway_git_tag)?;
Ok(Self { manifest, path })
}
Expand Down Expand Up @@ -144,11 +145,15 @@ impl ManifestFile {
}

/// The path to the `Forc.toml` from which this manifest was loaded.
///
/// This will always be a canonical path.
pub fn path(&self) -> &Path {
&self.path
}

/// The path to the directory containing the `Forc.toml` from which this manfiest was loaded.
///
/// This will always be a canonical path.
pub fn dir(&self) -> &Path {
self.path()
.parent()
Expand All @@ -157,6 +162,8 @@ impl ManifestFile {

/// Given the directory in which the file associated with this `Manifest` resides, produce the
/// path to the entry file as specified in the manifest.
///
/// This will always be a canonical path.
pub fn entry_path(&self) -> PathBuf {
self.dir()
.join(constants::SRC_DIR)
Expand Down Expand Up @@ -200,6 +207,19 @@ impl ManifestFile {
.as_ref()
.and_then(|profiles| profiles.get(profile_name))
}

/// Given the name of a `path` dependency, returns the full canonical `Path` to the dependency.
pub fn dep_path(&self, dep_name: &str) -> Option<PathBuf> {
let dir = self.dir();
let details = self.dep_detailed(dep_name)?;
details.path.as_ref().and_then(|path_str| {
let path = Path::new(path_str);
match path.is_absolute() {
true => Some(path.to_owned()),
false => dir.join(path).canonicalize().ok(),
}
})
}
}

impl Manifest {
Expand Down Expand Up @@ -333,6 +353,21 @@ impl Manifest {
.and_then(|deps| deps.get(dep_name))
}

/// Retrieve a reference to the dependency with the given name.
pub fn dep_detailed(&self, dep_name: &str) -> Option<&DependencyDetails> {
self.dep(dep_name).and_then(|dep| match dep {
Dependency::Simple(_) => None,
Dependency::Detailed(detailed) => Some(detailed),
})
}

/// Retrieve the listed patches for the given name.
pub fn patch(&self, patch_name: &str) -> Option<&PatchMap> {
self.patch
.as_ref()
.and_then(|patches| patches.get(patch_name))
}

/// Finds and returns the name of the dependency associated with a package of the specified
/// name if there is one.
///
Expand Down
Loading