Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore!: Make file manager read-only to the compiler #3760

Merged
merged 36 commits into from
Dec 14, 2023
Merged
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7e0a3d1
make `find_module` be an immutable reference
kevaundray Dec 10, 2023
b2ff05b
add method in nargo that pre-populates the FileManager
kevaundray Dec 10, 2023
5592348
change add_file to `name_to_id` -- we assume that the file manager ha…
kevaundray Dec 10, 2023
d5fd4e0
cargo
kevaundray Dec 10, 2023
2f137b7
Update tooling/nargo/src/lib.rs
kevaundray Dec 10, 2023
41ae81f
add a method in file manager that allows us to add a file with its so…
kevaundray Dec 10, 2023
c7f576c
add deprecation TODO to file_reader
kevaundray Dec 10, 2023
a2460e6
add stdlib file in noirc_driver that returns the stdlib paths alongsi…
kevaundray Dec 10, 2023
270a0f4
add the contents of the stdlib whenever we call prepare_crate
kevaundray Dec 10, 2023
52d33a5
cargo
kevaundray Dec 10, 2023
a38c27c
cargo fmt
kevaundray Dec 10, 2023
6a5bf33
move tempfile to dev dependencies
kevaundray Dec 10, 2023
f72ce99
add files into file manager in test since find_module does not add fi…
kevaundray Dec 10, 2023
b30ad54
insert all files for this packages dependencies into the file manager…
kevaundray Dec 10, 2023
15a94ee
cargo fmt
kevaundray Dec 10, 2023
4599361
remove un-needed fully qualified path
kevaundray Dec 10, 2023
f20109e
Add note on stdlib
kevaundray Dec 10, 2023
4413350
cargo fmt
kevaundray Dec 10, 2023
d6a24e8
add comment to process_dep_graph and fix setup_test_context
kevaundray Dec 11, 2023
552a4dd
replace expect with unwrap_or_else: expect doesn't allow you to add p…
kevaundray Dec 11, 2023
3f8551b
try: add a `PathToFileSourceMap` object
kevaundray Dec 11, 2023
dd94b46
remove extraneous forward slash
kevaundray Dec 11, 2023
3495de5
add file_manager_with_source_map method so we ensure that FileManager…
kevaundray Dec 11, 2023
851fab1
add expect
kevaundray Dec 11, 2023
3cd35fd
fix node tests
kevaundray Dec 11, 2023
d2590d5
fix browser tests and naming nit
kevaundray Dec 11, 2023
5b867d5
chore!: Remove `add_file` and `file_reader` from FileManager (#3762)
kevaundray Dec 11, 2023
5ebf83a
Update compiler/fm/src/lib.rs
kevaundray Dec 11, 2023
7a787d7
Merge branch 'master' into kw/make-fm-read-only
kevaundray Dec 11, 2023
419273e
file_reader is no longer being used
kevaundray Dec 11, 2023
d120852
Merge branch 'master' into kw/make-fm-read-only
kevaundray Dec 11, 2023
1e9ac97
chore: add simple doc comments for `add_file_with_source` methods
TomAFrench Dec 13, 2023
cc92c07
Merge remote-tracking branch 'origin/master' into kw/make-fm-read-only
kevaundray Dec 13, 2023
9aa0bb7
Update tooling/nargo/src/lib.rs
kevaundray Dec 13, 2023
3e26e3c
read entry_path and get all files in its parent
kevaundray Dec 13, 2023
0b53332
clippy
kevaundray Dec 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 83 additions & 8 deletions compiler/wasm/src/compile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,31 @@ struct DependencyGraph {
library_dependencies: HashMap<CrateName, Vec<CrateName>>,
}

#[wasm_bindgen]
// This is a map containing the paths of all of the files in the entry-point crate and
// the transitive dependencies of the entry-point crate.
//
// This is for all intents and purposes the file system that the compiler will use to resolve/compile
// files in the crate being compiled and its dependencies.
#[derive(Deserialize, Default)]
pub struct PathToFileSourceMap(HashMap<std::path::PathBuf, String>);
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
kevaundray marked this conversation as resolved.
Show resolved Hide resolved

#[wasm_bindgen]
impl PathToFileSourceMap {
#[wasm_bindgen(constructor)]
pub fn new() -> PathToFileSourceMap {
PathToFileSourceMap::default()
}
// Inserts a path and its source code into the map.
//
// Returns true, if there was already source code in the map for the given path
pub fn add_source_code(&mut self, path: String, source_code: String) -> bool {
let path_buf = Path::new(&path).to_path_buf();
let old_value = self.0.insert(path_buf, source_code);
old_value.is_some()
}
}

pub enum CompileResult {
Contract { contract: PreprocessedContract, debug: DebugArtifact },
Program { program: PreprocessedProgram, debug: DebugArtifact },
Expand All @@ -136,6 +161,7 @@ pub fn compile(
entry_point: String,
contracts: Option<bool>,
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
dependency_graph: Option<JsDependencyGraph>,
file_source_map: PathToFileSourceMap,
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
) -> Result<JsCompileResult, JsCompileError> {
console_error_panic_hook::set_once();

Expand All @@ -154,7 +180,7 @@ pub fn compile(
let path = Path::new(&entry_point);
let crate_id = prepare_crate(&mut context, path);

process_dependency_graph(&mut context, dependency_graph);
process_dependency_graph(&mut context, dependency_graph, file_source_map);

let compile_options = CompileOptions::default();

Expand Down Expand Up @@ -204,11 +230,20 @@ pub fn compile(
// These will be in the Nargo.toml of the package being compiled.
//
// Library dependencies are transitive dependencies; for example, if the entry-point relies
// upon some library `lib1`. Then the packages that `lib1` depend upon will be placed in the
// upon some library `lib1`. Then the packages that `lib1` depend upon will be placed in the
// `library_dependencies` list and the `lib1` will be placed in the `root_dependencies` list.
fn process_dependency_graph(context: &mut Context, dependency_graph: DependencyGraph) {
fn process_dependency_graph(
context: &mut Context,
dependency_graph: DependencyGraph,
file_source_map: PathToFileSourceMap,
) {
let mut crate_names: HashMap<&CrateName, CrateId> = HashMap::new();

// Add all files from the file_source_map into the file manager
for (path, source) in file_source_map.0 {
context.file_manager.add_file_with_source(path.as_path(), source);
}

for lib in &dependency_graph.root_dependencies {
let crate_id = add_noir_lib(context, lib);
crate_names.insert(lib, crate_id);
Expand Down Expand Up @@ -285,6 +320,14 @@ fn preprocess_contract(contract: CompiledContract) -> CompileResult {
CompileResult::Contract { contract: preprocessed_contract, debug: debug_artifact }
}

// TODO: This is no longer needed because we are passing in a map from every path
// TODO: to the source file.
// TODO: how things get resolved are now the responsibility of the caller
// TODO: We will have future PRs which make this resolution nicer by taking in a Nargo.toml
// TODO: and producing paths with source files, though for now, I think this API is okay
//
// TODO: We will also be able to remove the file_reader being a parameter to FileManager but
// TODO will stay until we have this working so we don't break the API too much.
cfg_if::cfg_if! {
if #[cfg(target_os = "wasi")] {
fn get_non_stdlib_asset(path_to_file: &Path) -> std::io::Result<String> {
Expand Down Expand Up @@ -318,6 +361,8 @@ mod test {
hir::Context,
};

use crate::compile::PathToFileSourceMap;

use super::{process_dependency_graph, DependencyGraph};
use std::{collections::HashMap, path::Path};

Expand Down Expand Up @@ -345,7 +390,7 @@ mod test {
let dependency_graph =
DependencyGraph { root_dependencies: vec![], library_dependencies: HashMap::new() };

process_dependency_graph(&mut context, dependency_graph);
process_dependency_graph(&mut context, dependency_graph, PathToFileSourceMap::default());

// one stdlib + one root crate
assert_eq!(context.crate_graph.number_of_crates(), 2);
Expand All @@ -359,7 +404,13 @@ mod test {
library_dependencies: HashMap::new(),
};

process_dependency_graph(&mut context, dependency_graph);
let path_to_source_map = PathToFileSourceMap(
vec![(Path::new("/lib1/lib.nr").to_path_buf(), "fn foo() {}".to_string())]
.into_iter()
.collect(),
);

process_dependency_graph(&mut context, dependency_graph, path_to_source_map);

assert_eq!(context.crate_graph.number_of_crates(), 3);
}
Expand All @@ -372,7 +423,12 @@ mod test {
library_dependencies: HashMap::new(),
};

process_dependency_graph(&mut context, dependency_graph);
let path_to_source_map = PathToFileSourceMap(
vec![(Path::new("lib1/lib.nr").to_path_buf(), "fn foo() {}".to_string())]
.into_iter()
.collect(),
);
process_dependency_graph(&mut context, dependency_graph, path_to_source_map);

assert_eq!(context.crate_graph.number_of_crates(), 3);
}
Expand All @@ -388,7 +444,17 @@ mod test {
]),
};

process_dependency_graph(&mut context, dependency_graph);
let path_to_source_map = PathToFileSourceMap(
vec![
(Path::new("lib1/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
(Path::new("lib2/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
(Path::new("lib3/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
]
.into_iter()
.collect(),
);

process_dependency_graph(&mut context, dependency_graph, path_to_source_map);

assert_eq!(context.crate_graph.number_of_crates(), 5);
}
Expand All @@ -401,7 +467,16 @@ mod test {
library_dependencies: HashMap::from([(crate_name("lib2"), vec![crate_name("lib3")])]),
};

process_dependency_graph(&mut context, dependency_graph);
let path_to_source_map = PathToFileSourceMap(
vec![
(Path::new("lib1/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
(Path::new("lib2/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
(Path::new("lib3/lib.nr").to_path_buf(), "fn foo() {}".to_string()),
]
.into_iter()
.collect(),
);
process_dependency_graph(&mut context, dependency_graph, path_to_source_map);

assert_eq!(context.crate_graph.number_of_crates(), 5);
}
Expand Down
Loading