Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC unit test infra, start adding GC tests #2573

Merged
merged 108 commits into from
Jun 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
6cf563e
WIP
osa1 Jun 8, 2021
0c18d9c
WIP refactoring
osa1 Jun 9, 2021
8da077d
Refactor type signatures to reduce duplication
osa1 Jun 9, 2021
02543c8
Refactor dump functions to allow use in tests
osa1 Jun 9, 2021
b62a64e
Fix a few bugs, start printing the heap (kinda works)
osa1 Jun 9, 2021
f25baca
Fixing bugs
osa1 Jun 9, 2021
3ef2717
Delete old code
osa1 Jun 9, 2021
1b00957
The heap is now recognized by the RTS
osa1 Jun 9, 2021
0799696
Start calling GC -- does not work yet, need to implement grow_memory
osa1 Jun 9, 2021
d062634
Fix a few more bugs, improve debug prints, run GC a few times just to…
osa1 Jun 10, 2021
03beefb
Minor refactoring, clarify "address" and "offset"
osa1 Jun 10, 2021
ae4bafd
Implement dynamic heap sanity check
osa1 Jun 10, 2021
2a2ca9e
Minor refactoring: simplify word reads/writes
osa1 Jun 10, 2021
a870a5e
Check heap after each GC
osa1 Jun 10, 2021
4c995b1
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 10, 2021
9784923
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 15, 2021
2114514
Post-merge fixups
osa1 Jun 15, 2021
030ab98
WIP
osa1 Jun 16, 2021
05f2cec
Introduce Heap trait
osa1 Jun 17, 2021
9656aff
Commit WIP refactoring
osa1 Jun 17, 2021
9e7b421
Simplify wrapper functions
osa1 Jun 17, 2021
fbce72c
Replace rest of no_mangles with ic_fn
osa1 Jun 21, 2021
5808b43
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 21, 2021
409ecbe
Fix compile errors, add a hack to Cargo.toml for vendoring, to be fix…
osa1 Jun 21, 2021
28d2f61
Gradually fixing compile errors -- WIP
osa1 Jun 21, 2021
38ba5ee
More refactoring, move GC methods out of Heap trait
osa1 Jun 21, 2021
79190d1
Start using heap trait in tests, enable bitmap tests
osa1 Jun 22, 2021
9a91c38
Enable closure_table tests
osa1 Jun 22, 2021
3383c09
Enable BigInt tests
osa1 Jun 22, 2021
d098ebe
Enable UTF8 tests
osa1 Jun 22, 2021
1baf71f
Enable crc32 tests
osa1 Jun 22, 2021
3c66f72
Enable principal id tests
osa1 Jun 22, 2021
709065a
Enable text tests
osa1 Jun 22, 2021
9acb4b9
Enable leb128 tests -- does not work yet
osa1 Jun 22, 2021
0bc4f8d
Remove quickcheck, port more tests to proptest
osa1 Jun 22, 2021
54ffd35
Enable mark_stack tests
osa1 Jun 22, 2021
f4b3d5d
Fix some of the warnings
osa1 Jun 22, 2021
e4f1d33
Disble failing test for now
osa1 Jun 22, 2021
dd3964f
Update rustDeps sha256
osa1 Jun 22, 2021
3a20d7c
Refactoring GC tests
osa1 Jun 22, 2021
4ddc361
Simplify GC function signatures
osa1 Jun 22, 2021
d049a0e
WIP: refactoring GC tests
osa1 Jun 22, 2021
27aad94
Fix GC tests, finally all tests run
osa1 Jun 22, 2021
b68e1ef
Disable big-array-access for now
osa1 Jun 23, 2021
870bd18
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 23, 2021
5869205
Fix live size computation in compacting GC
osa1 Jun 23, 2021
c908628
Simplify debug fn signatures
osa1 Jun 23, 2021
db66f1a
Refactor GC tests to allow running both GCs, fix some warns
osa1 Jun 23, 2021
69c597e
WIP refactoring GC tests to run both collectors
osa1 Jun 23, 2021
63beb36
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 23, 2021
687db6d
Compacting GC bug fixed in master, enable the test
osa1 Jun 23, 2021
58865ec
Remove duplicate values of TAG_{MUTBOX,ARRAY}, WORD_SIZE
osa1 Jun 23, 2021
a036205
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 23, 2021
3dd381f
WIP refactoring GC tests
osa1 Jun 24, 2021
fac274f
Slightly refactor create_dynamic_heap
osa1 Jun 24, 2021
77b0d98
Move creation of static heap to its own function
osa1 Jun 24, 2021
fa3186a
Move heap size calculation to its own function
osa1 Jun 24, 2021
dce0cd1
Refactor GC tests for top-down reading
osa1 Jun 24, 2021
4ad814b
More refactoring, move GC test types to utils
osa1 Jun 24, 2021
025ee6d
Remove old file
osa1 Jun 24, 2021
66d3f74
Remove old commented-out code
osa1 Jun 24, 2021
6a7ec55
Remove grow_memory from the trait
osa1 Jun 24, 2021
4eb606b
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 24, 2021
417fb93
Fix warnings in tests, except one
osa1 Jun 24, 2021
55cec0b
Remove panic_handler Cargo feature, document ic feature
osa1 Jun 24, 2021
607faf0
Minor clarification in a doc
osa1 Jun 24, 2021
e88fa36
Move GC specific types to gc module
osa1 Jun 24, 2021
760866d
Rename mem -> mem_utils
osa1 Jun 24, 2021
5affad9
Rename Heap -> Memory
osa1 Jun 24, 2021
d678d51
Revert accidental renamings in prev commit
osa1 Jun 24, 2021
3125f6d
Remove get_hp args from GC functions, get_hp is now in Memory
osa1 Jun 24, 2021
3a43f06
Revert more accidental renaming
osa1 Jun 24, 2021
ca6252f
Revert more accidental renaming
osa1 Jun 24, 2021
071c248
More accidental renaming
osa1 Jun 24, 2021
3090a72
Remove an unused var
osa1 Jun 24, 2021
95ff52c
Typo fix [ci skip]
osa1 Jun 24, 2021
339ce33
Replace hard-coded word sizes with WORD_SIZE
osa1 Jun 24, 2021
e53c4ed
Revert more accidental renaming
osa1 Jun 24, 2021
bbf91fe
Update motoko-rts-macros crate authors
osa1 Jun 24, 2021
521081a
Typo fix
osa1 Jun 24, 2021
c9e2740
Remove get_hp method of Memory
osa1 Jun 24, 2021
f96c1ee
Refactor check_dynamic_heap:
osa1 Jun 28, 2021
13e1bfe
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 28, 2021
75f5aa8
Check reachable objects in check_dynamic_heap
osa1 Jun 28, 2021
d85fbf9
More alloc_blob and alloc_array helpers out of Memory trait
osa1 Jun 28, 2021
2931780
Replace BTreeMaps in tests with HashMap
osa1 Jun 28, 2021
e67b10b
Improve leb128 tests
osa1 Jun 28, 2021
4edec95
Fix sleb128 decode
osa1 Jun 28, 2021
cf582a7
Fix compile error
osa1 Jun 28, 2021
25b0768
Documentation
osa1 Jun 28, 2021
3621e9c
Remove dependency hack in motoko-rts-tests
osa1 Jun 28, 2021
7634e7b
Carefully place two inline attributes
osa1 Jun 29, 2021
5ca4745
Remove commented-out code
osa1 Jun 29, 2021
5771fcf
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 29, 2021
65f0a14
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 29, 2021
f772fc0
Typo fix
osa1 Jun 29, 2021
9b0d209
Fix warnings when building tests
osa1 Jun 29, 2021
238b982
Remove ic_fn macro, not sure if it's worth adding more magic
osa1 Jun 29, 2021
974d118
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 29, 2021
01bede9
Apply suggestions from code review
osa1 Jun 30, 2021
f75cd79
Add functions for making scalar and pointers, minor refactoring
osa1 Jun 30, 2021
4e95bd5
Replace hard-coded header size constants with size_of
osa1 Jun 30, 2021
543ed46
Update ObjectIdx documentation
osa1 Jun 30, 2021
cac605d
Remove invalid TODO
osa1 Jun 30, 2021
003f07a
Document ic_mem_fn
osa1 Jun 30, 2021
3f88699
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 Jun 30, 2021
8d68639
Remove a few TODO comments
osa1 Jun 30, 2021
1fa43a8
Merge two #[allow] attributes
osa1 Jun 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ rec {
name = "motoko-rts-deps";
src = subpath ./rts;
sourceRoot = "rts/motoko-rts-tests";
sha256 = "13pcsik4nq9w7dwi3srhn1ddm30zi5japwn5q50vxknaj0fixi14";
sha256 = "1vr9mvjrddjv7xga6hhzq39x8qzdqsnhwic76apv7ksfkh0psfx2";
copyLockfile = true;
};
in
Expand Down
45 changes: 45 additions & 0 deletions rts/motoko-rts-macros/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions rts/motoko-rts-macros/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[package]
name = "motoko-rts-macros"
version = "0.1.0"
authors = ["dfinity <sdk@dfinity.org>"]
edition = "2018"

[lib]
proc_macro = true

[dependencies]
proc-macro2 = "1.0.27"
syn = { version = "1.0.73", features = ["full"] }
quote = "1.0.9"
138 changes: 138 additions & 0 deletions rts/motoko-rts-macros/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
use proc_macro::TokenStream;
use quote::quote;

/// This macro is used to generate monomorphic versions of allocating RTS functions, to allow
/// calling such functions in generated code. Example:
///
/// ```
/// #[ic_mem_fn]
/// pub unsafe fn text_concat<M: Memory>(mem: &mut M, s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr {
/// ...
/// }
/// ```
///
/// This functions has a `Memory` parameter to be able to allocate on heap. To compile this
/// function to use in generated code we need a monomorphic version, without a `Memory` parameter.
/// This macro generates the monomorphic version. Macro expansion looks like this:
///
/// ```
/// // Original function generated directly, to allow use from the test suite
/// pub unsafe fn text_concat<M: Memory>(mem: &mut M, s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr {
/// ...
/// }
///
/// // New, monomorphic version
/// #[cfg(feature = "ic")]
/// #[export_name = "text_concat"]
/// unsafe extern "C" fn ic_text_concat(s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr {
/// text_concat(crate::memory::ic::IcMemory, s1, s2)
/// }
/// ```
///
/// Reminder: `ic` feature is used when compiling the RTS to be linked with generated code. It's
/// disabled when compiling for testing.
///
/// `ic_mem_fn` takes an optional `ic_only` attribute which adds a `cfg(feature = "ic")` guard to
/// the original function:
///
/// ```
/// #[ic_mem_fn(ic_only)]
/// fn my_function<M: Memory>(mem: &mut M) { ... }
/// ```
///
/// Expansion:
///
/// ```
/// #[cfg(feature = "ic")]
/// fn my_function<M: Memory>(mem: &mut M) { ... }
///
/// #[cfg(feature = "ic")]
/// #[export_name = "text_concat"]
/// unsafe extern "C" fn ic_my_function() {
/// my_function(crate::memory::ic::IcMemory)
/// }
/// ```
///
/// This is useful when the function won't be used when compiling the RTS for testing.
#[proc_macro_attribute]
pub fn ic_mem_fn(attr: TokenStream, input: TokenStream) -> TokenStream {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here I guess.

let ic_only = if attr.is_empty() {
false
} else if attr.to_string() == "ic_only" {
true
} else {
panic!("Unknown attribute: {:?}", attr.to_string());
};

let fun = syn::parse_macro_input!(input as syn::ItemFn);
let fun_sig = &fun.sig;

// Some sanity checks
assert!(fun_sig.asyncness.is_none(), "IC functions cannot be async");
assert_eq!(
fun_sig.generics.params.len(),
1,
"IC memory functions should have one generic argument for the memory implementation"
);
assert!(
fun_sig.abi.is_none(),
"Functions with #[ic_fn] attribute cannot have ABI annotations"
);
assert!(
fun_sig.variadic.is_none(),
"IC functions cannot have variadic arguments"
);

let fn_ident = &fun_sig.ident;
let fn_wrapper_ident = syn::Ident::new(&format!("ic_{}", fn_ident), fn_ident.span());
let fn_name = fn_ident.to_string();
let wrapper_ret = fun_sig.output.clone();
let wrapper_args: Vec<(syn::Ident, syn::Type)> = fun_sig
.inputs
.iter()
.enumerate()
.filter_map(|(i, arg)| match arg {
syn::FnArg::Receiver(_) => {
panic!("IC functions can't have receivers (`&self`, `&mut self`, etc.)")
}
syn::FnArg::Typed(pat) => {
if i == 0 {
// First argument should be `memory`, skip
None
} else {
Some((
syn::Ident::new(&format!("arg{}", i), proc_macro2::Span::call_site()),
(*pat.ty).clone(),
))
}
}
})
.collect();

// Parameters of the wrapper function
let wrapper_params_syn: Vec<proc_macro2::TokenStream> = wrapper_args
.iter()
.map(|(ident, ty)| quote!(#ident: #ty))
.collect();

// Arguments passed to the original function
let wrapper_args_syn: Vec<&syn::Ident> = wrapper_args.iter().map(|(ident, _)| ident).collect();

let fun_attr = if ic_only {
quote!(#[cfg(feature = "ic")])
} else {
quote!()
};

quote!(
#fun_attr
#fun

#[cfg(feature = "ic")]
#[export_name = #fn_name]
unsafe extern "C" fn #fn_wrapper_ident(#(#wrapper_params_syn,)*) #wrapper_ret {
#fn_ident(&mut crate::memory::ic::IcMemory, #(#wrapper_args_syn,)*)
}
)
.into()
}
Loading