-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GC unit test infra, start adding GC tests #2573
Merged
Merged
Changes from all commits
Commits
Show all changes
108 commits
Select commit
Hold shift + click to select a range
6cf563e
WIP
osa1 0c18d9c
WIP refactoring
osa1 8da077d
Refactor type signatures to reduce duplication
osa1 02543c8
Refactor dump functions to allow use in tests
osa1 b62a64e
Fix a few bugs, start printing the heap (kinda works)
osa1 f25baca
Fixing bugs
osa1 3ef2717
Delete old code
osa1 1b00957
The heap is now recognized by the RTS
osa1 0799696
Start calling GC -- does not work yet, need to implement grow_memory
osa1 d062634
Fix a few more bugs, improve debug prints, run GC a few times just to…
osa1 03beefb
Minor refactoring, clarify "address" and "offset"
osa1 ae4bafd
Implement dynamic heap sanity check
osa1 2a2ca9e
Minor refactoring: simplify word reads/writes
osa1 a870a5e
Check heap after each GC
osa1 4c995b1
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 9784923
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 2114514
Post-merge fixups
osa1 030ab98
WIP
osa1 05f2cec
Introduce Heap trait
osa1 9656aff
Commit WIP refactoring
osa1 9e7b421
Simplify wrapper functions
osa1 fbce72c
Replace rest of no_mangles with ic_fn
osa1 5808b43
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 409ecbe
Fix compile errors, add a hack to Cargo.toml for vendoring, to be fix…
osa1 28d2f61
Gradually fixing compile errors -- WIP
osa1 38ba5ee
More refactoring, move GC methods out of Heap trait
osa1 79190d1
Start using heap trait in tests, enable bitmap tests
osa1 9a91c38
Enable closure_table tests
osa1 3383c09
Enable BigInt tests
osa1 d098ebe
Enable UTF8 tests
osa1 1baf71f
Enable crc32 tests
osa1 3c66f72
Enable principal id tests
osa1 709065a
Enable text tests
osa1 9acb4b9
Enable leb128 tests -- does not work yet
osa1 0bc4f8d
Remove quickcheck, port more tests to proptest
osa1 54ffd35
Enable mark_stack tests
osa1 f4b3d5d
Fix some of the warnings
osa1 e4f1d33
Disble failing test for now
osa1 dd3964f
Update rustDeps sha256
osa1 3a20d7c
Refactoring GC tests
osa1 4ddc361
Simplify GC function signatures
osa1 d049a0e
WIP: refactoring GC tests
osa1 27aad94
Fix GC tests, finally all tests run
osa1 b68e1ef
Disable big-array-access for now
osa1 870bd18
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 5869205
Fix live size computation in compacting GC
osa1 c908628
Simplify debug fn signatures
osa1 db66f1a
Refactor GC tests to allow running both GCs, fix some warns
osa1 69c597e
WIP refactoring GC tests to run both collectors
osa1 63beb36
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 687db6d
Compacting GC bug fixed in master, enable the test
osa1 58865ec
Remove duplicate values of TAG_{MUTBOX,ARRAY}, WORD_SIZE
osa1 a036205
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 3dd381f
WIP refactoring GC tests
osa1 fac274f
Slightly refactor create_dynamic_heap
osa1 77b0d98
Move creation of static heap to its own function
osa1 fa3186a
Move heap size calculation to its own function
osa1 dce0cd1
Refactor GC tests for top-down reading
osa1 4ad814b
More refactoring, move GC test types to utils
osa1 025ee6d
Remove old file
osa1 66d3f74
Remove old commented-out code
osa1 6a7ec55
Remove grow_memory from the trait
osa1 4eb606b
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 417fb93
Fix warnings in tests, except one
osa1 55cec0b
Remove panic_handler Cargo feature, document ic feature
osa1 607faf0
Minor clarification in a doc
osa1 e88fa36
Move GC specific types to gc module
osa1 760866d
Rename mem -> mem_utils
osa1 5affad9
Rename Heap -> Memory
osa1 d678d51
Revert accidental renamings in prev commit
osa1 3125f6d
Remove get_hp args from GC functions, get_hp is now in Memory
osa1 3a43f06
Revert more accidental renaming
osa1 ca6252f
Revert more accidental renaming
osa1 071c248
More accidental renaming
osa1 3090a72
Remove an unused var
osa1 95ff52c
Typo fix [ci skip]
osa1 339ce33
Replace hard-coded word sizes with WORD_SIZE
osa1 e53c4ed
Revert more accidental renaming
osa1 bbf91fe
Update motoko-rts-macros crate authors
osa1 521081a
Typo fix
osa1 c9e2740
Remove get_hp method of Memory
osa1 f96c1ee
Refactor check_dynamic_heap:
osa1 13e1bfe
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 75f5aa8
Check reachable objects in check_dynamic_heap
osa1 d85fbf9
More alloc_blob and alloc_array helpers out of Memory trait
osa1 2931780
Replace BTreeMaps in tests with HashMap
osa1 e67b10b
Improve leb128 tests
osa1 4edec95
Fix sleb128 decode
osa1 cf582a7
Fix compile error
osa1 25b0768
Documentation
osa1 3621e9c
Remove dependency hack in motoko-rts-tests
osa1 7634e7b
Carefully place two inline attributes
osa1 5ca4745
Remove commented-out code
osa1 5771fcf
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 65f0a14
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 f772fc0
Typo fix
osa1 9b0d209
Fix warnings when building tests
osa1 238b982
Remove ic_fn macro, not sure if it's worth adding more magic
osa1 974d118
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 01bede9
Apply suggestions from code review
osa1 f75cd79
Add functions for making scalar and pointers, minor refactoring
osa1 4e95bd5
Replace hard-coded header size constants with size_of
osa1 543ed46
Update ObjectIdx documentation
osa1 cac605d
Remove invalid TODO
osa1 003f07a
Document ic_mem_fn
osa1 3f88699
Merge remote-tracking branch 'origin/master' into osa1/rts_tests
osa1 8d68639
Remove a few TODO comments
osa1 1fa43a8
Merge two #[allow] attributes
osa1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
[package] | ||
name = "motoko-rts-macros" | ||
version = "0.1.0" | ||
authors = ["dfinity <sdk@dfinity.org>"] | ||
edition = "2018" | ||
|
||
[lib] | ||
proc_macro = true | ||
|
||
[dependencies] | ||
proc-macro2 = "1.0.27" | ||
syn = { version = "1.0.73", features = ["full"] } | ||
quote = "1.0.9" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
use proc_macro::TokenStream; | ||
use quote::quote; | ||
|
||
/// This macro is used to generate monomorphic versions of allocating RTS functions, to allow | ||
/// calling such functions in generated code. Example: | ||
/// | ||
/// ``` | ||
/// #[ic_mem_fn] | ||
/// pub unsafe fn text_concat<M: Memory>(mem: &mut M, s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr { | ||
/// ... | ||
/// } | ||
/// ``` | ||
/// | ||
/// This functions has a `Memory` parameter to be able to allocate on heap. To compile this | ||
/// function to use in generated code we need a monomorphic version, without a `Memory` parameter. | ||
/// This macro generates the monomorphic version. Macro expansion looks like this: | ||
/// | ||
/// ``` | ||
/// // Original function generated directly, to allow use from the test suite | ||
/// pub unsafe fn text_concat<M: Memory>(mem: &mut M, s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr { | ||
/// ... | ||
/// } | ||
/// | ||
/// // New, monomorphic version | ||
/// #[cfg(feature = "ic")] | ||
/// #[export_name = "text_concat"] | ||
/// unsafe extern "C" fn ic_text_concat(s1: SkewedPtr, s2: SkewedPtr) -> SkewedPtr { | ||
/// text_concat(crate::memory::ic::IcMemory, s1, s2) | ||
/// } | ||
/// ``` | ||
/// | ||
/// Reminder: `ic` feature is used when compiling the RTS to be linked with generated code. It's | ||
/// disabled when compiling for testing. | ||
/// | ||
/// `ic_mem_fn` takes an optional `ic_only` attribute which adds a `cfg(feature = "ic")` guard to | ||
/// the original function: | ||
/// | ||
/// ``` | ||
/// #[ic_mem_fn(ic_only)] | ||
/// fn my_function<M: Memory>(mem: &mut M) { ... } | ||
/// ``` | ||
/// | ||
/// Expansion: | ||
/// | ||
/// ``` | ||
/// #[cfg(feature = "ic")] | ||
/// fn my_function<M: Memory>(mem: &mut M) { ... } | ||
/// | ||
/// #[cfg(feature = "ic")] | ||
/// #[export_name = "text_concat"] | ||
/// unsafe extern "C" fn ic_my_function() { | ||
/// my_function(crate::memory::ic::IcMemory) | ||
/// } | ||
/// ``` | ||
/// | ||
/// This is useful when the function won't be used when compiling the RTS for testing. | ||
#[proc_macro_attribute] | ||
pub fn ic_mem_fn(attr: TokenStream, input: TokenStream) -> TokenStream { | ||
let ic_only = if attr.is_empty() { | ||
false | ||
} else if attr.to_string() == "ic_only" { | ||
true | ||
} else { | ||
panic!("Unknown attribute: {:?}", attr.to_string()); | ||
}; | ||
|
||
let fun = syn::parse_macro_input!(input as syn::ItemFn); | ||
let fun_sig = &fun.sig; | ||
|
||
// Some sanity checks | ||
assert!(fun_sig.asyncness.is_none(), "IC functions cannot be async"); | ||
assert_eq!( | ||
fun_sig.generics.params.len(), | ||
1, | ||
"IC memory functions should have one generic argument for the memory implementation" | ||
); | ||
assert!( | ||
fun_sig.abi.is_none(), | ||
"Functions with #[ic_fn] attribute cannot have ABI annotations" | ||
); | ||
assert!( | ||
fun_sig.variadic.is_none(), | ||
"IC functions cannot have variadic arguments" | ||
); | ||
|
||
let fn_ident = &fun_sig.ident; | ||
let fn_wrapper_ident = syn::Ident::new(&format!("ic_{}", fn_ident), fn_ident.span()); | ||
let fn_name = fn_ident.to_string(); | ||
let wrapper_ret = fun_sig.output.clone(); | ||
let wrapper_args: Vec<(syn::Ident, syn::Type)> = fun_sig | ||
.inputs | ||
.iter() | ||
.enumerate() | ||
.filter_map(|(i, arg)| match arg { | ||
syn::FnArg::Receiver(_) => { | ||
panic!("IC functions can't have receivers (`&self`, `&mut self`, etc.)") | ||
} | ||
syn::FnArg::Typed(pat) => { | ||
if i == 0 { | ||
// First argument should be `memory`, skip | ||
None | ||
} else { | ||
Some(( | ||
syn::Ident::new(&format!("arg{}", i), proc_macro2::Span::call_site()), | ||
(*pat.ty).clone(), | ||
)) | ||
} | ||
} | ||
}) | ||
.collect(); | ||
|
||
// Parameters of the wrapper function | ||
let wrapper_params_syn: Vec<proc_macro2::TokenStream> = wrapper_args | ||
.iter() | ||
.map(|(ident, ty)| quote!(#ident: #ty)) | ||
.collect(); | ||
|
||
// Arguments passed to the original function | ||
let wrapper_args_syn: Vec<&syn::Ident> = wrapper_args.iter().map(|(ident, _)| ident).collect(); | ||
|
||
let fun_attr = if ic_only { | ||
quote!(#[cfg(feature = "ic")]) | ||
} else { | ||
quote!() | ||
}; | ||
|
||
quote!( | ||
#fun_attr | ||
#fun | ||
|
||
#[cfg(feature = "ic")] | ||
#[export_name = #fn_name] | ||
unsafe extern "C" fn #fn_wrapper_ident(#(#wrapper_params_syn,)*) #wrapper_ret { | ||
#fn_ident(&mut crate::memory::ic::IcMemory, #(#wrapper_args_syn,)*) | ||
} | ||
) | ||
.into() | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto here I guess.