[Refactor] Refactor building system and adding compiled products #261
+1,965
−1,657
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactor building system and adding compiled products
Now, we can build
Function
,IRModule
,Task
andFlowGraph
toCompiledFunction
,CompiledModule
,CompiledTask
andCompiledModule
. The compiled products can be saved on the disk and loaded for running.The
CompiledTask
has alib.so
that contains all the scheduled kernels. So we will not have the problem of loading thousands of dynamic libraries when we support dynamic shape & kernel tuning (e.g., 100 operators * 200 schedules for each operator).The
CompiledGraph
contains a C runtime that invokles the kernels and allocates memory. With this, we do not need CUDA graph to get rid of the framework overhead when inference. TheCompiledGraph
can be stored as a single file on the disk. Thus, we can compile one model on one machine and then tune & run the model on another machine.Minors
target
argument tobuild_task
andbuild_ir_module
to specify the target for code generation and underlying compiler (e.g., nvcc vs. gcc)IRModule
fromhidet.ir.func
tohidet.ir.module
.