Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Refactor building system and adding compiled products #261

Merged
merged 2 commits into from
Jun 2, 2023

Conversation

yaoyaoding
Copy link
Member

Refactor building system and adding compiled products

Now, we can build Function, IRModule, Task and FlowGraph to CompiledFunction, CompiledModule, CompiledTask and CompiledModule. The compiled products can be saved on the disk and loaded for running.

The CompiledTask has a lib.so that contains all the scheduled kernels. So we will not have the problem of loading thousands of dynamic libraries when we support dynamic shape & kernel tuning (e.g., 100 operators * 200 schedules for each operator).

The CompiledGraph contains a C runtime that invokles the kernels and allocates memory. With this, we do not need CUDA graph to get rid of the framework overhead when inference. The CompiledGraph can be stored as a single file on the disk. Thus, we can compile one model on one machine and then tune & run the model on another machine.

Minors

  • Add target argument to build_task and build_ir_module to specify the target for code generation and underlying compiler (e.g., nvcc vs. gcc)
  • Move IRModule from hidet.ir.func to hidet.ir.module.
  • Changed the format to define tuning space (may change again in the future).

.
.

.

.

.

.

fix

.

.

.

.

.

.

.

.

.

.

.

.
@yaoyaoding yaoyaoding merged commit 3269ee5 into hidet-org:main Jun 2, 2023
@yaoyaoding yaoyaoding deleted the template branch June 2, 2023 15:29
vadiklyutiy added a commit that referenced this pull request Jul 22, 2024
vadiklyutiy added a commit that referenced this pull request Jul 23, 2024
vadiklyutiy added a commit that referenced this pull request Dec 26, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Use attention by default
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant