diff --git a/v0.10.0/en/_sources/software/cli.rst.txt b/v0.10.0/en/_sources/software/cli.rst.txt
index ef1fd13..bd43b6c 100644
--- a/v0.10.0/en/_sources/software/cli.rst.txt
+++ b/v0.10.0/en/_sources/software/cli.rst.txt
@@ -191,11 +191,12 @@ The following is an example usage of furiosa-bench without an output path option
99th percentile (us): 79874
99th percentile (us): 307568
-If an output path is specified, furiosa-bench will save a json document as the following:
+If an output path is specified, furiosa-bench will save a JSON document as the following:
.. code-block:: sh
- $ furiosa-bench mnist-8.onnx --workload L -n 1000 -w 8 -t 2 -o mnist.json | cat mnist.json
+ $ furiosa-bench mnist-8.onnx --workload L -n 1000 -w 8 -t 2 -o mnist.json
+ $ cat mnist.json
{
"model_data": {
diff --git a/v0.10.0/en/_sources/software/compiler.rst.txt b/v0.10.0/en/_sources/software/compiler.rst.txt
index eb6b951..63f6792 100644
--- a/v0.10.0/en/_sources/software/compiler.rst.txt
+++ b/v0.10.0/en/_sources/software/compiler.rst.txt
@@ -76,7 +76,7 @@ The usage of ``furiosa-compiler`` is as the following:
``SOURCE`` is the file path of
-`TFLite `_ or `Onnx `_.
+`TFLite `_ or `ONNX `_.
You have to use quantized models through :ref:`ModelQuantization` for NPU accleration.
You can omit the option `-o OUTPUT`, and you can also choose to designate the output file name.
diff --git a/v0.10.0/en/_sources/software/profiler.rst.txt b/v0.10.0/en/_sources/software/profiler.rst.txt
index bffa211..aeb2bc5 100644
--- a/v0.10.0/en/_sources/software/profiler.rst.txt
+++ b/v0.10.0/en/_sources/software/profiler.rst.txt
@@ -1,7 +1,7 @@
.. _Profiling:
***********************************
-Performance profiling
+Performance Profiling
***********************************
Low latency and high throughput performance are critical factors in many DNN applications.
@@ -19,7 +19,7 @@ It is thus not enabled by default. You can create trace by using one of the foll
.. _ProfilerEnabledByEnv:
-Tracing Enabled by Environmet Variable
+Tracing via Environment Variable
============================================================
You can enable trace generation by setting the path of the file to which the trace result will be written in ``FURIOSA_PROFILER_OUTPUT_PATH``. The advantage of this method is that the code remains unchanged. The downside is that you cannot set a specific section or category for measurement.
@@ -51,7 +51,7 @@ If you enter ``chrome://tracing`` in Chrome's address bar, the trace viewer will
.. _ProfilerEnabledByContext:
-Tracing Enabled by Profiler Context
+Tracing via Profiler Context
============================================================
You can also trace a model inference performance by using a Profiler Context in your Python code. The advantages of this method, in comparison to the tracing by environment variable, are as follows:
diff --git a/v0.10.0/en/api/python/furiosa.common.html b/v0.10.0/en/api/python/furiosa.common.html
index a12714b..e9359d1 100644
--- a/v0.10.0/en/api/python/furiosa.common.html
+++ b/v0.10.0/en/api/python/furiosa.common.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.common.transport.html b/v0.10.0/en/api/python/furiosa.common.transport.html
index 0011ef8..13cbfc2 100644
--- a/v0.10.0/en/api/python/furiosa.common.transport.html
+++ b/v0.10.0/en/api/python/furiosa.common.transport.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.frontend.html b/v0.10.0/en/api/python/furiosa.optimizer.frontend.html
index 822c30c..c9e4a50 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.frontend.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.frontend.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.html b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.html
index df01a0b..d3ef467 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.html b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.html
index c42a682..461428c 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.html b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.html
index 88edc6e..71e2a15 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.transformer.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.utils.html b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.utils.html
index d1dcc80..f3ce64d 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.utils.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.frontend.onnx.utils.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.html b/v0.10.0/en/api/python/furiosa.optimizer.html
index a07d019..4a361b7 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.optimizer.interfaces.html b/v0.10.0/en/api/python/furiosa.optimizer.interfaces.html
index 6720a31..13b5c42 100644
--- a/v0.10.0/en/api/python/furiosa.optimizer.interfaces.html
+++ b/v0.10.0/en/api/python/furiosa.optimizer.interfaces.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.quantizer.html b/v0.10.0/en/api/python/furiosa.quantizer.html
index 5ca5d8d..6bafd7a 100644
--- a/v0.10.0/en/api/python/furiosa.quantizer.html
+++ b/v0.10.0/en/api/python/furiosa.quantizer.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.runtime.html b/v0.10.0/en/api/python/furiosa.runtime.html
index e9c0d65..aa87536 100644
--- a/v0.10.0/en/api/python/furiosa.runtime.html
+++ b/v0.10.0/en/api/python/furiosa.runtime.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.api.cli.html b/v0.10.0/en/api/python/furiosa.server.api.cli.html
index c3597b9..6eb8203 100644
--- a/v0.10.0/en/api/python/furiosa.server.api.cli.html
+++ b/v0.10.0/en/api/python/furiosa.server.api.cli.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.api.grpc.generated.html b/v0.10.0/en/api/python/furiosa.server.api.grpc.generated.html
index ae14957..8da4de0 100644
--- a/v0.10.0/en/api/python/furiosa.server.api.grpc.generated.html
+++ b/v0.10.0/en/api/python/furiosa.server.api.grpc.generated.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.api.grpc.html b/v0.10.0/en/api/python/furiosa.server.api.grpc.html
index 23d00a3..8ab01f8 100644
--- a/v0.10.0/en/api/python/furiosa.server.api.grpc.html
+++ b/v0.10.0/en/api/python/furiosa.server.api.grpc.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.api.html b/v0.10.0/en/api/python/furiosa.server.api.html
index bdcdaac..ee50cf0 100644
--- a/v0.10.0/en/api/python/furiosa.server.api.html
+++ b/v0.10.0/en/api/python/furiosa.server.api.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.api.rest.html b/v0.10.0/en/api/python/furiosa.server.api.rest.html
index c2b5c4f..e4c137b 100644
--- a/v0.10.0/en/api/python/furiosa.server.api.rest.html
+++ b/v0.10.0/en/api/python/furiosa.server.api.rest.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.handlers.html b/v0.10.0/en/api/python/furiosa.server.handlers.html
index 7c9a503..656eda1 100644
--- a/v0.10.0/en/api/python/furiosa.server.handlers.html
+++ b/v0.10.0/en/api/python/furiosa.server.handlers.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.html b/v0.10.0/en/api/python/furiosa.server.html
index 78a3e20..51a9c3c 100644
--- a/v0.10.0/en/api/python/furiosa.server.html
+++ b/v0.10.0/en/api/python/furiosa.server.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.types.html b/v0.10.0/en/api/python/furiosa.server.types.html
index 3d5bb4c..21836c7 100644
--- a/v0.10.0/en/api/python/furiosa.server.types.html
+++ b/v0.10.0/en/api/python/furiosa.server.types.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.server.utils.html b/v0.10.0/en/api/python/furiosa.server.utils.html
index 9a75789..53812a7 100644
--- a/v0.10.0/en/api/python/furiosa.server.utils.html
+++ b/v0.10.0/en/api/python/furiosa.server.utils.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.serving.apps.html b/v0.10.0/en/api/python/furiosa.serving.apps.html
index 116a0d6..01b8d48 100644
--- a/v0.10.0/en/api/python/furiosa.serving.apps.html
+++ b/v0.10.0/en/api/python/furiosa.serving.apps.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.serving.html b/v0.10.0/en/api/python/furiosa.serving.html
index 0d0381d..40d165b 100644
--- a/v0.10.0/en/api/python/furiosa.serving.html
+++ b/v0.10.0/en/api/python/furiosa.serving.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/furiosa.serving.processors.html b/v0.10.0/en/api/python/furiosa.serving.processors.html
index a5f6fc6..9a9fb5a 100644
--- a/v0.10.0/en/api/python/furiosa.serving.processors.html
+++ b/v0.10.0/en/api/python/furiosa.serving.processors.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/api/python/modules.html b/v0.10.0/en/api/python/modules.html
index 78b8816..06f2fb4 100644
--- a/v0.10.0/en/api/python/modules.html
+++ b/v0.10.0/en/api/python/modules.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/customer-support/bugs.html b/v0.10.0/en/customer-support/bugs.html
index 5ce3666..0848bc9 100644
--- a/v0.10.0/en/customer-support/bugs.html
+++ b/v0.10.0/en/customer-support/bugs.html
@@ -61,7 +61,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/genindex.html b/v0.10.0/en/genindex.html
index 8e76ba7..d58b5d6 100644
--- a/v0.10.0/en/genindex.html
+++ b/v0.10.0/en/genindex.html
@@ -60,7 +60,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/index.html b/v0.10.0/en/index.html
index ec00150..636dc10 100644
--- a/v0.10.0/en/index.html
+++ b/v0.10.0/en/index.html
@@ -61,7 +61,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/npu/intro.html b/v0.10.0/en/npu/intro.html
index a5ff5b7..3d3933a 100644
--- a/v0.10.0/en/npu/intro.html
+++ b/v0.10.0/en/npu/intro.html
@@ -69,7 +69,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/npu/supported_operators.html b/v0.10.0/en/npu/supported_operators.html
index 3d15f4b..6e783c5 100644
--- a/v0.10.0/en/npu/supported_operators.html
+++ b/v0.10.0/en/npu/supported_operators.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/objects.inv b/v0.10.0/en/objects.inv
index 1241d28..aabd79c 100644
Binary files a/v0.10.0/en/objects.inv and b/v0.10.0/en/objects.inv differ
diff --git a/v0.10.0/en/py-modindex.html b/v0.10.0/en/py-modindex.html
index b05559c..6393dd5 100644
--- a/v0.10.0/en/py-modindex.html
+++ b/v0.10.0/en/py-modindex.html
@@ -63,7 +63,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/releases/0.5.0.html b/v0.10.0/en/releases/0.5.0.html
index 5f15ed3..0c77092 100644
--- a/v0.10.0/en/releases/0.5.0.html
+++ b/v0.10.0/en/releases/0.5.0.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/releases/0.6.0.html b/v0.10.0/en/releases/0.6.0.html
index 52267f0..914ec49 100644
--- a/v0.10.0/en/releases/0.6.0.html
+++ b/v0.10.0/en/releases/0.6.0.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/releases/0.7.0.html b/v0.10.0/en/releases/0.7.0.html
index e495c69..e630e1a 100644
--- a/v0.10.0/en/releases/0.7.0.html
+++ b/v0.10.0/en/releases/0.7.0.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
@@ -232,7 +232,7 @@ Performance Profiling Tools.
+More details can be found in Performance Profiling .
diff --git a/v0.10.0/en/releases/0.8.0.html b/v0.10.0/en/releases/0.8.0.html
index 56a4276..78b666e 100644
--- a/v0.10.0/en/releases/0.8.0.html
+++ b/v0.10.0/en/releases/0.8.0.html
@@ -63,7 +63,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/releases/0.9.0.html b/v0.10.0/en/releases/0.9.0.html
index 33a001a..138bbe3 100644
--- a/v0.10.0/en/releases/0.9.0.html
+++ b/v0.10.0/en/releases/0.9.0.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/search.html b/v0.10.0/en/search.html
index 2eac221..d9e6863 100644
--- a/v0.10.0/en/search.html
+++ b/v0.10.0/en/search.html
@@ -63,7 +63,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/searchindex.js b/v0.10.0/en/searchindex.js
index 49856a6..050011e 100644
--- a/v0.10.0/en/searchindex.js
+++ b/v0.10.0/en/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["api/python/furiosa.common", "api/python/furiosa.common.transport", "api/python/furiosa.optimizer", "api/python/furiosa.optimizer.frontend", "api/python/furiosa.optimizer.frontend.onnx", "api/python/furiosa.optimizer.frontend.onnx.transformer", "api/python/furiosa.optimizer.frontend.onnx.transformer.experimental", "api/python/furiosa.optimizer.frontend.onnx.utils", "api/python/furiosa.optimizer.interfaces", "api/python/furiosa.quantizer", "api/python/furiosa.runtime", "api/python/furiosa.server", "api/python/furiosa.server.api", "api/python/furiosa.server.api.cli", "api/python/furiosa.server.api.grpc", "api/python/furiosa.server.api.grpc.generated", "api/python/furiosa.server.api.rest", "api/python/furiosa.server.handlers", "api/python/furiosa.server.types", "api/python/furiosa.server.utils", "api/python/furiosa.serving", "api/python/furiosa.serving.apps", "api/python/furiosa.serving.processors", "api/python/modules", "customer-support/bugs", "index", "npu/intro", "npu/supported_operators", "releases/0.5.0", "releases/0.6.0", "releases/0.7.0", "releases/0.8.0", "releases/0.9.0", "software/c-sdk", "software/cli", "software/compiler", "software/installation", "software/intro", "software/kubernetes_support", "software/profiler", "software/python-sdk", "software/quantization", "software/references", "software/serving", "software/tutorials"], "filenames": ["api/python/furiosa.common.rst", "api/python/furiosa.common.transport.rst", "api/python/furiosa.optimizer.rst", "api/python/furiosa.optimizer.frontend.rst", "api/python/furiosa.optimizer.frontend.onnx.rst", "api/python/furiosa.optimizer.frontend.onnx.transformer.rst", "api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.rst", "api/python/furiosa.optimizer.frontend.onnx.utils.rst", "api/python/furiosa.optimizer.interfaces.rst", "api/python/furiosa.quantizer.rst", "api/python/furiosa.runtime.rst", "api/python/furiosa.server.rst", "api/python/furiosa.server.api.rst", "api/python/furiosa.server.api.cli.rst", "api/python/furiosa.server.api.grpc.rst", "api/python/furiosa.server.api.grpc.generated.rst", "api/python/furiosa.server.api.rest.rst", "api/python/furiosa.server.handlers.rst", "api/python/furiosa.server.types.rst", "api/python/furiosa.server.utils.rst", "api/python/furiosa.serving.rst", "api/python/furiosa.serving.apps.rst", "api/python/furiosa.serving.processors.rst", "api/python/modules.rst", "customer-support/bugs.rst", "index.rst", "npu/intro.rst", "npu/supported_operators.rst", "releases/0.5.0.rst", "releases/0.6.0.rst", "releases/0.7.0.rst", "releases/0.8.0.rst", "releases/0.9.0.rst", "software/c-sdk.rst", "software/cli.rst", "software/compiler.rst", "software/installation.rst", "software/intro.rst", "software/kubernetes_support.rst", "software/profiler.rst", "software/python-sdk.rst", "software/quantization.rst", "software/references.rst", "software/serving.rst", "software/tutorials.rst"], "titles": ["furiosa.common package", "furiosa.common.transport package", "furiosa.optimizer package", "furiosa.optimizer.frontend package", "furiosa.optimizer.frontend.onnx package", "furiosa.optimizer.frontend.onnx.transformer package", "furiosa.optimizer.frontend.onnx.transformer.experimental package", "furiosa.optimizer.frontend.onnx.utils package", "furiosa.optimizer.interfaces package", "furiosa.quantizer package", "furiosa.runtime package", "furiosa.server package", "furiosa.server.api package", "furiosa.server.api.cli package", "furiosa.server.api.grpc package", "furiosa.server.api.grpc.generated package", "furiosa.server.api.rest package", "furiosa.server.handlers package", "furiosa.server.types package", "furiosa.server.utils namespace", "furiosa.serving package", "furiosa.serving.apps package", "furiosa.serving.processors package", "Python SDK", "Bug Report", "FuriosaAI NPU & SDK 0.10.0 Documents", "FuriosaAI NPU", "List of Supported Operators for NPU Acceleration", "Release Notes - 0.5.0", "Release Notes - 0.6.0", "Release Notes - 0.7.0", "Release Notes - 0.8.0", "Release Notes - 0.9.0", "C SDK installation and user guide", "Command Line Tools", "Compiler", "Driver, Firmware, and Runtime Installation", "FuriosaAI SW Stack Introduction", "Kubernetes Support", "Performance profiling", "Python SDK installation and user guide", "Model Quantization", "References", "Model Server (Serving Framework)", "Tutorial and Code Examples"], "terms": {"transport": [0, 23, 36], "base": [0, 5, 6, 7, 8, 9, 10, 11, 20, 23, 25, 31, 33, 36, 37, 38, 41, 43, 44], "file": [0, 10, 23, 24, 28, 29, 33, 34, 36, 37, 38, 39], "github": [0, 6, 7, 23, 29, 39, 40, 43], "http": [0, 6, 7, 23, 24, 28, 36, 38, 39, 40, 43], "s3": [0, 23, 31], "except": [0, 1, 10, 11], "furiosaerror": [0, 10], "messag": [0, 10, 24, 43], "str": [0, 1, 2, 4, 5, 7, 9, 10, 11], "gener": [0, 8, 10, 12, 14, 23, 26, 28, 33, 34, 35, 37, 39, 40, 41], "caus": [0, 10, 39], "runtim": [0, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 39, 43], "is_err": 0, "err": 0, "c_int": 0, "int": [0, 2, 4, 5, 6, 7, 10, 31], "bool": [0, 1, 5, 6, 7, 9, 10], "is_ok": 0, "class": [0, 1, 5, 6, 7, 8, 9, 10, 31], "loglevel": [0, 43], "valu": [0, 5, 9, 10, 24, 28, 30, 35, 38], "intenum": [0, 9], "python": [0, 10, 24, 25, 28, 31, 32, 33, 34, 35, 39, 41, 42, 43, 44], "object": [0, 5, 7, 9, 23, 25, 26, 31, 37, 44], "correspondnig": 0, "nux_log_level_t": 0, "nux": [0, 30, 31, 43], "c": [0, 5, 10, 24, 25, 28, 30, 31, 32, 34, 38, 42, 43], "api": [0, 11, 25, 29, 32, 33, 35, 36, 39, 43, 44], "1": [0, 5, 9, 10, 24, 26, 28, 29, 30, 31, 32, 34, 35, 36, 39, 43], "info": [0, 5, 10, 24, 28, 31, 32, 38, 43], "3": [0, 9, 10, 24, 28, 30, 31, 34, 39, 40, 43], "off": [0, 10], "0": [0, 5, 9, 10, 24, 26, 27, 34, 35, 36, 38, 39, 40, 41, 43], "warn": [0, 10, 29, 30, 43], "2": [0, 5, 9, 10, 26, 27, 28, 29, 30, 31, 32, 34, 35, 37, 39], "find_global_lib_path": 0, "libnam": 0, "find_native_lib_path": 0, "none": [0, 2, 4, 5, 6, 7, 9, 10, 11, 32, 34, 38, 40], "find": [0, 1, 24, 28, 29, 30, 31, 32, 33, 35, 38, 39, 40, 43], "librari": [0, 24, 30, 31, 32, 33, 34, 36, 37, 40], "path": [0, 1, 10, 24, 28, 30, 33, 34, 35, 38, 39, 43], "accord": [0, 33, 36, 37, 40], "follow": [0, 1, 5, 10, 24, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "prioriti": 0, "If": [0, 1, 6, 9, 10, 24, 27, 28, 29, 31, 32, 34, 35, 36, 37, 38, 39, 40, 43], "environ": [0, 10, 28, 30, 31, 34, 35, 36, 37, 38, 39, 43], "variabl": [0, 10, 28, 30, 31, 35], "ld_library_path": 0, "i": [0, 1, 5, 9, 10, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "set": [0, 10, 23, 30, 31, 32, 33, 34, 35, 38, 39, 40], "thi": [0, 1, 5, 9, 10, 23, 24, 25, 27, 28, 31, 34, 35, 36, 37, 38, 39, 40, 41, 43], "function": [0, 5, 10, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 40], "tri": [0, 10], "found": [0, 26, 28, 29, 30, 31, 32, 34, 35, 37, 40], "from": [0, 1, 6, 9, 10, 24, 25, 28, 29, 30, 31, 32, 34, 35, 36, 39, 40, 41, 43, 44], "otherwis": [0, 10], "global": 0, "usr": [0, 24, 31, 33, 34, 38, 39], "lib": [0, 33, 38], "local": [0, 1, 24, 28, 30, 31, 34, 35, 43], "still": [0, 10], "cannot": [0, 10, 30, 33, 39], "return": [0, 1, 9, 10, 29, 31, 43], "find_native_lib": 0, "register_hook": 0, "type": [0, 1, 9, 11, 23, 26, 28, 29, 30, 35, 37, 38, 41], "callabl": [0, 5], "__register_common_capi": 0, "interfac": [0, 2, 23, 26, 40], "embed": [0, 37], "after": [0, 5, 10, 27, 29, 34, 35, 38, 40], "load": [0, 10, 24, 31, 34, 37, 39], "call": [0, 9, 10, 33, 35, 36, 37, 43], "cifac": 0, "find_user_lib_path": 0, "asynchron": [0, 10, 28, 37], "f": [0, 10, 28, 38], "replac": [0, 7, 10, 36, 41], "sync": [0, 10, 35, 39], "async": [0, 1, 10, 29, 31], "us": [0, 5, 9, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 41, 43, 44], "aysncio": 0, "pool": [0, 31], "synchron": [0, 10, 29, 31], "run": [0, 10, 27, 28, 30, 31, 34, 35, 36, 37, 38, 39, 40], "place": 0, "result": [0, 10, 26, 29, 30, 31, 32, 34, 35, 39], "furiosaversioninfo": 0, "version": [0, 10, 11, 24, 27, 28, 34, 35, 37, 38, 40], "eprint": 0, "arg": [0, 34, 38], "kwarg": 0, "get_sdk_vers": 0, "git": [0, 39, 40, 43], "commit": [0, 28, 29, 30, 38], "hash": [0, 38], "repres": [0, 9, 10], "current": [0, 10, 25, 33, 34, 36, 37, 38, 40], "applic": [0, 23, 31, 33, 35, 37, 38, 39, 40, 43], "removeprefix": 0, "word": 0, "prefix": 0, "9": [0, 9, 10, 24, 28, 30, 31, 34, 36, 39, 43], "see": [0, 10, 24, 29, 32, 34, 35, 38, 40], "doc": 0, "org": 0, "stdtype": 0, "html": [0, 34], "sdk": [0, 10, 24, 26, 27, 28, 34, 35, 38, 39, 41, 42, 43, 44], "abc": 1, "read": [1, 10, 35, 38, 40], "download": [1, 36, 40], "variou": [1, 26, 30, 32, 36, 37, 38, 40, 41, 43], "sourc": [1, 35, 36, 38, 39, 40, 43], "properti": [1, 10, 30, 31, 36], "cache_directori": 1, "cach": 1, "directori": [1, 10, 30, 33, 35, 38, 39], "save": [1, 24, 28, 30, 34, 35, 39, 43], "abstract": [1, 30, 31, 32], "uri": 1, "destin": 1, "paramet": [1, 9, 10, 28, 30], "data": [1, 5, 9, 10, 30, 31, 32, 34, 35, 37, 38, 39, 41, 43], "name": [1, 5, 9, 10, 11, 27, 28, 30, 31, 32, 34, 35, 36, 38, 39, 40, 43], "locat": [1, 29, 31, 33, 35], "static": [1, 5, 24, 28, 33], "is_support": 1, "decid": [1, 32], "whether": [1, 9, 10, 30, 32, 34, 38, 40], "support": [1, 23, 25, 26, 29, 31, 33, 34, 35, 41, 43], "specifi": [1, 9, 10, 30, 31, 32, 34, 35, 37, 39, 40, 43], "model": [1, 2, 4, 5, 6, 7, 8, 9, 23, 24, 26, 27, 29, 30, 32, 39, 42, 44], "dispatch": 1, "method": [1, 9, 10, 37, 39, 40, 41, 43], "signatur": 1, "transporterror": 1, "msg": [1, 11], "transportnotfound": 1, "list": [1, 2, 4, 5, 6, 7, 9, 10, 25, 28, 29, 30, 31, 35, 36, 37], "filetransport": 1, "check": [1, 5, 7, 24, 28, 29, 34, 36, 38, 40, 43], "ha": [1, 5, 10, 26, 29, 30, 31, 32, 34, 38], "valid": [1, 10, 29, 34, 35], "scheme": [1, 26], "e": [1, 10, 29, 34, 36, 38], "g": [1, 10, 29, 34], "just": [1, 10, 30, 43], "copi": [1, 10, 43], "githubtransport": 1, "header": [1, 33], "dict": [1, 5, 9, 10, 29], "param": 1, "httptransport": 1, "repositori": [1, 20, 23, 29, 30, 31, 32, 40], "url": [1, 31, 35, 43], "com": [1, 6, 7, 28, 36, 38, 39, 40, 43], "pars": 1, "tupl": [1, 9, 10], "ai": [1, 10, 24, 25, 28, 36, 38, 39, 40, 43], "main": [1, 10, 11, 12, 23, 28], "s3transport": 1, "network": [1, 36, 41], "util": [1, 3, 4, 9, 11, 23, 26, 29, 31, 32, 34, 35, 37, 40], "fetch": 1, "which": [1, 10, 26, 31, 34, 35, 37, 38, 39], "rais": [1, 10], "all": [1, 5, 10, 26, 28, 31, 34, 35, 40, 43], "avail": [1, 10, 31, 34, 36, 37, 38, 40], "ar": [1, 5, 10, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 43], "is_rel": 1, "rel": [1, 33], "we": [1, 25, 28, 32, 33, 36, 37, 40], "assum": [1, 6, 10], "": [1, 5, 7, 9, 10, 26, 28, 32, 34, 35, 37, 38, 39, 40, 41], "you": [1, 10, 24, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "should": [1, 10, 24, 26, 30, 34], "iter": [1, 5, 6, 9, 10], "frontend": [2, 23], "onnx": [2, 3, 9, 10, 23, 24, 26, 27, 28, 29, 30, 31, 34, 35, 37, 41, 43], "submodul": [2, 3, 4, 10, 12, 23], "transform": [2, 3, 4, 7, 23], "furiosaai": [2, 9, 24, 27, 28, 29, 31, 33, 34, 35, 36, 38, 43], "optimize_model": [2, 4, 32], "modelproto": [2, 4, 5, 6, 7, 9], "input_shap": [2, 4, 5, 7, 31, 39], "map": [2, 4, 5, 7, 9, 41], "opset_vers": [2, 4, 7], "13": [2, 4, 5, 7, 28, 35, 41], "convert_negative_pads_to_slic": [3, 4], "convert_prelu_to_relu": [3, 4], "fuse_batchnorm": [3, 4], "fuse_gather_matmul": [3, 4], "infer_squeeze_ax": [3, 4], "polish_model": [3, 4], "check_model": [3, 4], "inference_shap": [3, 4], "version_check": [3, 4], "experiment": [4, 5, 30, 35], "eliminate_detection_postprocess": [4, 5], "embedding_bag_port": [4, 5], "fuse_div_for_bert": [4, 5], "reify_conv_for_bert": [4, 5], "convertnegativepadstoslic": 5, "pattern_1": 5, "name_nod": [5, 6], "true": [5, 6, 7, 10, 38, 43], "onnxtransform": [5, 6], "prev": 5, "pad": [5, 27, 30], "next": [5, 33, 38, 40], "slice": [5, 27, 29, 30], "input": [5, 6, 9, 23, 24, 28, 30, 31, 34, 35, 37, 39, 41, 43], "an": [5, 7, 9, 24, 26, 28, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 43], "initi": [5, 9, 10, 24, 30, 43], "least": [5, 35], "one": [5, 9, 10, 35, 37, 38, 39, 40], "neg": 5, "sum": 5, "axi": [5, 10, 27, 29, 30], "doe": [5, 10, 30, 40, 43], "exce": 5, "correspond": [5, 10], "invalid": [5, 35], "absolut": 5, "each": [5, 10, 26, 30, 31, 32, 34, 37, 38, 39], "less": 5, "than": [5, 10, 27, 30, 32, 33, 40], "lead": [5, 32, 35], "nan": 5, "output": [5, 9, 23, 24, 28, 30, 31, 32, 34, 35, 37, 38, 39, 41], "make_new_init_and_vi": 5, "matched_nod": 5, "nodeproto": [5, 6], "valueinfoproto": 5, "tensorproto": 5, "make_new_nod": 5, "pattern_condition_check": [5, 6], "nodes_to_check": [5, 6], "pattern_match": [5, 6], "base_nod": [5, 6], "pattern_to_match": 5, "pattern_2": 5, "zero": [5, 10], "convertprelutorelu": 5, "prelu": [5, 31], "x": [5, 10, 38], "slope": 5, "relu": [5, 27], "make_new_init": 5, "make_new_vi": 5, "fusebatchnorm": 5, "conv": [5, 6, 27, 30, 31], "batchnorm": [5, 27, 28, 30], "convtranspos": [5, 27, 28, 31], "pattern_3": 5, "mul": [5, 27, 31], "add": [5, 6, 27, 28, 29, 31, 34, 36, 37, 38], "onli": [5, 6, 10, 25, 27, 28, 29, 30, 33, 35, 37, 38], "pattern_4": 5, "op_typ": 5, "fusegathermatmul": 5, "gather": [5, 30, 31], "matmul": [5, 6, 27, 30, 31], "must": [5, 33, 36, 37, 38, 39], "have": [5, 9, 10, 24, 28, 30, 31, 33, 34, 35, 36, 38, 40], "exactli": 5, "defin": [5, 7, 10], "graph": [5, 6, 34, 35, 41], "weight": [5, 27, 30, 37, 41], "data_typ": 5, "float": [5, 9, 10, 26, 37, 41], "rank": 5, "infersqueezeax": 5, "squeez": [5, 30], "ax": [5, 10, 30], "attribut": 5, "fill": [5, 10], "opset": [5, 7, 28, 30, 35], "exist": [5, 10, 28, 32, 40], "shape": [5, 10, 24, 28, 30, 37, 39, 43], "infer": [5, 25, 26, 29, 30, 31, 34, 35, 37, 39, 40, 43, 44], "polishmodel": 5, "essenti": 5, "check_value_info": 5, "eliminate_initializer_from_graph_input": 5, "eliminate_unused_initi": 5, "elimin": [5, 26, 32], "everi": [5, 34], "node": [5, 6, 28, 37], "regardless": [5, 30], "ani": [5, 10, 30, 34, 40], "field": [5, 40], "thei": [5, 10, 27, 33, 37, 39], "eliminate_unused_input": 5, "eliminate_unused_output": 5, "eliminate_unused_proto": 5, "eliminate_unused_value_info": 5, "fix_batch_size_as_on": 5, "fix": [5, 28, 29, 31, 32], "batch_siz": [5, 6, 10, 24, 28, 30, 31, 35], "dim_param": 5, "given": [5, 10, 24, 30, 35, 36], "fixed_point": 5, "t": [5, 6, 8, 10, 30, 34], "get_attribut": 5, "attr": 5, "attributeproto": 5, "attr_nam": 5, "default": [5, 9, 10, 24, 28, 30, 33, 34, 35, 38, 39, 43], "get_node_attribut": 5, "get_node_input_nam": 5, "get_node_output_nam": 5, "is_op_typ": 5, "target_op_typ": 5, "make_initializer_name_uniqu": 5, "make_unhashables_uniqu": 5, "rebuild_model": 5, "new_nod": 5, "renam": [5, 29], "bridge_disconnected_nod": 5, "node_0": 5, "next_nod": 5, "new_input": 5, "For": [5, 10, 27, 32, 35, 36, 37, 38, 39, 40], "chang": [5, 10, 28, 34, 36, 41], "exampl": [5, 10, 24, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 43], "befor": [5, 28, 29, 30, 31, 32, 40], "node_1": 5, "bridg": 5, "assign": 5, "y": [5, 10, 30, 31, 32, 33, 34, 36, 38, 40], "build_optimized_model": 5, "check_runn": [5, 7], "copy_value_info": 5, "find_next_nod": 5, "find_prev_nod": 5, "node_input": 5, "get_data_node_input": 5, "get_init_node_input": 5, "get_initializer_arrai": 5, "get_map_valu": 5, "get_value_info_dtyp": 5, "tensor_nam": [5, 9], "get_value_info_shap": 5, "is_same_shap": 5, "input_1": 5, "input_2": 5, "pop_multiple_optimizer_map": 5, "pop_single_optimizer_map": 5, "transform_to_elimin": 5, "nodes_to_remov": 5, "design": [5, 28, 30, 31, 32, 34, 35, 36, 37, 38, 39], "previou": [5, 35], "them": [5, 10, 25, 26, 32, 37, 40], "b": [5, 34, 35, 38, 41], "remov": [5, 10, 29, 32], "A": [5, 9, 10, 30, 32, 34, 40], "d": [5, 10, 34, 36], "connect": [5, 10, 28, 33], "make": [5, 10, 38, 39, 40], "transform_to_fus": 5, "nodes_to_add": 5, "inits_to_add": 5, "vis_to_add": 5, "traverse_prev_nod": 5, "producer_map_kei": 5, "update_graph_field": 5, "update_multiple_initializer_map": 5, "update_multiple_optimizer_map": 5, "dest_nam": 5, "update_multiple_value_info_map": 5, "value_info": 5, "update_single_initializer_map": 5, "update_single_optimizer_map": 5, "update_single_value_info_map": 5, "eliminatessddetectionpostprocess": 6, "ssd_output": 6, "softmax": [6, 27, 30], "graph_output": 6, "nchw": [6, 10], "embeddingbagpattern": 6, "appli": [6, 10, 28, 38], "dlrm": 6, "export": [6, 10, 26, 35, 39], "embeddingbag": 6, "equival": [6, 10], "torch": 6, "simplifi": [6, 7], "doesn": [6, 10], "work": [6, 10, 34, 37], "becaus": [6, 10, 35], "can": [6, 9, 10, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "handl": [6, 10], "empti": [6, 10], "bag": 6, "pytorch": [6, 26, 29], "blob": 6, "55d479aca5e959c5f2fe3089e162db710bb7632d": 6, "symbolic_opset11": 6, "py": [6, 39], "l836": 6, "l902": 6, "check_condition_3": 6, "embeddingbagport": 6, "128": [6, 27, 29], "fusedivforbert": 6, "some": [6, 10, 24, 27, 28, 30, 35], "bert": 6, "transform_matmul_add": 6, "reifyconvforbert": 6, "well": [7, 10, 28, 29, 31], "execut": [7, 10, 29, 30, 31, 32, 34, 35, 37, 38, 39, 43], "onnxruntim": 7, "inferenceshap": 7, "former": 7, "onnx_simplifi": 7, "daquexian": 7, "checkvers": 7, "convert": [7, 9, 10, 11, 12, 23, 26, 29, 31, 32, 40, 41], "12": [7, 27, 28, 30, 31, 34, 38, 39], "opset_bound": 7, "calibrationmethod": [9, 32], "min_max_asym": [9, 32], "min": [9, 10, 32, 34, 39, 41], "max": [9, 10, 26, 32, 34, 39, 41], "asymmetr": [9, 32, 41], "min_max_sym": [9, 32], "symmetr": [9, 41], "entropy_asym": [9, 32], "entropi": [9, 32, 41], "aymmetr": 9, "entropy_sym": [9, 32], "percentile_asym": [9, 32], "percentil": [9, 10, 32, 34, 39, 41], "percentile_sym": [9, 32], "mse_asym": [9, 32], "mean": [9, 10, 27, 32, 34, 39], "squar": [9, 32], "error": [9, 23, 24, 32, 35, 38, 40, 43], "mse": [9, 41], "mse_sym": [9, 32], "sqnr_asym": [9, 32], "signal": [9, 10, 24, 32], "nois": [9, 32], "ratio": [9, 32], "sqnr": [9, 41], "sqnr_sym": [9, 32], "6": [9, 24, 27, 32, 34, 39], "7": [9, 32, 38], "4": [9, 10, 31, 32, 34, 39], "5": [9, 10, 24, 26, 29, 32, 34, 35, 39, 41], "8": [9, 10, 24, 26, 28, 30, 32, 34, 39, 40, 41, 43], "byte": [9, 10], "calibration_method": 9, "percentag": [9, 34], "99": [9, 10, 32, 34, 39, 41], "collect": [9, 10, 24, 34], "tensor": [9, 23, 24, 28, 29, 30, 31, 32, 37], "comput": [9, 26, 27, 34], "rang": [9, 10, 31, 32, 39], "collect_data": [9, 32], "calibration_dataset": 9, "sequenc": 9, "ndarrai": [9, 10, 29, 31], "multipl": [9, 10, 29, 31, 37, 38, 43], "time": [9, 10, 26, 29, 30, 31, 32, 34, 38, 39, 40], "numpi": [9, 10, 29, 31, 32, 39, 43], "provid": [9, 10, 26, 29, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 43], "compute_rang": [9, 32], "verbos": [9, 10, 34, 35], "fals": [9, 10], "estim": 9, "basi": 9, "show": [9, 30, 32, 35, 36, 38, 43], "progress": [9, 10], "bar": [9, 39], "dictionari": 9, "modeleditor": 9, "manipul": 9, "convert_input_typ": 9, "tensor_typ": 9, "tensortyp": 9, "element": [9, 10, 26, 32], "The": [9, 10, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "desir": [9, 10, 35, 38], "convert_output_typ": 9, "tensor_rang": 9, "option": [9, 10, 30, 31, 32, 33, 34, 35, 36, 38, 40, 43], "new": [9, 10, 32, 33], "origin": [9, 10, 41], "retain": [9, 10, 26], "enumer": [9, 10], "convert_": 9, "_type": 9, "int8": [9, 10, 26, 37, 41], "uint8": [9, 10, 28, 32, 39, 43], "get_output_nam": 9, "get_pure_input_nam": 9, "associ": [9, 10], "qunat": 9, "tensor_name_to_rang": 9, "its": [9, 10, 25, 30, 38, 40, 43], "serial": [9, 34], "incorpor": 9, "inform": [9, 24, 28, 34, 37, 38, 40], "high": [10, 26, 31, 37, 39], "level": [10, 29, 33, 35, 37, 43], "npu": [10, 24, 29, 31, 32, 33, 34, 37, 39, 40, 41, 43], "divid": [10, 31], "three": [10, 41], "wa": [10, 28, 32, 34], "first": [10, 26, 31, 34, 35, 37, 41, 43], "introduc": [10, 33, 37], "furiosart": [10, 33], "10": [10, 24, 28, 30, 31, 32, 34, 35, 38, 39, 40], "backward": 10, "compat": [10, 29, 31, 38, 40, 43], "slate": 10, "futur": [10, 33], "releas": [10, 33], "more": [10, 28, 29, 31, 32, 33, 35, 37, 38, 40, 43], "further": [10, 41], "contain": [10, 24, 28, 30, 34, 37, 38, 40], "two": [10, 26, 34, 35, 38], "differ": 10, "singl": [10, 26, 28, 31, 38], "activ": [10, 26, 30, 31, 38], "either": [10, 34, 36], "possibl": [10, 35], "task": [10, 26, 29, 37, 38, 39], "thread": [10, 23, 29, 31, 34, 39], "separ": [10, 37, 40], "send": [10, 43], "recv": [10, 36], "respect": [10, 33, 37], "pair": 10, "context": [10, 30], "distinguish": 10, "while": [10, 26, 31, 37], "sent": 10, "deprec": [10, 33], "sinc": [10, 30, 35, 40], "strongli": 10, "discourag": 10, "mani": [10, 31, 32, 37, 38, 39], "histor": 10, "includ": [10, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 38], "full": [10, 26, 29, 32, 34, 38], "As": [10, 30, 31, 32, 33, 41], "report": [10, 25, 28, 34, 35, 39], "larg": [10, 26, 29, 30, 31, 39], "wrapper": [10, 34], "around": 10, "slight": 10, "incompat": 10, "most": [10, 26, 35, 40], "notabl": 10, "lack": 10, "subclass": 10, "those": 10, "intercompat": 10, "mark": [10, 36], "mai": [10, 26, 27, 31, 32, 34, 35, 36, 38, 40], "enabl": [10, 28, 31, 34, 35, 36, 37], "extra": [10, 38], "instal": [10, 25, 28, 29, 30, 35, 37], "forc": 10, "old": 10, "implement": [10, 37], "disabl": [10, 35, 38], "pleas": [10, 24, 25, 28, 32, 34, 36, 43], "note": [10, 34], "With": [10, 26, 28, 31, 32, 37, 39, 40], "full_vers": 10, "nativeerror": 10, "modelsourc": 10, "how": [10, 24, 25, 32, 33, 34, 36, 44], "real": 10, "alia": 10, "like": [10, 43], "string": 10, "arrai": [10, 29], "__bytes__": 10, "raw": [10, 28, 38], "allow": [10, 29, 30, 31, 35, 37, 39, 40, 43], "addit": [10, 27, 31, 32, 34, 37, 38, 39, 40], "format": [10, 26, 30, 31, 32, 34, 35, 37, 38, 39], "restrict": [10, 36], "primari": 10, "represent": 10, "alias": 10, "document": [10, 34, 38], "purpos": [10, 35], "tensorarrai": 10, "itself": 10, "shouldn": 10, "requir": [10, 26, 29, 31, 32, 33, 37, 38, 41, 43], "alwai": [10, 30], "scalar": 10, "dimension": 10, "other": [10, 29, 30, 31, 33, 35, 38, 40, 44], "same": [10, 30, 31, 37, 43], "due": 10, "technic": 10, "reason": [10, 31], "concret": 10, "so": [10, 24, 26, 28, 30, 33, 34, 35, 37, 40, 41, 43], "item": [10, 30, 34, 38], "len": [10, 39], "is_empti": 10, "view": [10, 30, 32, 34, 39], "also": [10, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39], "intern": [10, 31], "were": 10, "here": [10, 33, 35, 36, 37, 40], "numpy_dtyp": 10, "dtype": [10, 28], "contrari": 10, "didn": 10, "mislead": 10, "thu": [10, 39], "instead": [10, 32], "copy_from": 10, "entir": [10, 28, 30], "what": 10, "refer": [10, 24, 27, 32, 33, 34, 35, 37, 38, 40, 41, 43], "memori": [10, 26, 27, 29, 32, 34, 35, 38], "region": 10, "independ": [10, 26, 28, 35], "creat": [10, 24, 28, 30, 31, 34, 35, 36, 37, 39, 43], "previous": [10, 28], "could": [10, 28, 32, 40], "directli": [10, 28, 35, 36, 37, 39, 40, 43], "via": [10, 32, 35, 37], "create_async": [10, 28, 30], "textual": 10, "section": [10, 24, 31, 34, 36, 37, 39, 40], "mainli": 10, "manag": [10, 30, 31, 32, 34, 35, 37, 38], "create_runn": [10, 35, 39], "await": [10, 29, 31], "act": 10, "scope": 10, "subsequ": [10, 34], "Its": [10, 43], "lifetim": [10, 35], "start": [10, 25, 32, 35, 38, 39, 40, 41, 43, 44], "successfulli": [10, 38], "end": [10, 25], "close": [10, 30], "fail": [10, 24, 28], "been": [10, 29, 30, 31, 32, 38, 40], "yet": [10, 30], "wait": [10, 31, 43], "until": [10, 31], "inde": 10, "take": [10, 30, 35, 36, 39], "too": [10, 30], "much": [10, 39], "case": [10, 28, 30], "open": [10, 38, 39], "timeout": [10, 28], "reach": 10, "made": 10, "identifi": [10, 28, 31, 37, 38, 39], "implicit": 10, "arch": [10, 36, 38], "denot": 10, "where": [10, 30], "target": [10, 34], "architectur": [10, 26, 38, 40], "warboi": [10, 25, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 41], "number": [10, 31, 34, 35, 37, 38], "pe": [10, 24, 26, 28, 35, 38, 43], "per": [10, 34], "omit": [10, 35, 38], "index": 10, "indic": 10, "z": 10, "fuse": [10, 26, 30, 35], "intermedi": 10, "occupi": [10, 31, 34], "worth": 10, "mode": [10, 27, 29, 30, 34], "determin": 10, "kernel": [10, 25, 29, 34, 36, 40], "driver": [10, 25, 29, 30, 31, 32, 33, 34, 38, 40, 43], "heavili": 10, "reli": 10, "upon": [10, 36], "especi": [10, 35], "npuxpei": 10, "dev": [10, 28, 31, 32, 33, 34, 36, 38], "aforement": 10, "alloc": [10, 32, 35, 37, 38], "greedi": 10, "manner": [10, 30, 37], "even": [10, 27, 30, 35, 39], "mix": 10, "recommend": [10, 38, 40], "explicit": 10, "furiosa_devic": 10, "preced": 10, "over": [10, 32, 34, 35], "npu_devnam": [10, 28, 38], "never": 10, "overrid": 10, "relev": 10, "affect": 10, "width": 10, "height": [10, 26, 27, 29], "channel": 10, "batch": [10, 26, 27, 30, 34], "unknown": 10, "constant": 10, "known": 10, "abbrevi": 10, "descript": [10, 34, 35, 38, 43], "w": [10, 28, 31, 32, 34, 38, 39], "h": [10, 34, 35], "depth": [10, 29], "convolut": [10, 26, 29], "n": [10, 32, 34, 35, 38, 39, 40], "privat": 10, "Their": 10, "stabl": [10, 29, 30, 36], "datatyp": [10, 43], "v": [10, 34, 35], "constructor": 10, "import": [10, 24, 28, 29, 31, 35, 39, 43], "np": [10, 28, 29, 31, 39, 43], "float32": [10, 30], "float16": 10, "bfloat16": 10, "int16": 10, "int32": 10, "int64": 10, "ieee": 10, "754": 10, "half": [10, 26], "precis": 10, "binary16": 10, "point": [10, 26, 37, 41], "binary32": 10, "bit": [10, 41], "sign": [10, 36], "integ": [10, 30, 43], "16": [10, 24, 26, 28, 32, 39], "32": [10, 24, 26, 28], "64": [10, 26], "unsign": 10, "valueerror": 10, "tensordesc": 10, "describ": [10, 37, 39, 40, 43], "ndim": 10, "dimens": [10, 24, 26, 28], "dim": 10, "idx": 10, "size": [10, 26, 30, 34, 39], "th": 10, "desc": [10, 32], "conceptu": 10, "stride": 10, "It": [10, 26, 29, 30, 33, 35, 37, 39, 40], "distanc": 10, "between": [10, 34, 35, 39, 40], "adjac": 10, "convent": 10, "length": [10, 26], "total": [10, 26, 31, 34, 39], "concaten": 10, "possibli": [10, 32], "input_num": 10, "output_num": 10, "summari": [10, 35, 39], "human": 10, "readabl": 10, "print": [10, 24, 30, 31, 34, 35, 38, 39, 43], "28": [10, 24, 28, 32, 34, 43], "3136": 10, "784": 10, "40": [10, 26, 32, 34, 38, 39], "print_summari": [10, 39], "simpl": [10, 30, 34], "worker_num": [10, 31], "worker": [10, 34, 38], "abov": [10, 24, 28, 33, 36, 37, 38, 39, 40, 43], "implicitli": 10, "couldn": 10, "get": [10, 29, 30, 31, 32, 33, 34, 36, 38, 40, 43], "argument": [10, 30, 32, 34, 35], "about": [10, 28, 31, 32, 34, 35, 37, 40, 43], "num_input": 10, "multipli": 10, "buffer": 10, "modif": [10, 38], "dure": [10, 28, 34, 35, 37, 39], "unexpect": 10, "ensur": [10, 29, 32, 40], "do": [10, 24, 37, 38, 39, 40, 43], "crash": 10, "top": [10, 26, 32, 39], "unspecifi": 10, "subject": 10, "garbag": 10, "through": [10, 28, 30, 31, 34, 35, 36, 37, 38, 39, 41, 43], "compiler_hint": 10, "need": [10, 26, 34, 35, 36, 37, 38, 40, 43], "hint": 10, "now": [10, 28, 30, 31], "keyword": 10, "posit": 10, "accept": [10, 43], "behav": 10, "ident": 10, "against": 10, "oper": [10, 25, 26, 28, 29, 30, 31, 32, 35, 36, 37, 38, 39, 41], "longer": [10, 31], "prefer": 10, "unlik": 10, "unless": 10, "alter": 10, "run_with": [10, 28], "order": [10, 28, 33, 36, 38, 40, 41], "partial": 10, "present": [10, 34, 38], "howev": [10, 35], "few": [10, 35], "submitt": 10, "These": [10, 36, 39], "actual": [10, 35, 37, 39], "process": [10, 26, 27, 29, 30, 32, 34, 35, 37, 38, 39, 40, 41, 43], "hold": 10, "complet": [10, 31, 33, 43], "both": [10, 34, 36], "configur": [10, 26, 28, 31, 33, 34, 35], "finit": 10, "quickli": [10, 31, 34], "block": [10, 31, 37], "create_queu": 10, "input_queue_s": 10, "output_queue_s": 10, "below": [10, 24, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41], "asyncio": [10, 28, 31], "taskgroup": 10, "tg": 10, "create_task": 10, "submit_task": 10, "recv_task": 10, "_": [10, 31, 32, 39], "fresh": 10, "suitabl": 10, "content": [10, 23, 38], "probabl": 10, "yourself": [10, 24, 34], "friendli": 10, "wai": [10, 30, 35], "whenev": 10, "appropri": [10, 30, 35], "immedi": [10, 28, 38, 39], "would": [10, 24, 27, 35], "later": 10, "uuid": [10, 31, 32, 34], "long": [10, 35, 37, 39], "logic": [10, 29], "leak": 10, "remain": [10, 39], "nevertheless": 10, "num_output": 10, "remark": 10, "finish": [10, 25, 32, 34, 41, 44], "handle_output": 10, "usag": [10, 29, 31, 32, 34, 35, 37, 38, 40, 43], "strictli": 10, "alreadi": [10, 30, 34, 36, 38], "loop": 10, "manual": 10, "guarante": 10, "onc": [10, 31, 33, 34, 35, 38, 39, 40, 43], "submiss": 10, "put": 10, "recov": 10, "parallel": 10, "fine": 10, "benefit": 10, "On": [10, 26], "hand": 10, "eventu": 10, "up": [10, 30, 31, 38, 39, 43], "idiomat": 10, "try": [10, 37, 38], "timeouterror": 10, "Not": 10, "built": [10, 24, 26, 28, 34, 43], "out": [10, 34, 36, 40, 43], "notifi": 10, "imposs": 10, "second": [10, 34, 35, 38], "completionqueu": [10, 28], "standard": [10, 26], "asyncsess": 10, "unbound": 10, "In": [10, 26, 28, 31, 32, 33, 35, 36, 37, 38, 41], "facili": 10, "migrat": 10, "continu": 10, "larger": [10, 26, 27, 32, 35], "though": [10, 39], "spawn_thread_to_send_input": 10, "millisecond": 10, "queuewaittimeout": 10, "prone": 10, "potenti": 10, "incompatibil": 10, "basic": [10, 25, 38, 44], "facil": 10, "recordformat": [10, 31, 39], "record": [10, 31, 38, 39], "chrometrac": 10, "pandasdatafram": [10, 31, 39], "resourc": [10, 26, 35, 37, 38], "cpu": [10, 25, 27, 29, 35, 37, 38, 39, 44], "json": [10, 34, 38, 39, 43], "span": 10, "incur": 10, "non": 10, "trivial": 10, "perform": [10, 25, 29, 31, 32, 33, 34, 35, 37, 38, 41], "overhead": [10, 39], "trace": [10, 31, 34, 43], "written": [10, 39], "temporari": 10, "writter": 10, "depend": [10, 26, 29, 30, 31, 35, 38, 40], "chrome": [10, 34, 39], "csv": 10, "profilererorr": 10, "config": [10, 30, 34, 38, 43], "At": [10, 37], "enter": [10, 30, 32, 34, 39], "exit": [10, 43], "within": [10, 36, 37, 39], "child": 10, "profilerrecordobject": 10, "paus": 10, "temporarili": [10, 39], "stop": [10, 32, 34], "minim": [10, 41], "resum": 10, "state": [10, 24, 26, 28, 30, 36, 43], "noth": 10, "arbitrari": 10, "measur": [10, 32, 39], "event": [10, 39], "occur": 10, "interv": [10, 32, 34, 38], "detail": [10, 27, 28, 29, 30, 31, 32, 34, 35, 38, 40, 43], "get_pandas_datafram": [10, 31, 39], "datafram": [10, 31], "look": [10, 33], "trace_id": [10, 31, 39], "parent_span_id": 10, "span_id": [10, 39], "cat": [10, 30, 31, 34, 38], "id": [10, 31, 34, 39], "dram_bas": 10, "pe_index": [10, 39], "execution_index": [10, 39], "instruction_index": 10, "operator_index": [10, 39], "dur": [10, 31, 39], "6ffe9ac3080814bc134ae4c5e58269e0": 10, "0000000000000000": 10, "a61dd01a47ce8de": 10, "1690798389820453606": 10, "1690798390204660478": 10, "35": 10, "na": 10, "384206872": 10, "079f8437488528d5768780162ed59374": 10, "2d18b0e17e760325": 10, "1690798390205840825": 10, "1690798390267819096": 10, "programbinari": 10, "26": 10, "61978271": 10, "fb4610c2fd1be67e63e01ca9169b6fef": 10, "2a092524d04a4077": 10, "1690798390267849007": 10, "1690798390267857471": 10, "allocatedram": 10, "8464": 10, "009b425f06ca0065a64f0586d1a999b0": 10, "cdac229f8d8569d7": 10, "1690798389793627190": 10, "1690798390268011450": 10, "regist": [10, 25, 36, 38], "474384260": 10, "348ee82fdf97fad9f782cc12a58d447d": 10, "59b5a5d06439f9f1": 10, "1690798390270474367": 10, "1690798390270526470": 10, "enqueu": 10, "52103": 10, "27efb2c82a5ac93bed911142e9187c45": 10, "174b38c90d1f7a10": 10, "ff7c4f8798d75b63": 10, "1690798390270558295": 10, "1690798390270570293": 10, "feedinput": 10, "11998": 10, "panda": [10, 31], "get_pandas_dataframe_with_filt": 10, "column": 10, "row": 10, "whose": 10, "filter": 10, "get_cpu_pandas_datafram": 10, "categori": [10, 39], "get_npu_pandas_datafram": 10, "print_npu_oper": [10, 39], "termin": [10, 24, 28, 30, 40], "average_elaps": [10, 39], "count": [10, 34, 38, 39], "lowlevelconv2d": [10, 39], "5119": 10, "9375": 10, "lowleveldepthwiseconv2d": [10, 39], "1091": 10, "56": [10, 32, 34, 39], "lowlevelpad": [10, 39], "561": 10, "482143": 10, "lowlevelexpand": [10, 39], "lowlevelslic": [10, 39], "lowlevelreshap": [10, 39], "232": 10, "print_npu_execut": [10, 39], "execution_ind": 10, "iowait": [10, 39], "39ffc55ef7b2177": 10, "555899badb3f8e5": 10, "116971": 10, "105186": 10, "11785": 10, "5338e9fa2d1fb70": 10, "f1": 10, "9c8aa64bbb878e3": 10, "4e9a13e698f4fa1": 10, "117011": 10, "11825": 10, "b62194f8dec670": 10, "3c": 10, "0ce2a8ce2c591e3": 10, "5cd8a081758f41c": 10, "116961": 10, "105185": 10, "11776": 10, "4e92e0c421f3946": 10, "14": [10, 43], "a941ace17a2c5e6": 10, "a3726d0ebb2705c": 10, "116909": 10, "11723": 10, "15a8f05d8872fa9": 10, "print_external_oper": [10, 39], "extern": [10, 31], "7d65ff7ae5587d3345d5df5a77": 10, "53e3fb9c02964361": 10, "quantiz": [10, 23, 25, 26, 34, 35, 39], "175246": 10, "ebfaad": 10, "33371e09f89cfa06c41286df13": 10, "8d5a00c6e4e8c2c0": 10, "lower": [10, 31, 32, 33, 34, 35, 39], "183803": 10, "11a30f": 10, "9f7df939abc20da11431c18024": 10, "064dacd9a108c4a0": 10, "unlow": [10, 32, 39], "60459": 10, "c41af1": 10, "1bda703f4ffc878a4294ec6253": 10, "cb2f103208d2fa45": 10, "dequant": [10, 32, 39], "19468": 10, "3ac8d0": 10, "9f769c8951f39d98e6ee216e34": 10, "91c0bdd8c5b81327": 10, "85724": 10, "6bc7e5": 10, "048e5cab6d4d676e4e6b10e827": 10, "714834cb8dc59f4b": 10, "306893": 10, "6b5489": 10, "6bb481ca3b1eab843b795a7865": 10, "46d538d7b4c72d25": 10, "73313": 10, "49558b": 10, "e0f13a5fb0bf2942ed16171844": 10, "71a432e3e3dc55f6": 10, "37079": 10, "ccb293": 10, "c3b2fdba80f16f781e4b313af3": 10, "066e3916590edf38": 10, "67805": 10, "a571b6": 10, "4bebe5f61e84d502f5b5dc7d22": 10, "9dfb32069b2b5a98": 10, "310303": 10, "1e4f5a": 10, "b8cabf53ae39a4ad18144af26c": 10, "cb767fbdd718da89": 10, "72378": 10, "e136c9": 10, "e40956dda5ecc0a1774e39377b": 10, "090d9cbd5e60032a": 10, "33951": 10, "1ef245": 10, "3d13f40c0966940439adcce4c1": 10, "4702a924e4b6d38b": 10, "76999": 10, "9981a4": 10, "53746b998038e994a5e378f9a2": 10, "522b7a9e354de2b3": 10, "339339": 10, "8caa5a": 10, "76a2080bc0917db26b7313e29a": 10, "4b1b0bf55f344258": 10, "74708": 10, "81def3": 10, "4c0a04dc669b04416f18e781d6": 10, "8eb55fb2b618933a": 10, "33661": 10, "afc3c6": 10, "print_infer": [10, 39], "b5edc4d40493df2028d186d4073d5487": 10, "a61af3b9ad70b956": 10, "4430749": 10, "983e136f80e1c070dca3ad854f37cf97": 10, "f2dd4e899d52531d": 10, "4181392": 10, "dada8a5830272b5d255fda801568fc5": 10, "cda7127619be5c33": 10, "4275757": 10, "6ad054709f76095c86fba6dcd9254ca0": 10, "9d7f199a445003aa": 10, "4215571": 10, "overal": 10, "latenc": [10, 26, 31, 33, 34, 37, 39], "4275867": 10, "median": [10, 39], "4245664": 10, "90": [10, 32, 34, 39], "4384251": 10, "95": [10, 32, 34, 39], "4407500": 10, "97": [10, 32, 34, 39, 41], "4416800": 10, "4426099": 10, "4430284": 10, "export_chrome_trac": 10, "filenam": [10, 35, 43], "write": [10, 33, 35, 37, 39, 40], "furiosaruntimeerror": 10, "nativeexcept": 10, "room": 10, "upcom": 10, "restructur": 10, "furiosaruntimewarn": 10, "dedic": 10, "log": [10, 24, 28, 30, 34, 36, 38, 43], "furiosa_log_level": 10, "decreas": 10, "code": [10, 29, 30, 37, 38, 39, 40, 43], "major": [10, 26, 30, 37, 38, 43], "incompatibilit": 10, "behavior": 10, "expens": 10, "generate_compiler_log_path": 10, "current_npu_devic": 10, "is_compile_log_en": 10, "log_dir": 10, "furiosa_log_dir": 10, "store": [10, 30], "profiler_output": 10, "furiosa_profiler_output_path": [10, 39], "nux_profiler_path": 10, "being": [10, 24, 32, 33, 37, 43], "incompatiblemodel": 10, "compilationfail": 10, "internalerror": 10, "unsupportedtensortyp": 10, "unsupporteddatatyp": 10, "incompatibleapiclienterror": 10, "invalidyamlexcept": 10, "apiclientinitfail": 10, "noapikeyexcept": 10, "invalidsessionopt": 10, "sessiontermin": 10, "devicebusi": 10, "invalidinput": 10, "tensornamenotfound": 10, "unsupportedfeatur": 10, "invalidcompilerconfig": 10, "sessionclos": 10, "mostli": 10, "typeerror": 10, "reexport": 10, "elig": 10, "match": [10, 32, 40], "rand": [10, 31, 39], "random": [10, 28, 30, 34, 39], "meant": 10, "quick": [10, 38], "test": [10, 31, 34], "qualiti": [10, 29], "correct": 10, "correctli": 10, "cli": [11, 12, 23, 30, 31, 32, 35], "grpc": [11, 12, 23, 28, 37, 40, 43], "servic": [11, 12, 23, 24, 28, 31, 34, 37, 38], "rest": [11, 12, 23, 28, 37, 40, 43], "app": [11, 12, 20, 23, 31, 34], "endpoint": [11, 12, 23], "handler": [11, 23], "predict": [11, 23, 29, 31, 37], "model_repositori": [11, 23], "namespac": [11, 23, 38], "loader": [11, 23], "modelnotfound": 11, "modelservererror": 11, "model_repository_pb2": [12, 14], "model_repository_pb2_grpc": [12, 14], "predict_pb2": [12, 14], "predict_pb2_grpc": [12, 14], "health": [20, 23, 43], "processor": [20, 23], "imagenet": [20, 23, 29, 31], "welcom": 23, "common": [23, 34, 35, 41], "packag": [23, 29, 30, 31, 32, 33, 34, 35, 37, 38], "subpackag": 23, "modul": [23, 40], "nativ": [23, 24, 34, 37], "variant": 23, "legaci": 23, "devic": [23, 28, 31, 32, 34, 36, 37, 43], "specif": [23, 25, 31, 32, 38, 39, 40, 41, 43], "metadata": [23, 34, 37, 38, 43], "runner": [23, 35, 39], "queue": [23, 28], "profil": [23, 24, 26, 34], "diagnost": 23, "compil": [23, 24, 25, 26, 31, 40, 43], "const": 23, "env": [23, 39], "session": [23, 30, 31, 35, 37], "optim": [23, 26, 28, 30, 31, 32, 34, 37, 39, 40, 41], "calibr": [23, 30, 34, 41], "editor": 23, "server": [23, 25, 29, 30, 31, 32, 33, 34, 38], "registri": [23, 28], "serv": [23, 30, 32, 35, 37, 40], "telemetri": 23, "encount": [24, 32, 40], "unresolv": 24, "issu": [24, 28, 34, 36, 39, 40], "custom": [24, 25, 28, 34, 43], "center": [24, 25, 34, 36], "reproduc": 24, "screenshot": 24, "By": [24, 28, 34, 35, 36, 38], "when": [24, 27, 28, 30, 31, 32, 33, 34, 35, 37, 38, 39, 40, 43], "happen": 24, "furiosa": [24, 25, 27, 30, 33, 37, 38, 39, 40, 41, 43, 44], "dump": [24, 28, 34], "home": [24, 25, 28, 30, 35], "20211121223028": [24, 28, 30], "l5w4g6": [24, 28, 30], "rev": [24, 28, 34, 43], "407c0c51f": [24, 28], "modifi": [24, 30, 38], "2021": [24, 28, 34], "11": [24, 28, 30, 32, 34, 43], "18": [24, 28], "22": [24, 28, 36], "34": [24, 28, 34], "22t06": [24, 28], "30": [24, 28, 35], "392114z": 24, "npu0pe0": [24, 28, 31, 34, 38, 43], "397757z": 24, "nuxinn": [24, 43], "peid": [24, 43], "dfg": [24, 34, 35], "423026z": 24, "receiv": [24, 30, 41, 43], "423371z": [24, 28], "unsupport": [24, 28], "dimparam": [24, 28], "sep": [24, 28], "42": [24, 28], "gcc": [24, 28, 33], "libnux": [24, 28, 29, 30, 31, 32, 33, 36], "problem": [24, 28, 34, 39], "atlassian": [24, 28], "net": [24, 28], "servicedesk": [24, 28], "portal": [24, 28], "shown": [24, 28, 29, 30, 31, 34, 35, 38, 39, 40, 41], "instruct": [24, 28, 29, 30, 31, 32, 33, 34, 37, 41], "necessari": [24, 34, 35, 36, 40], "__full_version__": 24, "lib64": 24, "explain": [25, 33, 34, 36, 37, 38, 40], "compon": [25, 36, 37, 38], "firmwar": [25, 29, 30, 31, 32, 33, 34, 40, 43], "command": [25, 30, 31, 32, 35, 36, 37, 38, 40, 41, 43], "line": [25, 30, 31, 32, 35, 37, 38, 39, 40, 41], "tool": [25, 26, 35, 37, 38, 39, 40, 41], "offer": 25, "user": [25, 26, 29, 30, 31, 34, 35, 37, 39], "who": [25, 36], "earli": 25, "access": [25, 31, 36, 37, 39], "program": [25, 31, 33, 34, 35, 36, 37, 40], "eap": 25, "agre": 25, "licenc": 25, "agreement": 25, "eula": 25, "contact": [25, 36], "learn": [25, 26, 43], "introduct": [25, 38], "hw": 25, "acceler": [25, 26, 28, 29, 31, 32, 35, 37, 40, 41], "sw": [25, 36], "stack": 25, "guid": [25, 37, 38], "zoo": [25, 42], "kubernet": [25, 30, 43], "advanc": [25, 43, 44], "compar": [25, 41, 44], "accuraci": [25, 26, 31, 32, 44], "imag": [25, 26, 29, 31, 32, 38, 39, 40, 44], "classif": [25, 26, 44], "ssd": [25, 31, 35, 37, 41, 43, 44], "detect": [25, 26, 31, 38, 44], "framework": [25, 35, 37, 40], "bug": [25, 28, 29, 31, 32, 34], "chip": 26, "deep": 26, "demonstr": [26, 37], "maintain": [26, 36], "cost": [26, 35], "effici": [26, 35, 37], "low": [26, 37, 39], "request": [26, 31, 36, 38, 43], "maxim": [26, 35], "achiev": 26, "abl": [26, 36, 39, 43], "cnn": 26, "therebi": [26, 29, 35, 36, 37, 39], "bottleneck": [26, 31, 39], "energi": 26, "kei": [26, 28, 36, 37], "vision": [26, 28, 31, 37], "ocr": 26, "super": 26, "resolut": 26, "particular": 26, "superior": 26, "depthwis": 26, "group": [26, 27, 29, 30], "drive": 26, "art": 26, "deliv": 26, "32mb": 26, "sram": 26, "consist": [26, 30, 40], "deploi": [26, 28, 41], "respons": [26, 29, 37, 43], "speed": 26, "undergo": [26, 37], "fusion": [26, 28, 38], "aggreg": 26, "throughput": [26, 31, 34, 37, 39], "softwar": [26, 34, 40], "tensorflow": [26, 27, 41], "post": [26, 29, 30, 41, 43], "train": [26, 31, 32, 40, 41], "commonli": [26, 37, 39], "tensorflowlit": 26, "billion": 26, "transistor": 26, "180mm": 26, "clock": 26, "2ghz": 26, "peak": 26, "maximum": 26, "4266": 26, "lpddr4x": 26, "dram": [26, 35], "bandwidth": 26, "66gb": 26, "pcie": 26, "gen4": 26, "8x": 26, "mb": 26, "host": [26, 35, 43], "lane": 26, "form": [26, 40], "factor": [26, 39], "fhhl": 26, "hhhl": 26, "thermal": 26, "solut": 26, "passiv": 26, "fan": 26, "tdp": 26, "60w": 26, "temperatur": [26, 34, 38], "50": 26, "ghz": 26, "ddr": 26, "mbp": 26, "gb": 26, "66": 26, "submit": 26, "mlcommon": 26, "mlperf": [26, 35], "edg": [26, 35], "v2": [26, 34, 35, 43], "v1": [26, 31, 35, 38, 41], "lite": [27, 41], "certain": [27, 30, 37, 39], "condit": [27, 30, 37], "met": [27, 34], "split": [27, 34, 37], "sever": [27, 31, 35, 38, 43], "suffici": 27, "averagepool": 27, "clip": 27, "concat": [27, 29, 30], "dilat": [27, 30], "depthtospac": [27, 30], "exp": [27, 30], "expand": [27, 31], "flatten": [27, 43], "gemm": [27, 31], "leakyrelu": [27, 28], "lpnormal": 27, "p": [27, 31], "maxpool": [27, 31], "reducel2": 27, "reducesum": 27, "reshap": [27, 43], "pow": [27, 28, 30], "spacetodepth": [27, 30], "crd": [27, 29], "higher": [27, 29, 33, 35, 36, 38, 40, 43], "sigmoid": 27, "softplu": 27, "sub": [27, 28, 30, 31, 34], "sqrt": [27, 28], "transpos": [27, 29], "unsqueez": 27, "approxim": [28, 29, 30], "87": 28, "ad": [28, 29, 30, 32, 34, 41], "addition": [28, 37, 43], "b1d2b74": 28, "wherea": 28, "explicitli": [28, 35], "done": [28, 31, 34, 36], "sess1": 28, "model1": 28, "sess2": 28, "model2": 28, "npu0pe1": [28, 31, 43], "async_sess": 28, "npu1pe2": 28, "4f1f114": 28, "limit": [28, 37, 38], "pass": [28, 30, 34, 35], "np1": 28, "randint": 28, "255": [28, 30], "session_input": 28, "np2": 28, "output1": 28, "input2": 28, "input1": 28, "debug": [28, 43], "easier": 28, "mnist": [28, 31, 34, 43], "duplic": 28, "01aaa40": 28, "21cba85": 28, "hang": 28, "interrupt": 28, "a0f4bd7": 28, "easili": [28, 29, 37, 38, 40], "pip": [28, 29, 30, 31, 32, 34, 36, 40, 43], "mnistnet_uint8_quant_without_softmax": [28, 31, 43], "tflite": [28, 31, 34, 35, 37, 39, 43], "mlcommonsresnet50": [28, 31], "resnet50": [28, 31, 35, 37, 41], "sess": [28, 30, 31], "apt": [28, 29, 30, 31, 32, 33, 34, 35, 38, 40], "toolkit": [28, 38], "statu": [28, 38], "idl": [28, 34], "temp": [28, 31, 32, 34, 38], "power": [28, 31, 32, 34, 38], "pci": [28, 31, 32, 34, 38], "bdf": [28, 31, 32, 34, 38], "npu0": [28, 31, 38], "92": 28, "0000": [28, 31, 32, 34, 38], "01": [28, 38], "00": [28, 31, 32, 34, 38], "510": 28, "devnam": [28, 38], "readi": [28, 38, 43], "plugin": [28, 31, 32, 37], "label": [28, 37, 39], "schedul": [28, 32, 37, 38], "togeth": [28, 34, 37], "pod": [28, 37], "kubectl": [28, 38], "githubusercont": [28, 38], "deploy": [28, 38], "yaml": [28, 34, 38, 43], "234": 29, "pr": 29, "improv": [29, 32], "900": 29, "updat": [29, 30, 31, 32, 33, 38, 40], "pdma": [29, 30, 31, 32], "libnpu": [29, 34], "uninstal": 29, "space": [29, 39], "significantli": [29, 31], "reduc": [29, 31, 39], "frequenc": 29, "idempot": 29, "relianc": 29, "pylint": 29, "21": 29, "pyyaml": 29, "project": [29, 31], "structur": [29, 31, 39, 41], "litmu": 29, "accordingli": 29, "checker": 29, "busi": 29, "pre": [29, 31, 32, 40], "postprocess": [29, 31], "uploadfil": 29, "serveapi": 29, "servemodel": [29, 31], "asset": [29, 31, 39, 43], "image_classif": [29, 31], "def": [29, 31], "your": [29, 30, 35, 38, 39, 40, 43], "preprocess": [29, 31], "classifi": 29, "400": [30, 36], "toward": 30, "enhanc": [30, 31, 32], "hal": [30, 31, 32], "hardwar": [30, 31, 32, 41], "layer": [30, 31, 32, 37], "featur": [30, 31, 32, 33, 37, 43], "discoveri": [30, 31, 32], "furiosactl": [30, 31, 32, 38], "adopt": 30, "linear": 30, "nearest": 30, "resiz": 30, "dcr": 30, "chw": 30, "tanh": 30, "increas": [30, 32, 35, 41], "x12": 30, "gelu": 30, "erf": 30, "elu": 30, "binari": [30, 32, 35, 37], "reus": [30, 35], "redi": [30, 35], "storag": [30, 31, 35], "help": [30, 34, 35, 37, 43], "understand": [30, 39], "give": 30, "opportun": 30, "grep": [30, 38], "why": 30, "2022": 30, "05": [30, 32, 34, 43], "24t02": 30, "44": [30, 32, 34], "399402z": 30, "19": [30, 32, 34, 43], "logsoftmax": 30, "399407z": 30, "bigger": 30, "399408z": 30, "17": [30, 39], "36": 30, "had": [30, 32], "beta": [30, 38], "taken": 30, "step": [30, 34, 38, 39, 41], "shell": [30, 35, 39, 43], "wasn": 30, "effect": [30, 32, 35, 36], "without_quant": [30, 31, 32], "input_min": 30, "input_max": 30, "permut": 30, "compile_config": 30, "publish": 30, "nxcxhxw": 30, "nxcxd": 30, "subgraph": 30, "bia": 30, "qdq": 30, "operand": 30, "prevent": [30, 40], "relat": [30, 32], "reinforc": 30, "inspect": 30, "miss": 30, "decim": 30, "adjust": 30, "scale": [30, 37], "small": [30, 31, 39], "becom": 30, "latest": [30, 37, 38, 40], "select": [30, 36, 37, 38, 39, 40], "core": [31, 34, 38], "develop": [31, 32, 34, 35, 36, 39, 40], "upgrad": [31, 32, 40], "simpler": [31, 32], "wish": [31, 32, 36, 40], "setup": [31, 32, 34, 38, 43], "libhal": [31, 32, 36], "fastapi": 31, "web": [31, 39], "npuservemodel": 31, "npu_devic": [31, 43], "npu1pe0": [31, 32, 34], "shift": 31, "queri": [31, 34, 39], "frequent": [31, 35], "better": [31, 32], "self": 31, "complex": 31, "scenario": 31, "openvino": 31, "cloud": 31, "auth": [31, 36], "aioboto": 31, "densenet": 31, "93d63f654f0f192cc4ff5691be60fb9379e9d7fd": 31, "opentelemetri": 31, "collector": 31, "track": 31, "To": [31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 43], "get_trac": 31, "reset": [31, 35, 38], "tracer": 31, "start_as_current_span": 31, "__name__": 31, "input_tensor": 31, "output_tensor": 31, "furiosa_serving_otlp_endpoint": 31, "diagram": [31, 37, 41], "visual": [31, 38, 39], "grafana": [31, 38], "share": [31, 35, 39], "analyz": [31, 35, 37, 39], "analysi": [31, 34, 41], "df": [31, 39], "silu": 31, "usabl": 31, "div": 31, "auto_pad": 31, "newli": [31, 32, 40], "o": [31, 32, 34, 35, 38], "pid": [31, 32, 34], "cmd": [31, 34], "132529": [31, 34], "bin": [31, 34, 38, 39], "python3": [31, 34, 40], "uvicorn": [31, 34, 43], "image_classifi": [31, 34, 39], "uniqu": 31, "72212674": 31, "61be": 31, "4fca": 31, "a2c9": 31, "555e4ee67ab5": 31, "12180b0": 31, "49": [31, 32, 34, 43], "24": 31, "235": 31, "npu1": [31, 32, 34, 38], "df80fb54": 31, "8190": 31, "44bc": 31, "b9fb": 31, "664fa36c754a": 31, "54": [31, 32, 34], "53": 31, "6d": 31, "511": [31, 32, 34], "furioa": 31, "public": [31, 36], "yolov5": [31, 41], "medium": 31, "yolov5l": 31, "yolov5m": 31, "sota": 31, "explan": [31, 34, 36, 38], "nonblock": 31, "load_async": 31, "ouput": 31, "mobilenet": [31, 35, 37, 41], "resnet34": [31, 35, 41], "benchmark": [31, 35], "70": 31, "nativepostprocessor": 31, "postprocessor": 31, "jpg": [31, 39], "eval": 31, "changelog": 31, "partcular": 32, "signific": [32, 35], "setuptool": [32, 40], "wheel": [32, 40], "without": [32, 34, 35, 39, 40], "satisfi": [32, 37, 40], "impl": [32, 40], "No": [32, 40], "distribut": [32, 35, 36, 37, 38, 40], "flexibl": 32, "solid": 32, "optimized_onnx_model": 32, "source_onnx_model": 32, "calibration_data": 32, "tqdm": 32, "calibration_dataload": 32, "unit": [32, 34], "mininterv": 32, "quantizated_graph": 32, "begin": [32, 38], "with_quant": 32, "normalized_pixel_output": 32, "fp32": [32, 41], "pixel": 32, "quasisymmetr": 32, "popular": [32, 37], "among": [32, 40], "efficientnet": [32, 37, 41], "b0": [32, 41], "57": [32, 34], "452": 32, "104": 32, "contrast": 32, "73": [32, 41], "556": [32, 41], "experi": 32, "tutori": [32, 37, 40], "unnecessari": 32, "200": [32, 34, 43], "under": [32, 34, 38], "ctrl": [32, 34, 43], "datetim": [32, 34], "comp": [32, 34], "2023": [32, 34, 35, 43], "03": [32, 34], "21t09": [32, 34], "45": [32, 34], "699483936z": [32, 34], "152616": [32, 34], "06": [32, 34], "100": [32, 34, 41], "npu_runtime_test": [32, 34], "10000": [32, 34], "resnet": [32, 34], "ctc_kor1_200_nightly3_128dpes_8batch": [32, 34], "enf": [32, 34], "906443888z": [32, 34], "51": [32, 34], "09": [32, 34], "93": [32, 34, 41], "110489333z": [32, 34], "46": [32, 34], "98": [32, 34], "02": [32, 34], "316060982z": [32, 34], "43": [32, 34], "521140588z": [32, 34], "94": [32, 34, 41], "725910558z": [32, 34], "48": [32, 34], "07": [32, 34, 35], "935041998z": [32, 34], "47": [32, 34], "91": [32, 34], "58": [32, 34], "13929122z": [32, 34], "displai": 32, "concis": 32, "want": [32, 35, 36, 38, 39, 40, 43], "3c10fd3": [32, 34], "00000000": [32, 34], "000000000000": [32, 34], "wbyb0000000000000": [32, 34], "languag": [33, 37, 42], "abi": 33, "substitut": 33, "minimum": [33, 34], "ubuntu": [33, 36, 38, 40, 43], "20": [33, 34, 36, 38, 40, 41, 43], "04": [33, 34, 36, 38, 40, 43], "lt": [33, 36, 40, 43], "debian": [33, 36, 40, 43], "bullsey": [33, 36, 40, 43], "system": [33, 35, 36, 37, 38, 40], "administr": [33, 36], "privileg": [33, 36], "root": [33, 36, 38, 39, 43], "authent": 33, "x86_64": [33, 40], "linux": [33, 36, 37, 38], "gnu": 33, "simpli": [33, 43], "lnux": 33, "monitor": 34, "sudo": [34, 36, 40], "varieti": [34, 41], "subcommand": 34, "abil": 34, "obtain": [34, 41], "control": [34, 35, 37, 38], "recogn": [34, 37, 38], "consumpt": 34, "visibl": 34, "mount": [34, 37], "machin": [34, 35, 36, 38], "devfil": 34, "npu1pe1": 34, "calcul": 34, "m": [34, 41], "observ": 34, "carri": 34, "workload": [34, 37, 38], "tail": 34, "qp": 34, "expon": 34, "io": [34, 38], "durat": [34, 39], "min_query_count": 34, "min_dur": 34, "min_duration_m": 34, "orient": 34, "l": [34, 39, 41], "model_path": [34, 39], "produc": [34, 39], "1000": 34, "priorit": 34, "individu": 34, "stat": 34, "u": 34, "8399": 34, "307568": 34, "29040": 34, "50th": 34, "19329": 34, "95th": 34, "62797": 34, "99th": 34, "79874": 34, "model_data": 34, "md5": 34, "d7cd24a0a76cd492f31065301d468c3d": 34, "compiler_vers": 34, "2d862de8a": 34, "built_at": 34, "13t20": 34, "04z": 34, "hal_vers": 34, "nightli": 34, "230716": 34, "git_revis": 34, "fe6f77a": 34, "30025": 34, "num": 34, "33": 34, "31": 34, "8840": 34, "113254": 34, "29989": 34, "18861": 34, "64927": 34, "87052": 34, "9th": 34, "meta": 34, "pythonsdk": [34, 35], "extens": [34, 40], "prepar": 34, "verifi": 34, "fe1fca3": 34, "49b97492a": 34, "08": 34, "page": [34, 37], "normal": [34, 38], "goe": 34, "archiv": [34, 36], "step1": 34, "step2": 34, "step3": 34, "step4": 34, "skip": [34, 36, 40], "output_prefix": 34, "target_npu": [34, 35], "libfuriosa_h": [34, 43], "v0": [34, 35, 38, 43], "43c901f": [34, 43], "libfuriosa_compil": 34, "d7548b7f6": 34, "9ecebb6": 34, "famili": [34, 35, 38], "2pe": [34, 35], "09272794": 34, "ldfg": 34, "la": 34, "034934": 34, "140083": 34, "019548794": 34, "29": 34, "196825": 34, "cdfg": [34, 35], "001701888": 34, "gir": [34, 35], "015205072": 34, "lir": [34, 35], "0038304": 34, "020943863": 34, "331545": 34, "000001198": 34, "125": 34, "7448": 34, "unix_epoch": 34, "zip": 34, "zipinfo": 34, "1690438803": 34, "16904388032l4hoi3h": 34, "dot": [34, 35], "etc": [35, 36, 38], "analys": 35, "automat": [35, 37, 38], "But": 35, "f8f05c8ea": 35, "31t19": 35, "30z": 35, "ir": 35, "target_ir": 35, "dot_graph": 35, "analyze_memori": 35, "accler": 35, "choos": 35, "stand": 35, "foo": 35, "let": 35, "npu\ub294": 35, "sampl": [35, 43], "across": 35, "exceed": 35, "degrad": 35, "usual": 35, "trial": 35, "final": 35, "minut": 35, "Then": 35, "instantli": 35, "veri": 35, "coupl": 35, "remot": [35, 37], "fc_cache_en": 35, "fc_cache_store_url": 35, "rediss": 35, "ssl": 35, "tmp": 35, "cluster": [35, 37, 38], "password": [35, 36], "127": 35, "6379": 35, "25945": 35, "dai": 35, "hour": 35, "7200": 35, "sec": 35, "aliv": 35, "forev": 35, "expir": 35, "site": 36, "registr": 36, "evalu": 36, "focal": [36, 38], "fossa": 36, "internet": 36, "delin": 36, "ca": 36, "certif": 36, "gnupg": 36, "adv": 36, "keyserv": 36, "5f03afa423a751913f249259814f888b20b09a7": 36, "tee": 36, "conf": [36, 38], "null": 36, "eot": 36, "login": 36, "secret": 36, "chmod": 36, "tab": 36, "bookworm": 36, "deb": 36, "amd64": [36, 38], "jammi": 36, "multi": [36, 37], "owner": 36, "exclus": 36, "member": 36, "usermod": 36, "ag": [36, 38], "usernam": 36, "back": 36, "membership": 36, "libonnxruntim": 36, "held": 36, "showhold": 36, "outlin": 37, "role": 37, "guidelin": 37, "lowest": 37, "acknowledg": 37, "reinstal": [37, 40], "There": 37, "plai": 37, "dnn": [37, 39, 40, 43], "research": 37, "area": 37, "smaller": 37, "balanc": 37, "facilit": 37, "expos": 37, "protocol": 37, "horizont": 37, "platform": [37, 38, 43], "container": [37, 38], "enterpris": 37, "tenant": 37, "physic": 37, "particip": 37, "spec": [37, 38], "nodeselector": [37, 38], "nodeaffin": 37, "kuberent": 38, "awar": [38, 39], "simultan": 38, "equip": 38, "nfd": 38, "intel": 38, "recognit": 38, "again": [38, 40], "reboot": 38, "37": 38, "509": 38, "daemonset": 38, "jq": 38, "cpuid": 38, "adx": 38, "aesni": 38, "master": 38, "vendor": 38, "devicelabelfield": 38, "deviceclasswhitelist": 38, "mandatori": 38, "kube": 38, "alpha": 38, "1ed2": 38, "attach": 38, "TO": 38, "date": 38, "selector": 38, "128m": 38, "162m": 38, "tabl": [38, 41, 43], "product": 38, "warboyb0": 38, "renegad": 38, "manufactur": 38, "nmae": 38, "minor": 38, "patch": 38, "57ac7b0": 38, "build": 38, "node01": 38, "65d": 38, "node02": 38, "12d": 38, "apivers": 38, "kind": 38, "ghcr": 38, "search": 38, "comma": 38, "dir": 38, "kubelet": 38, "var": 38, "socket": 38, "permit": 38, "defaultp": 38, "disableddevic": 38, "nodenam": 38, "plugindir": 38, "socketnam": 38, "resourcenam": 38, "text": [38, 43], "configmap": 38, "reflect": 38, "edit": 38, "15": 38, "npu2": 38, "001": 38, "creation": 38, "eol": 38, "8gi": 38, "exec": 38, "bash": [38, 43], "echo": 38, "daemon": 38, "express": [38, 39], "prometheu": 38, "review": 38, "furiosa_npu_al": 38, "furiosa_npu_uptim": 38, "furiosa_npu_error": 38, "furiosa_npu_hw_temperatur": 38, "mc": 38, "furiosa_npu_hw_pow": 38, "instantan": 38, "\u00b5w": 38, "furiosa_npu_hw_voltag": 38, "instantaen": 38, "voltag": 38, "mv": 38, "furiosa_npu_hw_curr": 38, "ma": 38, "critic": 39, "ml": [39, 40], "engin": 39, "assist": 39, "browser": 39, "tempor": 39, "advantag": 39, "unchang": 39, "downsid": 39, "clone": [39, 40, 43], "cd": [39, 40, 43], "pwd": 39, "car": 39, "rw": 39, "r": 39, "456493": 39, "jul": 39, "27": 39, "address": [39, 43], "viewer": 39, "click": 39, "button": 39, "upper": 39, "left": 39, "corner": 39, "comparison": 39, "interact": 39, "interpret": 39, "jupyt": 39, "notebook": 39, "tree": 39, "quantized_model": [39, 43], "imagenet_224x224_mobilenet_v1_uint8_quant": 39, "trained_dm_1": 39, "0_without_softmax": 39, "mobilenet_v1_trac": 39, "warm": 39, "mnist_trac": 39, "job": 39, "huge": 39, "disk": 39, "difficult": 39, "hard": 39, "interest": 39, "avoid": 39, "exclud": 39, "phase": 39, "warmup": 39, "assert": 39, "1584494": 39, "3027309": 39, "2136984": 39, "1968066": 39, "2752525": 39, "2889917": 39, "2944874": 39, "2999831": 39, "3024561": 39, "7cf3d3b7439cf4c3fac1a47998783102": 39, "403ada67f1d8220": 39, "16d65f6f8f1db256d0f39953855dea72": 39, "78b065c19c3675ef": 39, "2111363": 39, "d0534e3a9f19edadab81954ad28ab44f": 39, "9a7addaf0f28c9f": 39, "1824769": 39, "70512188522f45b87cfe4f545de3cf2c": 39, "c75f697f8e72d333": 39, "elaps": 39, "8f6fce6c0e52b4735cae3379732a0943": 39, "3e1e4a76523cbf89": 39, "119145": 39, "108134": 39, "18446744073709540605": 39, "195366613b1da9b0350c0a3c2a608f42": 39, "07dff2e92172fabd": 39, "119363": 39, "18446744073709540387": 39, "3b65b8fa3eabfaf8f815ec9f41fcc7d9": 39, "639a366a7f932a23": 39, "119157": 39, "18446744073709540593": 39, "e48825df32a07e5559f7f50048c08e1f": 39, "ecaab4915bfda725": 39, "119219": 39, "18446744073709540531": 39, "5327": 39, "60": 39, "1412": 39, "285714": 39, "575": 39, "785714": 39, "lowleveltranspos": 39, "250": 39, "240": 39, "e7ab6656cc090a8d05992a9e4683b8b7": 39, "206a1d6f351ca4b1": 39, "136285": 39, "03636fd6c7dbc42f0a9dd29a7283d3fc": 39, "f636740983e095a6": 39, "133350": 39, "c9a0858f7e0885a976f51c6cb57d3e0f": 39, "bb6c84f88e453055": 39, "44775": 39, "8777c67ad9fe597139bbd6970362c2fc": 39, "63bac982c7b98aba": 39, "14682": 39, "98aeba2a25b0525166b6a4065ab01774": 39, "34ccd560571d733f": 39, "45465": 39, "420525dc13ba9624083e0a276f7ee718": 39, "9f6d342da5eb86bc": 39, "152748": 39, "cb67393f6949bbbb396053c1e00931ff": 39, "2d724fa6ab8ca024": 39, "67140": 39, "00424b4f02039ae0ca98388a964062b0": 39, "a5fb9fbd5bffe6a6": 39, "32388": 39, "d7412c59d360067e8b7a2508a30d1079": 39, "8e426d778fa95722": 39, "71736": 39, "6820acf9345c5b373c512f6cd5edcbc7": 39, "2d787c2df381f010": 39, "311310": 39, "84d24b02a95c63c3e40f7682384749e4": 39, "1236a974a619ff1a": 39, "51930": 39, "8d25dff1cfd6624509cbf95503e93382": 39, "673efb3bfb8deac6": 39, "12362": 39, "4cc60ec1eee7d9f3cdd290d07b303a18": 39, "e7903b0a584d6388": 39, "56736": 39, "c5f04d9fea26e5b52c6ec5e5406775fc": 39, "701118dabd065e6f": 39, "265447": 39, "c5fdfb9cf454da130148e8e364eeee93": 39, "5cf3750def19c6e8": 39, "35869": 39, "e1e650d23061140404915f1df36daf9c": 39, "ddd76ff19b5cd713": 39, "14688": 39, "direct": 39, "487": 39, "f3b158734e3684f2e043ed41309c4c2d": 39, "11204385": 39, "kit": 40, "ecosystem": 40, "wide": 40, "familiar": 40, "therefor": 40, "often": 40, "concern": 40, "conflict": 40, "easi": 40, "ye": 40, "question": 40, "sh": 40, "miniconda3": 40, "wget": 40, "repo": 40, "anaconda": 40, "miniconda": 40, "bashrc": 40, "deactiv": 40, "proper": 40, "upload": 40, "pypi": 40, "subdirectori": 40, "rather": 40, "adot": 40, "simul": 40, "fp16": 41, "techniqu": 41, "provi": 41, "greater": 41, "topolog": 41, "drop": 41, "fp": 41, "convnext": 41, "85": 41, "80": 41, "376": 41, "678": 41, "77": 41, "698": 41, "669": 41, "efficientnetv2": 41, "84": 41, "228": 41, "83": 41, "566": 41, "214": 41, "76": 41, "456": 41, "702": 41, "retinanet": 41, "3757": 41, "37373": 41, "476": 41, "23": 41, "23215": 41, "21626": 41, "108": 41, "yolox": 41, "497": 41, "48524": 41, "634": 41, "490": 41, "47443": 41, "96": 41, "822": 41, "454": 41, "43963": 41, "835": 41, "apu": 41, "8bit": 41, "link": 41, "bind": 43, "port": 43, "ip": 43, "listen": 43, "8080": 43, "zsh": 43, "fish": 43, "powershel": 43, "pwsh": 43, "model_config_list": 43, "mnistnet_uint8_qu": 43, "compiler_config": 43, "keep_unsigned": 43, "split_unit": 43, "ssd512_mobilenet_v2_bdd_int_without_reshap": 43, "model_config_exampl": 43, "20230509151914": 43, "axpfej": 43, "e626c458c": 43, "19t13": 43, "26z": 43, "09t06": 43, "560585z": 43, "565216z": 43, "d063sw": 43, "591795z": 43, "595298z": 43, "1184080": 43, "startup": 43, "press": 43, "quit": 43, "curl": 43, "mnist_imag": 43, "train_imag": 43, "60000": 43, "astyp": 43, "localhost": 43, "tolist": 43, "live": 43, "ok": 43, "respond": 43, "livenessprob": 43, "inferenc": 43, "readinessprob": 43, "model_nam": 43, "model_vers": 43}, "objects": {"furiosa": [[0, 0, 0, "-", "common"], [2, 0, 0, "-", "optimizer"], [9, 0, 0, "-", "quantizer"], [10, 0, 0, "-", "runtime"]], "furiosa.common": [[0, 0, 0, "-", "error"], [0, 2, 1, "", "get_sdk_version"], [0, 0, 0, "-", "native"], [0, 0, 0, "-", "thread"], [1, 0, 0, "-", "transport"], [0, 0, 0, "-", "utils"]], "furiosa.common.error": [[0, 1, 1, "", "FuriosaError"], [0, 2, 1, "", "is_err"], [0, 2, 1, "", "is_ok"]], "furiosa.common.native": [[0, 3, 1, "", "LogLevel"], [0, 2, 1, "", "find_global_lib_path"], [0, 2, 1, "", "find_native_lib_path"], [0, 2, 1, "", "find_native_libs"], [0, 2, 1, "", "find_user_lib_path"]], "furiosa.common.native.LogLevel": [[0, 4, 1, "", "ERROR"], [0, 4, 1, "", "INFO"], [0, 4, 1, "", "OFF"], [0, 4, 1, "", "WARN"]], "furiosa.common.thread": [[0, 2, 1, "", "asynchronous"], [0, 2, 1, "", "synchronous"]], "furiosa.common.transport": [[1, 3, 1, "", "FileTransport"], [1, 3, 1, "", "GithubTransport"], [1, 3, 1, "", "HTTPTransport"], [1, 3, 1, "", "S3Transport"], [1, 3, 1, "", "Transport"], [1, 0, 0, "-", "base"], [1, 2, 1, "", "download"], [1, 0, 0, "-", "error"], [1, 0, 0, "-", "file"], [1, 0, 0, "-", "github"], [1, 0, 0, "-", "http"], [1, 2, 1, "", "is_relative"], [1, 0, 0, "-", "s3"], [1, 2, 1, "", "supported"]], "furiosa.common.transport.FileTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.GithubTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.HTTPTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.S3Transport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.Transport": [[1, 6, 1, "", "cache_directory"], [1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"]], "furiosa.common.transport.base": [[1, 3, 1, "", "Transport"]], "furiosa.common.transport.base.Transport": [[1, 6, 1, "", "cache_directory"], [1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"]], "furiosa.common.transport.error": [[1, 1, 1, "", "TransportError"], [1, 1, 1, "", "TransportNotFound"]], "furiosa.common.transport.file": [[1, 3, 1, "", "FileTransport"]], "furiosa.common.transport.file.FileTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.github": [[1, 3, 1, "", "GithubTransport"]], "furiosa.common.transport.github.GithubTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.http": [[1, 3, 1, "", "HTTPTransport"]], "furiosa.common.transport.http.HTTPTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.s3": [[1, 3, 1, "", "S3Transport"]], "furiosa.common.transport.s3.S3Transport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.utils": [[0, 3, 1, "", "FuriosaVersionInfo"], [0, 2, 1, "", "eprint"], [0, 2, 1, "", "get_sdk_version"], [0, 2, 1, "", "removeprefix"]], "furiosa.optimizer": [[3, 0, 0, "-", "frontend"], [8, 0, 0, "-", "interfaces"], [2, 2, 1, "", "optimize_model"]], "furiosa.optimizer.frontend": [[4, 0, 0, "-", "onnx"]], "furiosa.optimizer.frontend.onnx": [[4, 2, 1, "", "optimize_model"], [5, 0, 0, "-", "transformer"], [7, 0, 0, "-", "utils"]], "furiosa.optimizer.frontend.onnx.transformer": [[5, 3, 1, "", "ONNXTransformer"], [5, 0, 0, "-", "convert_negative_pads_to_slice"], [5, 0, 0, "-", "convert_prelu_to_relu"], [6, 0, 0, "-", "experimental"], [5, 0, 0, "-", "fuse_batchnorm"], [5, 0, 0, "-", "fuse_gather_matmul"], [5, 0, 0, "-", "infer_squeeze_axes"], [5, 0, 0, "-", "polish_model"], [5, 0, 0, "-", "utils"]], "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer": [[5, 5, 1, "", "bridge_disconnected_nodes"], [5, 5, 1, "", "build_optimized_model"], [5, 4, 1, "", "check_runnable"], [5, 5, 1, "", "copy_value_info"], [5, 5, 1, "", "find_next_node"], [5, 5, 1, "", "find_prev_node"], [5, 5, 1, "", "get_data_node_input"], [5, 5, 1, "", "get_init_node_input"], [5, 5, 1, "", "get_initializer_array"], [5, 5, 1, "", "get_map_values"], [5, 5, 1, "", "get_value_info_dtype"], [5, 5, 1, "", "get_value_info_shape"], [5, 5, 1, "", "is_same_shape"], [5, 5, 1, "", "pattern_matcher"], [5, 5, 1, "", "pattern_matching"], [5, 5, 1, "", "pop_multiple_optimizer_map"], [5, 5, 1, "", "pop_single_optimizer_map"], [5, 5, 1, "", "transform"], [5, 5, 1, "", "transform_to_eliminate"], [5, 5, 1, "", "transform_to_fuse"], [5, 5, 1, "", "traverse_prev_node"], [5, 5, 1, "", "update_graph_fields"], [5, 5, 1, "", "update_multiple_initializer_map"], [5, 5, 1, "", "update_multiple_optimizer_map"], [5, 5, 1, "", "update_multiple_value_info_map"], [5, 5, 1, "", "update_single_initializer_map"], [5, 5, 1, "", "update_single_optimizer_map"], [5, 5, 1, "", "update_single_value_info_map"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice": [[5, 3, 1, "", "ConvertNegativePadsToSlice"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1": [[5, 5, 1, "", "make_new_init_and_vi"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2": [[5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu": [[5, 3, 1, "", "ConvertPReluToRelu"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.experimental": [[6, 0, 0, "-", "eliminate_detection_postprocess"], [6, 0, 0, "-", "embedding_bag_porting"], [6, 0, 0, "-", "fuse_div_for_bert"], [6, 0, 0, "-", "reify_conv_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess": [[6, 3, 1, "", "EliminateSSDDetectionPostprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess": [[6, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting": [[6, 3, 1, "", "EmbeddingBagPattern"], [6, 3, 1, "", "EmbeddingBagPorting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern": [[6, 5, 1, "", "check_condition_3"], [6, 5, 1, "", "pattern_condition_checker"], [6, 5, 1, "", "pattern_matching"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting": [[6, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert": [[6, 3, 1, "", "FuseDivForBert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert": [[6, 5, 1, "", "transform"], [6, 5, 1, "", "transform_matmul_add"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert": [[6, 3, 1, "", "ReifyConvForBert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert": [[6, 5, 1, "", "transform"], [6, 5, 1, "", "transform_matmul_add"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm": [[5, 3, 1, "", "FuseBatchNorm"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"], [5, 3, 1, "", "Pattern_3"], [5, 3, 1, "", "Pattern_4"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1": [[5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2": [[5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3": [[5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul": [[5, 3, 1, "", "FuseGatherMatMul"], [5, 3, 1, "", "Pattern_1"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes": [[5, 3, 1, "", "InferSqueezeAxes"], [5, 3, 1, "", "Pattern_1"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1": [[5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model": [[5, 3, 1, "", "PolishModel"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.utils": [[5, 2, 1, "", "check_value_info"], [5, 2, 1, "", "eliminate_initializer_from_graph_input"], [5, 2, 1, "", "eliminate_unused_initializer"], [5, 2, 1, "", "eliminate_unused_input"], [5, 2, 1, "", "eliminate_unused_output"], [5, 2, 1, "", "eliminate_unused_protos"], [5, 2, 1, "", "eliminate_unused_value_info"], [5, 2, 1, "", "fix_batch_size_as_one"], [5, 2, 1, "", "fixed_point"], [5, 2, 1, "", "get_attribute"], [5, 2, 1, "", "get_node_attributes"], [5, 2, 1, "", "get_node_input_names"], [5, 2, 1, "", "get_node_output_names"], [5, 2, 1, "", "is_op_type"], [5, 2, 1, "", "make_initializer_name_unique"], [5, 2, 1, "", "make_unhashables_unique"], [5, 2, 1, "", "name_nodes"], [5, 2, 1, "", "rebuild_model"]], "furiosa.optimizer.frontend.onnx.utils": [[7, 0, 0, "-", "check_model"], [7, 0, 0, "-", "inference_shape"], [7, 0, 0, "-", "version_checker"]], "furiosa.optimizer.frontend.onnx.utils.check_model": [[7, 2, 1, "", "check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape": [[7, 3, 1, "", "InferenceShape"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape": [[7, 5, 1, "", "inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker": [[7, 3, 1, "", "CheckVersion"]], "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion": [[7, 4, 1, "", "opset_bound"], [7, 5, 1, "", "transform"]], "furiosa.optimizer.interfaces": [[8, 0, 0, "-", "transformer"]], "furiosa.optimizer.interfaces.transformer": [[8, 3, 1, "", "Transformer"]], "furiosa.optimizer.interfaces.transformer.Transformer": [[8, 5, 1, "", "transform"]], "furiosa.quantizer": [[9, 3, 1, "", "CalibrationMethod"], [9, 3, 1, "", "Calibrator"], [9, 3, 1, "", "ModelEditor"], [9, 3, 1, "", "TensorType"], [9, 0, 0, "-", "calibrator"], [9, 0, 0, "-", "editor"], [9, 2, 1, "", "get_output_names"], [9, 2, 1, "", "get_pure_input_names"], [9, 2, 1, "", "quantize"]], "furiosa.quantizer.CalibrationMethod": [[9, 4, 1, "id10", "ENTROPY_ASYM"], [9, 4, 1, "id11", "ENTROPY_SYM"], [9, 4, 1, "id12", "MIN_MAX_ASYM"], [9, 4, 1, "id13", "MIN_MAX_SYM"], [9, 4, 1, "id14", "MSE_ASYM"], [9, 4, 1, "id15", "MSE_SYM"], [9, 4, 1, "id16", "PERCENTILE_ASYM"], [9, 4, 1, "id17", "PERCENTILE_SYM"], [9, 4, 1, "id18", "SQNR_ASYM"], [9, 4, 1, "id19", "SQNR_SYM"]], "furiosa.quantizer.Calibrator": [[9, 5, 1, "", "collect_data"], [9, 5, 1, "", "compute_range"]], "furiosa.quantizer.ModelEditor": [[9, 5, 1, "", "convert_input_type"], [9, 5, 1, "", "convert_output_type"]], "furiosa.quantizer.TensorType": [[9, 4, 1, "", "INT8"], [9, 4, 1, "", "UINT8"]], "furiosa.quantizer.calibrator": [[9, 3, 1, "", "CalibrationMethod"], [9, 3, 1, "", "Calibrator"]], "furiosa.quantizer.calibrator.CalibrationMethod": [[9, 4, 1, "id0", "ENTROPY_ASYM"], [9, 4, 1, "id1", "ENTROPY_SYM"], [9, 4, 1, "id2", "MIN_MAX_ASYM"], [9, 4, 1, "id3", "MIN_MAX_SYM"], [9, 4, 1, "id4", "MSE_ASYM"], [9, 4, 1, "id5", "MSE_SYM"], [9, 4, 1, "id6", "PERCENTILE_ASYM"], [9, 4, 1, "id7", "PERCENTILE_SYM"], [9, 4, 1, "id8", "SQNR_ASYM"], [9, 4, 1, "id9", "SQNR_SYM"]], "furiosa.quantizer.calibrator.Calibrator": [[9, 5, 1, "", "collect_data"], [9, 5, 1, "", "compute_range"]], "furiosa.quantizer.editor": [[9, 3, 1, "", "ModelEditor"], [9, 3, 1, "", "TensorType"], [9, 2, 1, "", "get_output_names"], [9, 2, 1, "", "get_pure_input_names"]], "furiosa.quantizer.editor.ModelEditor": [[9, 5, 1, "", "convert_input_type"], [9, 5, 1, "", "convert_output_type"]], "furiosa.quantizer.editor.TensorType": [[9, 4, 1, "", "INT8"], [9, 4, 1, "", "UINT8"]], "furiosa.runtime": [[10, 3, 1, "", "Axis"], [10, 3, 1, "", "DataType"], [10, 1, 1, "", "FuriosaRuntimeError"], [10, 1, 1, "", "FuriosaRuntimeWarning"], [10, 3, 1, "", "Model"], [10, 3, 1, "", "ModelSource"], [10, 3, 1, "", "Receiver"], [10, 3, 1, "", "Runner"], [10, 3, 1, "", "Runtime"], [10, 3, 1, "", "Submitter"], [10, 3, 1, "", "Tensor"], [10, 3, 1, "", "TensorArray"], [10, 3, 1, "", "TensorDesc"], [10, 0, 0, "-", "compiler"], [10, 0, 0, "-", "consts"], [10, 2, 1, "", "create_queue"], [10, 2, 1, "", "create_runner"], [10, 0, 0, "-", "envs"], [10, 0, 0, "-", "errors"], [10, 2, 1, "", "full_version"], [10, 0, 0, "-", "model"], [10, 0, 0, "-", "profiler"], [10, 0, 0, "-", "session"], [10, 0, 0, "-", "tensor"]], "furiosa.runtime.Axis": [[10, 6, 1, "", "BATCH"], [10, 6, 1, "", "CHANNEL"], [10, 6, 1, "", "HEIGHT"], [10, 6, 1, "", "UNKNOWN"], [10, 6, 1, "", "WIDTH"]], "furiosa.runtime.DataType": [[10, 6, 1, "", "BFLOAT16"], [10, 6, 1, "", "FLOAT16"], [10, 6, 1, "", "FLOAT32"], [10, 6, 1, "", "INT16"], [10, 6, 1, "", "INT32"], [10, 6, 1, "", "INT64"], [10, 6, 1, "", "INT8"], [10, 6, 1, "", "UINT8"], [10, 6, 1, "", "numpy"], [10, 6, 1, "", "numpy_dtype"]], "furiosa.runtime.Model": [[10, 5, 1, "", "input"], [10, 6, 1, "", "input_num"], [10, 5, 1, "", "inputs"], [10, 5, 1, "", "output"], [10, 6, 1, "", "output_num"], [10, 5, 1, "", "outputs"], [10, 5, 1, "", "print_summary"], [10, 5, 1, "", "summary"]], "furiosa.runtime.Receiver": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "recv"]], "furiosa.runtime.Runner": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "run"]], "furiosa.runtime.Runtime": [[10, 5, 1, "", "close"]], "furiosa.runtime.Submitter": [[10, 5, 1, "", "allocate"], [10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "submit"]], "furiosa.runtime.TensorDesc": [[10, 5, 1, "", "axis"], [10, 5, 1, "", "dim"], [10, 6, 1, "", "dtype"], [10, 6, 1, "", "format"], [10, 6, 1, "", "length"], [10, 6, 1, "", "name"], [10, 6, 1, "", "ndim"], [10, 6, 1, "", "numpy_dtype"], [10, 6, 1, "", "shape"], [10, 6, 1, "", "size"], [10, 5, 1, "", "stride"]], "furiosa.runtime.compiler": [[10, 2, 1, "", "generate_compiler_log_path"]], "furiosa.runtime.envs": [[10, 2, 1, "", "current_npu_device"], [10, 2, 1, "", "is_compile_log_enabled"], [10, 2, 1, "", "log_dir"], [10, 2, 1, "", "profiler_output"]], "furiosa.runtime.errors": [[10, 1, 1, "", "ApiClientInitFailed"], [10, 1, 1, "", "CompilationFailed"], [10, 1, 1, "", "DeviceBusy"], [10, 1, 1, "", "IncompatibleApiClientError"], [10, 1, 1, "", "IncompatibleModel"], [10, 1, 1, "", "InternalError"], [10, 1, 1, "", "InvalidCompilerConfig"], [10, 1, 1, "", "InvalidInput"], [10, 1, 1, "", "InvalidSessionOption"], [10, 1, 1, "", "InvalidYamlException"], [10, 1, 1, "", "NoApiKeyException"], [10, 1, 1, "", "QueueWaitTimeout"], [10, 1, 1, "", "SessionClosed"], [10, 1, 1, "", "SessionTerminated"], [10, 1, 1, "", "TensorNameNotFound"], [10, 1, 1, "", "UnsupportedDataType"], [10, 1, 1, "", "UnsupportedFeature"], [10, 1, 1, "", "UnsupportedTensorType"]], "furiosa.runtime.furiosa.runtime.Runtime": [[10, 5, 1, "", "create_queue"], [10, 5, 1, "", "create_runner"]], "furiosa.runtime.profiler": [[10, 3, 1, "", "RecordFormat"], [10, 3, 1, "", "Resource"], [10, 3, 1, "", "profile"]], "furiosa.runtime.profiler.RecordFormat": [[10, 6, 1, "", "ChromeTrace"], [10, 6, 1, "", "PandasDataFrame"]], "furiosa.runtime.profiler.Resource": [[10, 6, 1, "", "All"], [10, 6, 1, "", "Cpu"], [10, 6, 1, "", "Npu"]], "furiosa.runtime.profiler.profile": [[10, 5, 1, "", "export_chrome_trace"], [10, 5, 1, "", "get_cpu_pandas_dataframe"], [10, 5, 1, "", "get_npu_pandas_dataframe"], [10, 5, 1, "", "get_pandas_dataframe"], [10, 5, 1, "", "get_pandas_dataframe_with_filter"], [10, 5, 1, "", "pause"], [10, 5, 1, "", "print_external_operators"], [10, 5, 1, "", "print_inferences"], [10, 5, 1, "", "print_npu_executions"], [10, 5, 1, "", "print_npu_operators"], [10, 5, 1, "", "print_summary"], [10, 5, 1, "", "record"], [10, 5, 1, "", "resume"]], "furiosa.runtime.session": [[10, 3, 1, "", "AsyncSession"], [10, 3, 1, "", "CompletionQueue"], [10, 3, 1, "", "Session"], [10, 2, 1, "", "create"], [10, 2, 1, "", "create_async"]], "furiosa.runtime.session.AsyncSession": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "submit"]], "furiosa.runtime.session.CompletionQueue": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "recv"]], "furiosa.runtime.session.Session": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "run"], [10, 5, 1, "", "run_with"]], "furiosa.runtime.sync": [[10, 3, 1, "", "Receiver"], [10, 3, 1, "", "Runner"], [10, 3, 1, "", "Runtime"], [10, 3, 1, "", "Submitter"], [10, 2, 1, "", "create_queue"], [10, 2, 1, "", "create_runner"]], "furiosa.runtime.sync.Receiver": [[10, 5, 1, "", "recv"]], "furiosa.runtime.sync.furiosa.runtime.sync.Runtime": [[10, 5, 1, "", "create_queue"], [10, 5, 1, "", "create_runner"]], "furiosa.runtime.tensor": [[10, 3, 1, "", "Tensor"], [10, 3, 1, "", "TensorArray"], [10, 2, 1, "", "numpy_dtype"], [10, 2, 1, "", "rand"], [10, 2, 1, "", "zeros"]], "furiosa.runtime.tensor.Tensor": [[10, 5, 1, "", "copy_from"], [10, 5, 1, "", "numpy"], [10, 6, 1, "", "numpy_dtype"], [10, 6, 1, "", "shape"], [10, 5, 1, "", "view"]], "furiosa.runtime.tensor.TensorArray": [[10, 5, 1, "", "is_empty"], [10, 5, 1, "", "numpy"], [10, 5, 1, "", "view"]], "furiosa.server": [[11, 0, 0, "-", "errors"], [19, 0, 0, "-", "utils"]], "furiosa.server.errors": [[11, 1, 1, "", "ModelNotFound"], [11, 1, 1, "", "ModelServerError"]], "": [[10, 7, 1, "-", "FURIOSA_DEVICES"], [10, 7, 1, "-", "FURIOSA_LOG_LEVEL"], [10, 7, 1, "-", "NPU_DEVNAME"]]}, "objtypes": {"0": "py:module", "1": "py:exception", "2": "py:function", "3": "py:class", "4": "py:attribute", "5": "py:method", "6": "py:property", "7": "std:envvar"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "exception", "Python exception"], "2": ["py", "function", "Python function"], "3": ["py", "class", "Python class"], "4": ["py", "attribute", "Python attribute"], "5": ["py", "method", "Python method"], "6": ["py", "property", "Python property"], "7": ["std", "envvar", "environment variable"]}, "titleterms": {"furiosa": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 28, 29, 31, 32, 34, 35, 36], "common": [0, 1], "packag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 28, 36, 40], "subpackag": [0, 2, 3, 4, 5, 11, 12, 14, 20], "submodul": [0, 1, 5, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], "error": [0, 1, 10, 11, 28], "modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], "nativ": [0, 31], "thread": 0, "util": [0, 5, 7, 19], "content": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22], "transport": 1, "base": [1, 22, 29], "file": [1, 30, 35, 43], "github": 1, "http": 1, "s3": 1, "optim": [2, 3, 4, 5, 6, 7, 8, 35], "frontend": [3, 4, 5, 6, 7], "onnx": [4, 5, 6, 7], "transform": [5, 6, 8], "convert_negative_pads_to_slic": 5, "convert_prelu_to_relu": 5, "fuse_batchnorm": 5, "fuse_gather_matmul": 5, "infer_squeeze_ax": 5, "polish_model": 5, "experiment": [6, 34], "eliminate_detection_postprocess": 6, "embedding_bag_port": 6, "fuse_div_for_bert": 6, "reify_conv_for_bert": 6, "check_model": 7, "inference_shap": 7, "version_check": 7, "interfac": [8, 10], "quantiz": [9, 29, 30, 31, 32, 37, 40, 41], "calibr": [9, 32], "editor": 9, "runtim": [10, 36, 37, 40], "variant": 10, "us": [10, 35, 39, 40], "legaci": [10, 40], "model": [10, 11, 20, 21, 25, 28, 31, 34, 35, 37, 40, 41, 43], "input": 10, "tensor": 10, "output": 10, "when": 10, "an": 10, "type": [10, 18], "object": 10, "devic": [10, 30, 38], "specif": [10, 26, 36], "metadata": 10, "runner": 10, "api": [10, 12, 13, 14, 15, 16, 23, 28, 31, 37, 40, 41], "inform": [10, 30, 31, 32], "infer": 10, "miscellan": 10, "queue": 10, "submit": 10, "receiv": 10, "profil": [10, 30, 31, 39], "diagnost": 10, "support": [10, 27, 28, 30, 32, 37, 38], "compil": [10, 28, 29, 30, 32, 33, 34, 35, 37], "const": 10, "env": 10, "session": [10, 28], "server": [11, 12, 13, 14, 15, 16, 17, 18, 19, 28, 36, 37, 40, 43], "registri": 11, "repositori": [11, 17, 21], "set": 11, "cli": 13, "main": 13, "grpc": [14, 15], "convert": 14, "servic": 14, "gener": 15, "model_repository_pb2": 15, "model_repository_pb2_grpc": 15, "predict_pb2": 15, "predict_pb2_grpc": 15, "rest": 16, "app": [16, 21], "endpoint": [16, 43], "handler": 17, "predict": [17, 18, 43], "model_repositori": 18, "namespac": 19, "loader": 19, "serv": [20, 21, 22, 25, 28, 29, 31, 43], "applic": 20, "telemetri": 20, "health": 21, "processor": 22, "imagenet": 22, "sdk": [23, 25, 29, 30, 31, 32, 33, 36, 37, 40], "0": [23, 25, 28, 29, 30, 31, 32], "10": [23, 25], "document": [23, 25], "bug": [24, 30], "report": 24, "furiosaai": [25, 26, 37, 40], "npu": [25, 26, 27, 28, 30, 35, 36, 38], "softwar": 25, "tutori": [25, 44], "exampl": [25, 44], "deploy": 25, "mlop": 25, "refer": [25, 42], "other": 25, "link": 25, "warboi": 26, "hw": 26, "hardwar": 26, "perform": [26, 30, 39], "see": 26, "also": 26, "list": [27, 34, 38], "oper": 27, "acceler": [27, 30], "releas": [28, 29, 30, 31, 32], "note": [28, 29, 30, 31, 32], "5": [28, 38], "improv": [28, 30, 31], "diagnosi": 28, "messag": 28, "handl": 28, "introduc": 28, "framework": [28, 31, 43], "command": [28, 34], "line": [28, 34], "manag": 28, "tool": [28, 30, 31, 32, 34], "furiosactl": [28, 34], "kubernet": [28, 37, 38], "6": 29, "how": [29, 30, 41], "upgrad": [29, 30], "major": [29, 31, 32], "chang": [29, 30, 31, 32], "python": [29, 30, 37, 40], "addit": 29, "fastapi": 29, "advanc": 29, "librari": 29, "7": 30, "compon": [30, 31, 32], "version": [30, 31, 32, 36, 43], "kei": 30, "more": 30, "cach": [30, 35], "hint": 30, "fix": 30, "plugin": [30, 38], "configur": [30, 36, 38, 40, 43], "8": 31, "instal": [31, 32, 33, 34, 36, 38, 40, 43], "latest": [31, 32], "toolkit": [31, 32, 34], "zoo": 31, "ad": [31, 36], "post": 31, "process": 31, "code": [31, 44], "9": 32, "method": 32, "c": [33, 37], "user": [33, 36, 40], "guid": [33, 40], "info": 34, "p": 34, "top": 34, "field": 34, "bench": 34, "benchmark": 34, "litmu": [34, 40], "compat": 34, "checker": 34, "target": 35, "batch": 35, "size": 35, "well": 35, "known": 35, "enf": 35, "behavior": 35, "accord": 35, "fc_cache_lifetim": 35, "driver": [36, 37], "firmwar": [36, 37], "minimum": 36, "requir": [36, 40], "apt": 36, "group": 36, "hold": 36, "unhold": 36, "updat": 36, "sw": 37, "stack": 37, "introduct": 37, "kernel": 37, "1": 38, "prepar": 38, "node": 38, "2": [38, 43], "featur": 38, "discoveri": 38, "3": 38, "label": 38, "argument": 38, "k8": 38, "4": 38, "creat": [38, 40], "pod": 38, "monitor": 38, "metric": 38, "export": 38, "collect": 38, "categori": 38, "trace": 39, "analysi": 39, "enabl": 39, "environmet": 39, "variabl": 39, "context": 39, "paus": 39, "resum": 39, "panda": 39, "datafram": 39, "execut": 40, "environ": 40, "setup": 40, "conda": 40, "activ": 40, "isol": 40, "linux": 40, "extra": 40, "It": 41, "work": 41, "accuraci": 41, "run": 43, "kserv": 43, "protocol": 43}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinx": 57}, "alltitles": {"furiosa.common package": [[0, "furiosa-common-package"]], "Subpackages": [[0, "subpackages"], [2, "subpackages"], [3, "subpackages"], [4, "subpackages"], [5, "subpackages"], [11, "subpackages"], [12, "subpackages"], [14, "subpackages"], [20, "subpackages"]], "Submodules": [[0, "submodules"], [1, "submodules"], [5, "submodules"], [6, "submodules"], [7, "submodules"], [8, "submodules"], [9, "submodules"], [11, "submodules"], [13, "submodules"], [14, "submodules"], [15, "submodules"], [16, "submodules"], [17, "submodules"], [18, "submodules"], [19, "submodules"], [20, "submodules"], [21, "submodules"], [22, "submodules"]], "furiosa.common.error module": [[0, "module-furiosa.common.error"]], "furiosa.common.native module": [[0, "module-furiosa.common.native"]], "furiosa.common.thread module": [[0, "module-furiosa.common.thread"]], "furiosa.common.utils module": [[0, "module-furiosa.common.utils"]], "Module contents": [[0, "module-furiosa.common"], [1, "module-furiosa.common.transport"], [2, "module-furiosa.optimizer"], [3, "module-furiosa.optimizer.frontend"], [4, "module-furiosa.optimizer.frontend.onnx"], [5, "module-furiosa.optimizer.frontend.onnx.transformer"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"], [7, "module-furiosa.optimizer.frontend.onnx.utils"], [8, "module-furiosa.optimizer.interfaces"], [9, "module-furiosa.quantizer"], [11, "module-contents"], [12, "module-contents"], [13, "module-contents"], [14, "module-contents"], [15, "module-contents"], [16, "module-contents"], [17, "module-contents"], [18, "module-contents"], [20, "module-contents"], [21, "module-contents"], [22, "module-contents"]], "furiosa.common.transport package": [[1, "furiosa-common-transport-package"]], "furiosa.common.transport.base module": [[1, "module-furiosa.common.transport.base"]], "furiosa.common.transport.error module": [[1, "module-furiosa.common.transport.error"]], "furiosa.common.transport.file module": [[1, "module-furiosa.common.transport.file"]], "furiosa.common.transport.github module": [[1, "module-furiosa.common.transport.github"]], "furiosa.common.transport.http module": [[1, "module-furiosa.common.transport.http"]], "furiosa.common.transport.s3 module": [[1, "module-furiosa.common.transport.s3"]], "furiosa.optimizer package": [[2, "furiosa-optimizer-package"]], "furiosa.optimizer.frontend package": [[3, "furiosa-optimizer-frontend-package"]], "furiosa.optimizer.frontend.onnx package": [[4, "furiosa-optimizer-frontend-onnx-package"]], "furiosa.optimizer.frontend.onnx.transformer package": [[5, "furiosa-optimizer-frontend-onnx-transformer-package"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"]], "furiosa.optimizer.frontend.onnx.transformer.utils module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"]], "furiosa.optimizer.frontend.onnx.transformer.experimental package": [[6, "furiosa-optimizer-frontend-onnx-transformer-experimental-package"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"]], "furiosa.optimizer.frontend.onnx.utils package": [[7, "furiosa-optimizer-frontend-onnx-utils-package"]], "furiosa.optimizer.frontend.onnx.utils.check_model module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"]], "furiosa.optimizer.interfaces package": [[8, "furiosa-optimizer-interfaces-package"]], "furiosa.optimizer.interfaces.transformer module": [[8, "module-furiosa.optimizer.interfaces.transformer"]], "furiosa.quantizer package": [[9, "furiosa-quantizer-package"]], "furiosa.quantizer.calibrator module": [[9, "module-furiosa.quantizer.calibrator"]], "furiosa.quantizer.editor module": [[9, "module-furiosa.quantizer.editor"]], "furiosa.runtime package": [[10, "module-furiosa.runtime"]], "Runtime Variants": [[10, "runtime-variants"]], "Use of legacy modules": [[10, "use-of-legacy-modules"]], "Model Inputs": [[10, "model-inputs"]], "Tensor Inputs and Outputs": [[10, "tensor-inputs-and-outputs"]], "Legacy Interface": [[10, "legacy-interface"], [10, "legacy-runner-api"], [10, "legacy-queue-api"]], "When used as an input type": [[10, "legacy-tensorarray-as-input"], [10, "legacy-tensor-as-input"]], "Runtime Object": [[10, "runtime-object"]], "Device Specification": [[10, "device-specification"]], "Model Metadata": [[10, "model-metadata"]], "Runner API": [[10, "runner-api"]], "Model informations": [[10, null], [10, null], [10, null]], "Inference": [[10, null]], "Miscellaneous": [[10, null], [10, null], [10, null]], "Queue API": [[10, "queue-api"]], "Submitting": [[10, null]], "Receiving": [[10, null]], "Profiler": [[10, "profiler"], [31, "profiler"]], "Diagnostics": [[10, "diagnostics"]], "Legacy Supports": [[10, "legacy-supports"]], "furiosa.runtime.compiler": [[10, "module-furiosa.runtime.compiler"]], "furiosa.runtime.consts": [[10, "module-furiosa.runtime.consts"]], "furiosa.runtime.envs": [[10, "module-furiosa.runtime.envs"]], "furiosa.runtime.errors": [[10, "module-furiosa.runtime.errors"]], "furiosa.runtime.model": [[10, "module-furiosa.runtime.model"]], "furiosa.runtime.session": [[10, "module-furiosa.runtime.session"]], "furiosa.runtime.tensor": [[10, "module-furiosa.runtime.tensor"]], "furiosa.server package": [[11, "furiosa-server-package"]], "furiosa.server.errors module": [[11, "module-furiosa.server.errors"]], "furiosa.server.model module": [[11, "furiosa-server-model-module"]], "furiosa.server.registry module": [[11, "furiosa-server-registry-module"]], "furiosa.server.repository module": [[11, "furiosa-server-repository-module"]], "furiosa.server.server module": [[11, "furiosa-server-server-module"]], "furiosa.server.settings module": [[11, "furiosa-server-settings-module"]], "furiosa.server.api package": [[12, "furiosa-server-api-package"]], "furiosa.server.api.cli package": [[13, "furiosa-server-api-cli-package"]], "furiosa.server.api.cli.main module": [[13, "furiosa-server-api-cli-main-module"]], "furiosa.server.api.grpc package": [[14, "furiosa-server-api-grpc-package"]], "furiosa.server.api.grpc.converters module": [[14, "furiosa-server-api-grpc-converters-module"]], "furiosa.server.api.grpc.server module": [[14, "furiosa-server-api-grpc-server-module"]], "furiosa.server.api.grpc.servicers module": [[14, "furiosa-server-api-grpc-servicers-module"]], "furiosa.server.api.grpc.generated package": [[15, "furiosa-server-api-grpc-generated-package"]], "furiosa.server.api.grpc.generated.model_repository_pb2 module": [[15, "furiosa-server-api-grpc-generated-model-repository-pb2-module"]], "furiosa.server.api.grpc.generated.model_repository_pb2_grpc module": [[15, "furiosa-server-api-grpc-generated-model-repository-pb2-grpc-module"]], "furiosa.server.api.grpc.generated.predict_pb2 module": [[15, "furiosa-server-api-grpc-generated-predict-pb2-module"]], "furiosa.server.api.grpc.generated.predict_pb2_grpc module": [[15, "furiosa-server-api-grpc-generated-predict-pb2-grpc-module"]], "furiosa.server.api.rest package": [[16, "furiosa-server-api-rest-package"]], "furiosa.server.api.rest.app module": [[16, "furiosa-server-api-rest-app-module"]], "furiosa.server.api.rest.endpoints module": [[16, "furiosa-server-api-rest-endpoints-module"]], "furiosa.server.api.rest.server module": [[16, "furiosa-server-api-rest-server-module"]], "furiosa.server.handlers package": [[17, "furiosa-server-handlers-package"]], "furiosa.server.handlers.predict module": [[17, "furiosa-server-handlers-predict-module"]], "furiosa.server.handlers.repository module": [[17, "furiosa-server-handlers-repository-module"]], "furiosa.server.types package": [[18, "furiosa-server-types-package"]], "furiosa.server.types.model_repository module": [[18, "furiosa-server-types-model-repository-module"]], "furiosa.server.types.predict module": [[18, "furiosa-server-types-predict-module"]], "furiosa.server.utils namespace": [[19, "module-furiosa.server.utils"]], "furiosa.server.utils.loader module": [[19, "furiosa-server-utils-loader-module"]], "furiosa.serving package": [[20, "furiosa-serving-package"]], "furiosa.serving.application module": [[20, "furiosa-serving-application-module"]], "furiosa.serving.model module": [[20, "furiosa-serving-model-module"]], "furiosa.serving.telemetry module": [[20, "furiosa-serving-telemetry-module"]], "furiosa.serving.apps package": [[21, "furiosa-serving-apps-package"]], "furiosa.serving.apps.health module": [[21, "furiosa-serving-apps-health-module"]], "furiosa.serving.apps.model module": [[21, "furiosa-serving-apps-model-module"]], "furiosa.serving.apps.repository module": [[21, "furiosa-serving-apps-repository-module"]], "furiosa.serving.processors package": [[22, "furiosa-serving-processors-package"]], "furiosa.serving.processors.base module": [[22, "furiosa-serving-processors-base-module"]], "furiosa.serving.processors.imagenet module": [[22, "furiosa-serving-processors-imagenet-module"]], "Furiosa SDK 0.10.0 API Documentation": [[23, "furiosa-sdk-release-api-documentation"]], "Bug Report": [[24, "bug-report"]], "FuriosaAI NPU & SDK 0.10.0 Documents": [[25, "furiosaai-npu-sdk-release-documents"]], "FuriosaAI NPU": [[25, "furiosaai-npu"], [26, "furiosaai-npu"]], "FuriosaAI Software": [[25, "furiosaai-software"]], "FuriosaAI SDK Tutorial and Examples": [[25, "furiosaai-sdk-tutorial-and-examples"]], "Serving, Model Deployment, MLOps": [[25, "serving-model-deployment-mlops"]], "References": [[25, "references"], [42, "references"]], "Other Links": [[25, "other-links"]], "FuriosaAI Warboy": [[26, "furiosaai-warboy"]], "FuriosaAI Warboy HW Specifications": [[26, "furiosaai-warboy-hw-specifications"]], "Warboy Hardware Specification": [[26, "id2"]], "FuriosaAI Warboy Performance": [[26, "furiosaai-warboy-performance"]], "See Also": [[26, "see-also"]], "List of Supported Operators for NPU Acceleration": [[27, "list-of-supported-operators-for-npu-acceleration"]], "Operators Accelerated on NPU": [[27, "id1"]], "Release Notes - 0.5.0": [[28, "release-notes-0-5-0"]], "Compiler Improvement": [[28, "compiler-improvement"]], "Session API Improvement": [[28, "session-api-improvement"]], "Error Diagnosis Message & Error Handling Improvements": [[28, "error-diagnosis-message-error-handling-improvements"]], "Introducing Furiosa Server (serving framework)": [[28, "introducing-furiosa-server-serving-framework"]], "Introducing Furiosa Model package": [[28, "introducing-furiosa-model-package"]], "Command line NPU management tool: furiosactl": [[28, "command-line-npu-management-tool-furiosactl"]], "Kubernetes support": [[28, "kubernetes-support"]], "Release Notes - 0.6.0": [[29, "release-notes-0-6-0"]], "How to upgrade": [[29, "how-to-upgrade"], [30, "how-to-upgrade"]], "Major changes": [[29, "major-changes"], [31, "major-changes"], [32, "major-changes"]], "Compiler": [[29, "compiler"], [32, "compiler"], [35, "compiler"], [37, "compiler"]], "Quantizer": [[29, "quantizer"], [40, "quantizer"]], "Python SDK": [[29, "python-sdk"]], "Furiosa Serving: Addition of FastAPI-based advanced serving library": [[29, "furiosa-serving-addition-of-fastapi-based-advanced-serving-library"]], "Release Notes - 0.7.0": [[30, "release-notes-0-7-0"]], "component version information": [[30, "id1"]], "Key changes": [[30, "key-changes"]], "Compiler - More NPU acceleration supports": [[30, "compiler-more-npu-acceleration-supports"]], "Compiler - Compiler Cache": [[30, "compiler-compiler-cache"]], "Compiler - Compiler Hint": [[30, "compiler-compiler-hint"]], "Performance Profiling Tools": [[30, "performance-profiling-tools"]], "Improvements/Bug fixes of Python SDK": [[30, "improvements-bug-fixes-of-python-sdk"]], "Improvements/Bug fixes of Quantization tools": [[30, "improvements-bug-fixes-of-quantization-tools"]], "Device Plugin - Configuration file support": [[30, "device-plugin-configuration-file-support"]], "Release Notes - 0.8.0": [[31, "release-notes-0-8-0"]], "Component Version Information": [[31, "id3"], [32, "id1"]], "Installing the latest SDK": [[31, "installing-the-latest-sdk"], [32, "installing-the-latest-sdk"]], "Improvements to serving framework API": [[31, "improvements-to-serving-framework-api"]], "Quantization tool": [[31, "quantization-tool"], [32, "quantization-tool"]], "furiosa-toolkit": [[31, "furiosa-toolkit"], [32, "furiosa-toolkit"], [34, "furiosa-toolkit"]], "Model Zoo API improvements, added models, and added native post-processing code": [[31, "model-zoo-api-improvements-added-models-and-added-native-post-processing-code"]], "Release Notes - 0.9.0": [[32, "release-notes-0-9-0"]], "Supported Calibration Methods": [[32, "id2"]], "C SDK installation and user guide": [[33, "c-sdk-installation-and-user-guide"]], "C SDK installation": [[33, "c-sdk-installation"]], "Compiling with C SDK": [[33, "compiling-with-c-sdk"]], "Command Line Tools": [[34, "command-line-tools"]], "furiosa-toolkit installation": [[34, "furiosa-toolkit-installation"]], "furiosactl": [[34, "furiosactl"]], "furiosactl info": [[34, "furiosactl-info"]], "furiosactl list": [[34, "furiosactl-list"]], "furiosactl ps": [[34, "furiosactl-ps"]], "furiosactl top (experimental)": [[34, "furiosactl-top-experimental"]], "furiosa top fields": [[34, "id1"]], "furiosa-bench (Benchmark Tool)": [[34, "furiosa-bench-benchmark-tool"]], "furiosa": [[34, "furiosa"]], "furiosa compile": [[34, "furiosa-compile"]], "furiosa litmus (Model Compatibility Checker)": [[34, "furiosa-litmus-model-compatibility-checker"]], "furiosa-compiler": [[35, "furiosa-compiler"]], "Target NPUs": [[35, "id4"]], "Optimal Batch Size for Well-known Models": [[35, "id5"]], "Using ENF files": [[35, "using-enf-files"]], "Compiler Cache": [[35, "compiler-cache"]], "Cache behaviors according to FC_CACHE_LIFETIME": [[35, "id6"]], "Driver, Firmware, and Runtime Installation": [[36, "driver-firmware-and-runtime-installation"]], "Minimum requirements for SDK installation": [[36, "minimum-requirements-for-sdk-installation"]], "APT server configuration": [[36, "apt-server-configuration"]], "Installing required packages.": [[36, "installing-required-packages"]], "Adding a user to the furiosa Group": [[36, "adding-a-user-to-the-furiosa-group"]], "Holding/unholding installed version": [[36, "holding-unholding-installed-version"]], "Installing a specific version": [[36, "installing-a-specific-version"]], "NPU Firmware Update": [[36, "npu-firmware-update"]], "FuriosaAI SW Stack Introduction": [[37, "furiosaai-sw-stack-introduction"]], "Kernel Driver and Firmware": [[37, "kernel-driver-and-firmware"]], "Runtime": [[37, "runtime"]], "Python SDK and C SDK": [[37, "python-sdk-and-c-sdk"]], "Model quantizer API": [[37, "model-quantizer-api"]], "Model Server": [[37, "model-server"], [40, "model-server"]], "Kubernetes Support": [[37, "kubernetes-support"], [38, "kubernetes-support"]], "1. Preparing NPU nodes": [[38, "preparing-npu-nodes"]], "2. Installing Node Feature Discovery": [[38, "installing-node-feature-discovery"]], "3. Installing Device Plugin and NPU Feature Discovery": [[38, "installing-device-plugin-and-npu-feature-discovery"]], "NPU Node Labels": [[38, "id1"]], "Device Plugin Configuration": [[38, "device-plugin-configuration"]], "arguments of k8s-device-plugin": [[38, "id2"]], "4. Creating a Pod with NPUs": [[38, "creating-a-pod-with-npus"]], "5. NPU monitoring": [[38, "npu-monitoring"]], "npu-metrics-exporter collection category list": [[38, "id5"]], "Performance profiling": [[39, "performance-profiling"]], "Trace Analysis": [[39, "trace-analysis"]], "Tracing Enabled by Environmet Variable": [[39, "tracing-enabled-by-environmet-variable"]], "Tracing Enabled by Profiler Context": [[39, "tracing-enabled-by-profiler-context"]], "Pause/Resume of Profiler Context": [[39, "pause-resume-of-profiler-context"]], "Trace analysis using Pandas DataFrame": [[39, "trace-analysis-using-pandas-dataframe"]], "Python SDK installation and user guide": [[40, "python-sdk-installation-and-user-guide"]], "Requirements": [[40, "requirements"]], "Python execution environment setup": [[40, "python-execution-environment-setup"]], "Python environment configuration with Conda": [[40, "python-environment-configuration-with-conda"]], "Creating and activating isolated Python execution environment": [[40, "creating-and-activating-isolated-python-execution-environment"]], "Configuring Python environment using Linux packages": [[40, "configuring-python-environment-using-linux-packages"]], "Installing Python SDK package": [[40, "installing-python-sdk-package"]], "Extra packages": [[40, "extra-packages"]], "Legacy Runtime/API": [[40, "legacy-runtime-api"]], "FuriosaAI Models": [[40, "furiosaai-models"]], "Litmus": [[40, "litmus"]], "Model Quantization": [[41, "model-quantization"]], "How It Works": [[41, "how-it-works"]], "Accuracy of Quantized Models": [[41, "accuracy-of-quantized-models"]], "Quantization Accuracy": [[41, "id1"]], "Model Quantization APIs": [[41, "model-quantization-apis"]], "Model Server (Serving Framework)": [[43, "model-server-serving-framework"]], "Installation": [[43, "installation"]], "Running a Model Server": [[43, "running-a-model-server"]], "Running a Model Server with a Configuration File": [[43, "running-a-model-server-with-a-configuration-file"]], "Endpoints": [[43, "endpoints"]], "Endpoints of KServe Predict Protocol Version 2": [[43, "id1"]], "Tutorial and Code Examples": [[44, "tutorial-and-code-examples"]], "Tutorial": [[44, "id1"]], "Code Examples": [[44, "code-examples"]]}, "indexentries": {"error (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.ERROR"]], "furiosaerror": [[0, "furiosa.common.error.FuriosaError"]], "furiosaversioninfo (class in furiosa.common.utils)": [[0, "furiosa.common.utils.FuriosaVersionInfo"]], "info (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.INFO"]], "loglevel (class in furiosa.common.native)": [[0, "furiosa.common.native.LogLevel"]], "off (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.OFF"]], "warn (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.WARN"]], "asynchronous() (in module furiosa.common.thread)": [[0, "furiosa.common.thread.asynchronous"]], "eprint() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.eprint"]], "find_global_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_global_lib_path"]], "find_native_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_native_lib_path"]], "find_native_libs() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_native_libs"]], "find_user_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_user_lib_path"]], "furiosa.common": [[0, "module-furiosa.common"]], "furiosa.common.error": [[0, "module-furiosa.common.error"]], "furiosa.common.native": [[0, "module-furiosa.common.native"]], "furiosa.common.thread": [[0, "module-furiosa.common.thread"]], "furiosa.common.utils": [[0, "module-furiosa.common.utils"]], "get_sdk_version() (in module furiosa.common)": [[0, "furiosa.common.get_sdk_version"]], "get_sdk_version() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.get_sdk_version"]], "is_err() (in module furiosa.common.error)": [[0, "furiosa.common.error.is_err"]], "is_ok() (in module furiosa.common.error)": [[0, "furiosa.common.error.is_ok"]], "module": [[0, "module-furiosa.common"], [0, "module-furiosa.common.error"], [0, "module-furiosa.common.native"], [0, "module-furiosa.common.thread"], [0, "module-furiosa.common.utils"], [1, "module-furiosa.common.transport"], [1, "module-furiosa.common.transport.base"], [1, "module-furiosa.common.transport.error"], [1, "module-furiosa.common.transport.file"], [1, "module-furiosa.common.transport.github"], [1, "module-furiosa.common.transport.http"], [1, "module-furiosa.common.transport.s3"], [2, "module-furiosa.optimizer"], [3, "module-furiosa.optimizer.frontend"], [4, "module-furiosa.optimizer.frontend.onnx"], [5, "module-furiosa.optimizer.frontend.onnx.transformer"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"], [7, "module-furiosa.optimizer.frontend.onnx.utils"], [7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"], [7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"], [7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"], [8, "module-furiosa.optimizer.interfaces"], [8, "module-furiosa.optimizer.interfaces.transformer"], [9, "module-furiosa.quantizer"], [9, "module-furiosa.quantizer.calibrator"], [9, "module-furiosa.quantizer.editor"], [10, "module-furiosa.runtime"], [10, "module-furiosa.runtime.compiler"], [10, "module-furiosa.runtime.consts"], [10, "module-furiosa.runtime.envs"], [10, "module-furiosa.runtime.errors"], [10, "module-furiosa.runtime.model"], [10, "module-furiosa.runtime.profiler"], [10, "module-furiosa.runtime.session"], [10, "module-furiosa.runtime.tensor"], [11, "module-furiosa.server.errors"], [19, "module-furiosa.server.utils"]], "removeprefix() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.removeprefix"]], "synchronous() (in module furiosa.common.thread)": [[0, "furiosa.common.thread.synchronous"]], "filetransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.FileTransport"]], "filetransport (class in furiosa.common.transport.file)": [[1, "furiosa.common.transport.file.FileTransport"]], "githubtransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.GithubTransport"]], "githubtransport (class in furiosa.common.transport.github)": [[1, "furiosa.common.transport.github.GithubTransport"]], "httptransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.HTTPTransport"]], "httptransport (class in furiosa.common.transport.http)": [[1, "furiosa.common.transport.http.HTTPTransport"]], "s3transport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.S3Transport"]], "s3transport (class in furiosa.common.transport.s3)": [[1, "furiosa.common.transport.s3.S3Transport"]], "transport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.Transport"]], "transport (class in furiosa.common.transport.base)": [[1, "furiosa.common.transport.base.Transport"]], "transporterror": [[1, "furiosa.common.transport.error.TransportError"]], "transportnotfound": [[1, "furiosa.common.transport.error.TransportNotFound"]], "cache_directory (furiosa.common.transport.transport property)": [[1, "furiosa.common.transport.Transport.cache_directory"]], "cache_directory (furiosa.common.transport.base.transport property)": [[1, "furiosa.common.transport.base.Transport.cache_directory"]], "download() (furiosa.common.transport.filetransport method)": [[1, "furiosa.common.transport.FileTransport.download"]], "download() (furiosa.common.transport.githubtransport method)": [[1, "furiosa.common.transport.GithubTransport.download"]], "download() (furiosa.common.transport.httptransport method)": [[1, "furiosa.common.transport.HTTPTransport.download"]], "download() (furiosa.common.transport.s3transport method)": [[1, "furiosa.common.transport.S3Transport.download"]], "download() (furiosa.common.transport.transport method)": [[1, "furiosa.common.transport.Transport.download"]], "download() (furiosa.common.transport.base.transport method)": [[1, "furiosa.common.transport.base.Transport.download"]], "download() (furiosa.common.transport.file.filetransport method)": [[1, "furiosa.common.transport.file.FileTransport.download"]], "download() (furiosa.common.transport.github.githubtransport method)": [[1, "furiosa.common.transport.github.GithubTransport.download"]], "download() (furiosa.common.transport.http.httptransport method)": [[1, "furiosa.common.transport.http.HTTPTransport.download"]], "download() (furiosa.common.transport.s3.s3transport method)": [[1, "furiosa.common.transport.s3.S3Transport.download"]], "download() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.download"]], "furiosa.common.transport": [[1, "module-furiosa.common.transport"]], "furiosa.common.transport.base": [[1, "module-furiosa.common.transport.base"]], "furiosa.common.transport.error": [[1, "module-furiosa.common.transport.error"]], "furiosa.common.transport.file": [[1, "module-furiosa.common.transport.file"]], "furiosa.common.transport.github": [[1, "module-furiosa.common.transport.github"]], "furiosa.common.transport.http": [[1, "module-furiosa.common.transport.http"]], "furiosa.common.transport.s3": [[1, "module-furiosa.common.transport.s3"]], "is_relative() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.is_relative"]], "is_supported() (furiosa.common.transport.filetransport static method)": [[1, "furiosa.common.transport.FileTransport.is_supported"]], "is_supported() (furiosa.common.transport.githubtransport static method)": [[1, "furiosa.common.transport.GithubTransport.is_supported"]], "is_supported() (furiosa.common.transport.httptransport static method)": [[1, "furiosa.common.transport.HTTPTransport.is_supported"]], "is_supported() (furiosa.common.transport.s3transport static method)": [[1, "furiosa.common.transport.S3Transport.is_supported"]], "is_supported() (furiosa.common.transport.transport static method)": [[1, "furiosa.common.transport.Transport.is_supported"]], "is_supported() (furiosa.common.transport.base.transport static method)": [[1, "furiosa.common.transport.base.Transport.is_supported"]], "is_supported() (furiosa.common.transport.file.filetransport static method)": [[1, "furiosa.common.transport.file.FileTransport.is_supported"]], "is_supported() (furiosa.common.transport.github.githubtransport static method)": [[1, "furiosa.common.transport.github.GithubTransport.is_supported"]], "is_supported() (furiosa.common.transport.http.httptransport static method)": [[1, "furiosa.common.transport.http.HTTPTransport.is_supported"]], "is_supported() (furiosa.common.transport.s3.s3transport static method)": [[1, "furiosa.common.transport.s3.S3Transport.is_supported"]], "parse() (furiosa.common.transport.githubtransport method)": [[1, "furiosa.common.transport.GithubTransport.parse"]], "parse() (furiosa.common.transport.s3transport static method)": [[1, "furiosa.common.transport.S3Transport.parse"]], "parse() (furiosa.common.transport.github.githubtransport method)": [[1, "furiosa.common.transport.github.GithubTransport.parse"]], "parse() (furiosa.common.transport.s3.s3transport static method)": [[1, "furiosa.common.transport.s3.S3Transport.parse"]], "read (furiosa.common.transport.filetransport attribute)": [[1, "furiosa.common.transport.FileTransport.read"]], "read (furiosa.common.transport.githubtransport attribute)": [[1, "furiosa.common.transport.GithubTransport.read"]], "read (furiosa.common.transport.httptransport attribute)": [[1, "furiosa.common.transport.HTTPTransport.read"]], "read (furiosa.common.transport.s3transport attribute)": [[1, "furiosa.common.transport.S3Transport.read"]], "read (furiosa.common.transport.transport attribute)": [[1, "furiosa.common.transport.Transport.read"]], "read (furiosa.common.transport.base.transport attribute)": [[1, "furiosa.common.transport.base.Transport.read"]], "read (furiosa.common.transport.file.filetransport attribute)": [[1, "furiosa.common.transport.file.FileTransport.read"]], "read (furiosa.common.transport.github.githubtransport attribute)": [[1, "furiosa.common.transport.github.GithubTransport.read"]], "read (furiosa.common.transport.http.httptransport attribute)": [[1, "furiosa.common.transport.http.HTTPTransport.read"]], "read (furiosa.common.transport.s3.s3transport attribute)": [[1, "furiosa.common.transport.s3.S3Transport.read"]], "scheme (furiosa.common.transport.filetransport attribute)": [[1, "furiosa.common.transport.FileTransport.scheme"]], "scheme (furiosa.common.transport.s3transport attribute)": [[1, "furiosa.common.transport.S3Transport.scheme"]], "scheme (furiosa.common.transport.file.filetransport attribute)": [[1, "furiosa.common.transport.file.FileTransport.scheme"]], "scheme (furiosa.common.transport.s3.s3transport attribute)": [[1, "furiosa.common.transport.s3.S3Transport.scheme"]], "schemes (furiosa.common.transport.githubtransport attribute)": [[1, "furiosa.common.transport.GithubTransport.schemes"]], "schemes (furiosa.common.transport.httptransport attribute)": [[1, "furiosa.common.transport.HTTPTransport.schemes"]], "schemes (furiosa.common.transport.github.githubtransport attribute)": [[1, "furiosa.common.transport.github.GithubTransport.schemes"]], "schemes (furiosa.common.transport.http.httptransport attribute)": [[1, "furiosa.common.transport.http.HTTPTransport.schemes"]], "supported() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.supported"]], "furiosa.optimizer": [[2, "module-furiosa.optimizer"]], "optimize_model() (in module furiosa.optimizer)": [[2, "furiosa.optimizer.optimize_model"]], "furiosa.optimizer.frontend": [[3, "module-furiosa.optimizer.frontend"]], "furiosa.optimizer.frontend.onnx": [[4, "module-furiosa.optimizer.frontend.onnx"]], "optimize_model() (in module furiosa.optimizer.frontend.onnx)": [[4, "furiosa.optimizer.frontend.onnx.optimize_model"]], "convertnegativepadstoslice (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice"]], "convertprelutorelu (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu"]], "fusebatchnorm (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm"]], "fusegathermatmul (class in furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul"]], "infersqueezeaxes (class in furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes"]], "onnxtransformer (class in furiosa.optimizer.frontend.onnx.transformer)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2"]], "pattern_3 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3"]], "pattern_4 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4"]], "polishmodel (class in furiosa.optimizer.frontend.onnx.transformer.polish_model)": [[5, "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel"]], "bridge_disconnected_nodes() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.bridge_disconnected_nodes"]], "build_optimized_model() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.build_optimized_model"]], "check_runnable (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.check_runnable"]], "check_value_info() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.check_value_info"]], "copy_value_info() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.copy_value_info"]], "eliminate_initializer_from_graph_input() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_initializer_from_graph_input"]], "eliminate_unused_initializer() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_initializer"]], "eliminate_unused_input() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_input"]], "eliminate_unused_output() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_output"]], "eliminate_unused_protos() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_protos"]], "eliminate_unused_value_info() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_value_info"]], "find_next_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.find_next_node"]], "find_prev_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.find_prev_node"]], "fix_batch_size_as_one() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.fix_batch_size_as_one"]], "fixed_point() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.fixed_point"]], "furiosa.optimizer.frontend.onnx.transformer": [[5, "module-furiosa.optimizer.frontend.onnx.transformer"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"]], "furiosa.optimizer.frontend.onnx.transformer.utils": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"]], "get_attribute() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_attribute"]], "get_data_node_input() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_data_node_input"]], "get_init_node_input() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_init_node_input"]], "get_initializer_array() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_initializer_array"]], "get_map_values() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_map_values"]], "get_node_attributes() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_attributes"]], "get_node_input_names() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_input_names"]], "get_node_output_names() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_output_names"]], "get_value_info_dtype() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_value_info_dtype"]], "get_value_info_shape() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_value_info_shape"]], "is_op_type() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.is_op_type"]], "is_same_shape() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.is_same_shape"]], "make_initializer_name_unique() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.make_initializer_name_unique"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.make_new_init"]], "make_new_init_and_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.make_new_init_and_vi"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 static method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.make_new_node"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_vi"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_vi"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.make_new_vi"]], "make_unhashables_unique() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.make_unhashables_unique"]], "name_nodes() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.name_nodes"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_condition_checker"]], "pattern_matcher() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pattern_matcher"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_matching"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_to_match"]], "pop_multiple_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pop_multiple_optimizer_map"]], "pop_single_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pop_single_optimizer_map"]], "rebuild_model() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.rebuild_model"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.convertnegativepadstoslice method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.convertprelutorelu method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.fusebatchnorm method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.fusegathermatmul method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.infersqueezeaxes method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.polish_model.polishmodel method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel.transform"]], "transform_to_eliminate() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform_to_eliminate"]], "transform_to_fuse() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform_to_fuse"]], "traverse_prev_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.traverse_prev_node"]], "update_graph_fields() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_graph_fields"]], "update_multiple_initializer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_initializer_map"]], "update_multiple_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_optimizer_map"]], "update_multiple_value_info_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_value_info_map"]], "update_single_initializer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_initializer_map"]], "update_single_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_optimizer_map"]], "update_single_value_info_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_value_info_map"]], "eliminatessddetectionpostprocess (class in furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess"]], "embeddingbagpattern (class in furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern"]], "embeddingbagporting (class in furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting"]], "fusedivforbert (class in furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert"]], "reifyconvforbert (class in furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert"]], "check_condition_3() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.check_condition_3"]], "furiosa.optimizer.frontend.onnx.transformer.experimental": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.pattern_condition_checker"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.pattern_matching"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.eliminatessddetectionpostprocess method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagporting method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.fusedivforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.reifyconvforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert.transform"]], "transform_matmul_add() (furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.fusedivforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert.transform_matmul_add"]], "transform_matmul_add() (furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.reifyconvforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert.transform_matmul_add"]], "checkversion (class in furiosa.optimizer.frontend.onnx.utils.version_checker)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion"]], "inferenceshape (class in furiosa.optimizer.frontend.onnx.utils.inference_shape)": [[7, "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape"]], "check_model() (in module furiosa.optimizer.frontend.onnx.utils.check_model)": [[7, "furiosa.optimizer.frontend.onnx.utils.check_model.check_model"]], "furiosa.optimizer.frontend.onnx.utils": [[7, "module-furiosa.optimizer.frontend.onnx.utils"]], "furiosa.optimizer.frontend.onnx.utils.check_model": [[7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape": [[7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker": [[7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"]], "inference_shape() (furiosa.optimizer.frontend.onnx.utils.inference_shape.inferenceshape method)": [[7, "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape.inference_shape"]], "opset_bound (furiosa.optimizer.frontend.onnx.utils.version_checker.checkversion attribute)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion.opset_bound"]], "transform() (furiosa.optimizer.frontend.onnx.utils.version_checker.checkversion method)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion.transform"]], "transformer (class in furiosa.optimizer.interfaces.transformer)": [[8, "furiosa.optimizer.interfaces.transformer.Transformer"]], "furiosa.optimizer.interfaces": [[8, "module-furiosa.optimizer.interfaces"]], "furiosa.optimizer.interfaces.transformer": [[8, "module-furiosa.optimizer.interfaces.transformer"]], "transform() (furiosa.optimizer.interfaces.transformer.transformer method)": [[8, "furiosa.optimizer.interfaces.transformer.Transformer.transform"]], "calibrationmethod (class in furiosa.quantizer)": [[9, "furiosa.quantizer.CalibrationMethod"]], "calibrationmethod (class in furiosa.quantizer.calibrator)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod"]], "calibrator (class in furiosa.quantizer)": [[9, "furiosa.quantizer.Calibrator"]], "calibrator (class in furiosa.quantizer.calibrator)": [[9, "furiosa.quantizer.calibrator.Calibrator"]], "entropy_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.ENTROPY_ASYM"], [9, "id10"]], "entropy_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.ENTROPY_ASYM"], [9, "id0"]], "entropy_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.ENTROPY_SYM"], [9, "id11"]], "entropy_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.ENTROPY_SYM"], [9, "id1"]], "int8 (furiosa.quantizer.tensortype attribute)": [[9, "furiosa.quantizer.TensorType.INT8"]], "int8 (furiosa.quantizer.editor.tensortype attribute)": [[9, "furiosa.quantizer.editor.TensorType.INT8"]], "min_max_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MIN_MAX_ASYM"], [9, "id12"]], "min_max_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MIN_MAX_ASYM"], [9, "id2"]], "min_max_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MIN_MAX_SYM"], [9, "id13"]], "min_max_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MIN_MAX_SYM"], [9, "id3"]], "mse_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MSE_ASYM"], [9, "id14"]], "mse_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MSE_ASYM"], [9, "id4"]], "mse_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MSE_SYM"], [9, "id15"]], "mse_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MSE_SYM"], [9, "id5"]], "modeleditor (class in furiosa.quantizer)": [[9, "furiosa.quantizer.ModelEditor"]], "modeleditor (class in furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.ModelEditor"]], "percentile_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.PERCENTILE_ASYM"], [9, "id16"]], "percentile_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.PERCENTILE_ASYM"], [9, "id6"]], "percentile_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.PERCENTILE_SYM"], [9, "id17"]], "percentile_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.PERCENTILE_SYM"], [9, "id7"]], "sqnr_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.SQNR_ASYM"], [9, "id18"]], "sqnr_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.SQNR_ASYM"], [9, "id8"]], "sqnr_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.SQNR_SYM"], [9, "id19"]], "sqnr_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.SQNR_SYM"], [9, "id9"]], "tensortype (class in furiosa.quantizer)": [[9, "furiosa.quantizer.TensorType"]], "tensortype (class in furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.TensorType"]], "uint8 (furiosa.quantizer.tensortype attribute)": [[9, "furiosa.quantizer.TensorType.UINT8"]], "uint8 (furiosa.quantizer.editor.tensortype attribute)": [[9, "furiosa.quantizer.editor.TensorType.UINT8"]], "collect_data() (furiosa.quantizer.calibrator method)": [[9, "furiosa.quantizer.Calibrator.collect_data"]], "collect_data() (furiosa.quantizer.calibrator.calibrator method)": [[9, "furiosa.quantizer.calibrator.Calibrator.collect_data"]], "compute_range() (furiosa.quantizer.calibrator method)": [[9, "furiosa.quantizer.Calibrator.compute_range"]], "compute_range() (furiosa.quantizer.calibrator.calibrator method)": [[9, "furiosa.quantizer.calibrator.Calibrator.compute_range"]], "convert_input_type() (furiosa.quantizer.modeleditor method)": [[9, "furiosa.quantizer.ModelEditor.convert_input_type"]], "convert_input_type() (furiosa.quantizer.editor.modeleditor method)": [[9, "furiosa.quantizer.editor.ModelEditor.convert_input_type"]], "convert_output_type() (furiosa.quantizer.modeleditor method)": [[9, "furiosa.quantizer.ModelEditor.convert_output_type"]], "convert_output_type() (furiosa.quantizer.editor.modeleditor method)": [[9, "furiosa.quantizer.editor.ModelEditor.convert_output_type"]], "furiosa.quantizer": [[9, "module-furiosa.quantizer"]], "furiosa.quantizer.calibrator": [[9, "module-furiosa.quantizer.calibrator"]], "furiosa.quantizer.editor": [[9, "module-furiosa.quantizer.editor"]], "get_output_names() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.get_output_names"]], "get_output_names() (in module furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.get_output_names"]], "get_pure_input_names() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.get_pure_input_names"]], "get_pure_input_names() (in module furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.get_pure_input_names"]], "quantize() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.quantize"]], "all (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.All"]], "apiclientinitfailed": [[10, "furiosa.runtime.errors.ApiClientInitFailed"]], "asyncsession (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.AsyncSession"]], "axis (class in furiosa.runtime)": [[10, "furiosa.runtime.Axis"]], "batch (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.BATCH"]], "bfloat16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.BFLOAT16"]], "channel (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.CHANNEL"]], "chrometrace (furiosa.runtime.profiler.recordformat property)": [[10, "furiosa.runtime.profiler.RecordFormat.ChromeTrace"]], "compilationfailed": [[10, "furiosa.runtime.errors.CompilationFailed"]], "completionqueue (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.CompletionQueue"]], "cpu (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.Cpu"]], "datatype (class in furiosa.runtime)": [[10, "furiosa.runtime.DataType"]], "devicebusy": [[10, "furiosa.runtime.errors.DeviceBusy"]], "float16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.FLOAT16"]], "float32 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.FLOAT32"]], "furiosa_devices": [[10, "envvar-FURIOSA_DEVICES"], [10, "index-1"]], "furiosa_log_level": [[10, "envvar-FURIOSA_LOG_LEVEL"]], "furiosaruntimeerror": [[10, "furiosa.runtime.FuriosaRuntimeError"]], "furiosaruntimewarning": [[10, "furiosa.runtime.FuriosaRuntimeWarning"]], "height (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.HEIGHT"]], "int16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT16"]], "int32 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT32"]], "int64 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT64"]], "int8 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT8"]], "incompatibleapiclienterror": [[10, "furiosa.runtime.errors.IncompatibleApiClientError"]], "incompatiblemodel": [[10, "furiosa.runtime.errors.IncompatibleModel"]], "input queue": [[10, "term-Input-queue"]], "internalerror": [[10, "furiosa.runtime.errors.InternalError"]], "invalidcompilerconfig": [[10, "furiosa.runtime.errors.InvalidCompilerConfig"]], "invalidinput": [[10, "furiosa.runtime.errors.InvalidInput"]], "invalidsessionoption": [[10, "furiosa.runtime.errors.InvalidSessionOption"]], "invalidyamlexception": [[10, "furiosa.runtime.errors.InvalidYamlException"]], "model (class in furiosa.runtime)": [[10, "furiosa.runtime.Model"]], "modelsource (class in furiosa.runtime)": [[10, "furiosa.runtime.ModelSource"]], "npu_devname": [[10, "envvar-NPU_DEVNAME"], [10, "index-0"]], "noapikeyexception": [[10, "furiosa.runtime.errors.NoApiKeyException"]], "npu (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.Npu"]], "output queue": [[10, "term-Output-queue"]], "pandasdataframe (furiosa.runtime.profiler.recordformat property)": [[10, "furiosa.runtime.profiler.RecordFormat.PandasDataFrame"]], "queuewaittimeout": [[10, "furiosa.runtime.errors.QueueWaitTimeout"]], "queues": [[10, "term-Queues"]], "receiver (class in furiosa.runtime)": [[10, "furiosa.runtime.Receiver"]], "receiver (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Receiver"]], "recordformat (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.RecordFormat"]], "resource (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.Resource"]], "runner (class in furiosa.runtime)": [[10, "furiosa.runtime.Runner"]], "runner (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Runner"]], "runners": [[10, "term-Runners"]], "runtime (class in furiosa.runtime)": [[10, "furiosa.runtime.Runtime"]], "runtime (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Runtime"]], "session (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.Session"]], "sessionclosed": [[10, "furiosa.runtime.errors.SessionClosed"]], "sessionterminated": [[10, "furiosa.runtime.errors.SessionTerminated"]], "submitter (class in furiosa.runtime)": [[10, "furiosa.runtime.Submitter"]], "submitter (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Submitter"]], "tensor (class in furiosa.runtime)": [[10, "furiosa.runtime.Tensor"]], "tensor (class in furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.Tensor"]], "tensorarray (class in furiosa.runtime)": [[10, "furiosa.runtime.TensorArray"]], "tensorarray (class in furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.TensorArray"]], "tensordesc (class in furiosa.runtime)": [[10, "furiosa.runtime.TensorDesc"]], "tensornamenotfound": [[10, "furiosa.runtime.errors.TensorNameNotFound"]], "uint8 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.UINT8"]], "unknown (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.UNKNOWN"]], "unsupporteddatatype": [[10, "furiosa.runtime.errors.UnsupportedDataType"]], "unsupportedfeature": [[10, "furiosa.runtime.errors.UnsupportedFeature"]], "unsupportedtensortype": [[10, "furiosa.runtime.errors.UnsupportedTensorType"]], "width (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.WIDTH"]], "allocate() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.allocate"]], "axis() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.axis"]], "close() (furiosa.runtime.receiver method)": [[10, "furiosa.runtime.Receiver.close"]], "close() (furiosa.runtime.runner method)": [[10, "furiosa.runtime.Runner.close"]], "close() (furiosa.runtime.runtime method)": [[10, "furiosa.runtime.Runtime.close"]], "close() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.close"]], "close() (furiosa.runtime.session.asyncsession method)": [[10, "furiosa.runtime.session.AsyncSession.close"]], "close() (furiosa.runtime.session.completionqueue method)": [[10, "furiosa.runtime.session.CompletionQueue.close"]], "close() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.close"]], "copy_from() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.copy_from"]], "create() (in module furiosa.runtime.session)": [[10, "furiosa.runtime.session.create"]], "create_async() (in module furiosa.runtime.session)": [[10, "furiosa.runtime.session.create_async"]], "create_queue() (furiosa.runtime.furiosa.runtime.runtime method)": [[10, "furiosa.runtime.furiosa.runtime.Runtime.create_queue"]], "create_queue() (furiosa.runtime.sync.furiosa.runtime.sync.runtime method)": [[10, "furiosa.runtime.sync.furiosa.runtime.sync.Runtime.create_queue"]], "create_queue() (in module furiosa.runtime)": [[10, "furiosa.runtime.create_queue"]], "create_queue() (in module furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.create_queue"]], "create_runner() (furiosa.runtime.furiosa.runtime.runtime method)": [[10, "furiosa.runtime.furiosa.runtime.Runtime.create_runner"]], "create_runner() (furiosa.runtime.sync.furiosa.runtime.sync.runtime method)": [[10, "furiosa.runtime.sync.furiosa.runtime.sync.Runtime.create_runner"]], "create_runner() (in module furiosa.runtime)": [[10, "furiosa.runtime.create_runner"]], "create_runner() (in module furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.create_runner"]], "current_npu_device() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.current_npu_device"]], "dim() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.dim"]], "dtype (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.dtype"]], "environment variable": [[10, "envvar-FURIOSA_DEVICES"], [10, "envvar-FURIOSA_LOG_LEVEL"], [10, "envvar-NPU_DEVNAME"], [10, "index-0"], [10, "index-1"]], "export_chrome_trace() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.export_chrome_trace"]], "format (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.format"]], "full_version() (in module furiosa.runtime)": [[10, "furiosa.runtime.full_version"]], "furiosa.runtime": [[10, "module-furiosa.runtime"]], "furiosa.runtime.compiler": [[10, "module-furiosa.runtime.compiler"]], "furiosa.runtime.consts": [[10, "module-furiosa.runtime.consts"]], "furiosa.runtime.envs": [[10, "module-furiosa.runtime.envs"]], "furiosa.runtime.errors": [[10, "module-furiosa.runtime.errors"]], "furiosa.runtime.model": [[10, "module-furiosa.runtime.model"]], "furiosa.runtime.profiler": [[10, "module-furiosa.runtime.profiler"]], "furiosa.runtime.session": [[10, "module-furiosa.runtime.session"]], "furiosa.runtime.tensor": [[10, "module-furiosa.runtime.tensor"]], "generate_compiler_log_path() (in module furiosa.runtime.compiler)": [[10, "furiosa.runtime.compiler.generate_compiler_log_path"]], "get_cpu_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_cpu_pandas_dataframe"]], "get_npu_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_npu_pandas_dataframe"]], "get_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_pandas_dataframe"]], "get_pandas_dataframe_with_filter() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_pandas_dataframe_with_filter"]], "input() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.input"]], "input_num (furiosa.runtime.model property)": [[10, "furiosa.runtime.Model.input_num"]], "inputs() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.inputs"]], "is_compile_log_enabled() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.is_compile_log_enabled"]], "is_empty() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.is_empty"]], "length (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.length"]], "log_dir() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.log_dir"]], "model (furiosa.runtime.receiver property)": [[10, "furiosa.runtime.Receiver.model"]], "model (furiosa.runtime.runner property)": [[10, "furiosa.runtime.Runner.model"]], "model (furiosa.runtime.submitter property)": [[10, "furiosa.runtime.Submitter.model"]], "model (furiosa.runtime.session.asyncsession property)": [[10, "furiosa.runtime.session.AsyncSession.model"]], "model (furiosa.runtime.session.completionqueue property)": [[10, "furiosa.runtime.session.CompletionQueue.model"]], "model (furiosa.runtime.session.session property)": [[10, "furiosa.runtime.session.Session.model"]], "name (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.name"]], "ndim (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.ndim"]], "numpy (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.numpy"]], "numpy() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.numpy"]], "numpy() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.numpy"]], "numpy_dtype (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.numpy_dtype"]], "numpy_dtype (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.numpy_dtype"]], "numpy_dtype (furiosa.runtime.tensor.tensor property)": [[10, "furiosa.runtime.tensor.Tensor.numpy_dtype"]], "numpy_dtype() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.numpy_dtype"]], "output() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.output"]], "output_num (furiosa.runtime.model property)": [[10, "furiosa.runtime.Model.output_num"]], "outputs() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.outputs"]], "pause() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.pause"]], "print_external_operators() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_external_operators"]], "print_inferences() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_inferences"]], "print_npu_executions() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_npu_executions"]], "print_npu_operators() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_npu_operators"]], "print_summary() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.print_summary"]], "print_summary() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_summary"]], "profile (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.profile"]], "profiler_output() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.profiler_output"]], "rand() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.rand"]], "record() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.record"]], "recv() (furiosa.runtime.receiver method)": [[10, "furiosa.runtime.Receiver.recv"]], "recv() (furiosa.runtime.session.completionqueue method)": [[10, "furiosa.runtime.session.CompletionQueue.recv"]], "recv() (furiosa.runtime.sync.receiver method)": [[10, "furiosa.runtime.sync.Receiver.recv"]], "resume() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.resume"]], "run() (furiosa.runtime.runner method)": [[10, "furiosa.runtime.Runner.run"]], "run() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.run"]], "run_with() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.run_with"]], "shape (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.shape"]], "shape (furiosa.runtime.tensor.tensor property)": [[10, "furiosa.runtime.tensor.Tensor.shape"]], "size (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.size"]], "stride() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.stride"]], "submit() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.submit"]], "submit() (furiosa.runtime.session.asyncsession method)": [[10, "furiosa.runtime.session.AsyncSession.submit"]], "summary() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.summary"]], "view() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.view"]], "view() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.view"]], "zeros() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.zeros"]], "modelnotfound": [[11, "furiosa.server.errors.ModelNotFound"]], "modelservererror": [[11, "furiosa.server.errors.ModelServerError"]], "furiosa.server.errors": [[11, "module-furiosa.server.errors"]], "furiosa.server.utils": [[19, "module-furiosa.server.utils"]]}})
\ No newline at end of file
+Search.setIndex({"docnames": ["api/python/furiosa.common", "api/python/furiosa.common.transport", "api/python/furiosa.optimizer", "api/python/furiosa.optimizer.frontend", "api/python/furiosa.optimizer.frontend.onnx", "api/python/furiosa.optimizer.frontend.onnx.transformer", "api/python/furiosa.optimizer.frontend.onnx.transformer.experimental", "api/python/furiosa.optimizer.frontend.onnx.utils", "api/python/furiosa.optimizer.interfaces", "api/python/furiosa.quantizer", "api/python/furiosa.runtime", "api/python/furiosa.server", "api/python/furiosa.server.api", "api/python/furiosa.server.api.cli", "api/python/furiosa.server.api.grpc", "api/python/furiosa.server.api.grpc.generated", "api/python/furiosa.server.api.rest", "api/python/furiosa.server.handlers", "api/python/furiosa.server.types", "api/python/furiosa.server.utils", "api/python/furiosa.serving", "api/python/furiosa.serving.apps", "api/python/furiosa.serving.processors", "api/python/modules", "customer-support/bugs", "index", "npu/intro", "npu/supported_operators", "releases/0.5.0", "releases/0.6.0", "releases/0.7.0", "releases/0.8.0", "releases/0.9.0", "software/c-sdk", "software/cli", "software/compiler", "software/installation", "software/intro", "software/kubernetes_support", "software/profiler", "software/python-sdk", "software/quantization", "software/references", "software/serving", "software/tutorials"], "filenames": ["api/python/furiosa.common.rst", "api/python/furiosa.common.transport.rst", "api/python/furiosa.optimizer.rst", "api/python/furiosa.optimizer.frontend.rst", "api/python/furiosa.optimizer.frontend.onnx.rst", "api/python/furiosa.optimizer.frontend.onnx.transformer.rst", "api/python/furiosa.optimizer.frontend.onnx.transformer.experimental.rst", "api/python/furiosa.optimizer.frontend.onnx.utils.rst", "api/python/furiosa.optimizer.interfaces.rst", "api/python/furiosa.quantizer.rst", "api/python/furiosa.runtime.rst", "api/python/furiosa.server.rst", "api/python/furiosa.server.api.rst", "api/python/furiosa.server.api.cli.rst", "api/python/furiosa.server.api.grpc.rst", "api/python/furiosa.server.api.grpc.generated.rst", "api/python/furiosa.server.api.rest.rst", "api/python/furiosa.server.handlers.rst", "api/python/furiosa.server.types.rst", "api/python/furiosa.server.utils.rst", "api/python/furiosa.serving.rst", "api/python/furiosa.serving.apps.rst", "api/python/furiosa.serving.processors.rst", "api/python/modules.rst", "customer-support/bugs.rst", "index.rst", "npu/intro.rst", "npu/supported_operators.rst", "releases/0.5.0.rst", "releases/0.6.0.rst", "releases/0.7.0.rst", "releases/0.8.0.rst", "releases/0.9.0.rst", "software/c-sdk.rst", "software/cli.rst", "software/compiler.rst", "software/installation.rst", "software/intro.rst", "software/kubernetes_support.rst", "software/profiler.rst", "software/python-sdk.rst", "software/quantization.rst", "software/references.rst", "software/serving.rst", "software/tutorials.rst"], "titles": ["furiosa.common package", "furiosa.common.transport package", "furiosa.optimizer package", "furiosa.optimizer.frontend package", "furiosa.optimizer.frontend.onnx package", "furiosa.optimizer.frontend.onnx.transformer package", "furiosa.optimizer.frontend.onnx.transformer.experimental package", "furiosa.optimizer.frontend.onnx.utils package", "furiosa.optimizer.interfaces package", "furiosa.quantizer package", "furiosa.runtime package", "furiosa.server package", "furiosa.server.api package", "furiosa.server.api.cli package", "furiosa.server.api.grpc package", "furiosa.server.api.grpc.generated package", "furiosa.server.api.rest package", "furiosa.server.handlers package", "furiosa.server.types package", "furiosa.server.utils namespace", "furiosa.serving package", "furiosa.serving.apps package", "furiosa.serving.processors package", "Python SDK", "Bug Report", "FuriosaAI NPU & SDK 0.10.0 Documents", "FuriosaAI NPU", "List of Supported Operators for NPU Acceleration", "Release Notes - 0.5.0", "Release Notes - 0.6.0", "Release Notes - 0.7.0", "Release Notes - 0.8.0", "Release Notes - 0.9.0", "C SDK installation and user guide", "Command Line Tools", "Compiler", "Driver, Firmware, and Runtime Installation", "FuriosaAI SW Stack Introduction", "Kubernetes Support", "Performance Profiling", "Python SDK installation and user guide", "Model Quantization", "References", "Model Server (Serving Framework)", "Tutorial and Code Examples"], "terms": {"transport": [0, 23, 36], "base": [0, 5, 6, 7, 8, 9, 10, 11, 20, 23, 25, 31, 33, 36, 37, 38, 41, 43, 44], "file": [0, 10, 23, 24, 28, 29, 33, 34, 36, 37, 38, 39], "github": [0, 6, 7, 23, 29, 39, 40, 43], "http": [0, 6, 7, 23, 24, 28, 36, 38, 39, 40, 43], "s3": [0, 23, 31], "except": [0, 1, 10, 11], "furiosaerror": [0, 10], "messag": [0, 10, 24, 43], "str": [0, 1, 2, 4, 5, 7, 9, 10, 11], "gener": [0, 8, 10, 12, 14, 23, 26, 28, 33, 34, 35, 37, 39, 40, 41], "caus": [0, 10, 39], "runtim": [0, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 39, 43], "is_err": 0, "err": 0, "c_int": 0, "int": [0, 2, 4, 5, 6, 7, 10, 31], "bool": [0, 1, 5, 6, 7, 9, 10], "is_ok": 0, "class": [0, 1, 5, 6, 7, 8, 9, 10, 31], "loglevel": [0, 43], "valu": [0, 5, 9, 10, 24, 28, 30, 35, 38], "intenum": [0, 9], "python": [0, 10, 24, 25, 28, 31, 32, 33, 34, 35, 39, 41, 42, 43, 44], "object": [0, 5, 7, 9, 23, 25, 26, 31, 37, 44], "correspondnig": 0, "nux_log_level_t": 0, "nux": [0, 30, 31, 43], "c": [0, 5, 10, 24, 25, 28, 30, 31, 32, 34, 38, 42, 43], "api": [0, 11, 25, 29, 32, 33, 35, 36, 39, 43, 44], "1": [0, 5, 9, 10, 24, 26, 28, 29, 30, 31, 32, 34, 35, 36, 39, 43], "info": [0, 5, 10, 24, 28, 31, 32, 38, 43], "3": [0, 9, 10, 24, 28, 30, 31, 34, 39, 40, 43], "off": [0, 10], "0": [0, 5, 9, 10, 24, 26, 27, 34, 35, 36, 38, 39, 40, 41, 43], "warn": [0, 10, 29, 30, 43], "2": [0, 5, 9, 10, 26, 27, 28, 29, 30, 31, 32, 34, 35, 37, 39], "find_global_lib_path": 0, "libnam": 0, "find_native_lib_path": 0, "none": [0, 2, 4, 5, 6, 7, 9, 10, 11, 32, 34, 38, 40], "find": [0, 1, 24, 28, 29, 30, 31, 32, 33, 35, 38, 39, 40, 43], "librari": [0, 24, 30, 31, 32, 33, 34, 36, 37, 40], "path": [0, 1, 10, 24, 28, 30, 33, 34, 35, 38, 39, 43], "accord": [0, 33, 36, 37, 40], "follow": [0, 1, 5, 10, 24, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "prioriti": 0, "If": [0, 1, 6, 9, 10, 24, 27, 28, 29, 31, 32, 34, 35, 36, 37, 38, 39, 40, 43], "environ": [0, 10, 28, 30, 31, 34, 35, 36, 37, 38, 43], "variabl": [0, 10, 28, 30, 31, 35], "ld_library_path": 0, "i": [0, 1, 5, 9, 10, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "set": [0, 10, 23, 30, 31, 32, 33, 34, 35, 38, 39, 40], "thi": [0, 1, 5, 9, 10, 23, 24, 25, 27, 28, 31, 34, 35, 36, 37, 38, 39, 40, 41, 43], "function": [0, 5, 10, 28, 29, 30, 31, 32, 34, 35, 37, 38, 39, 40], "tri": [0, 10], "found": [0, 26, 28, 29, 30, 31, 32, 34, 35, 37, 40], "from": [0, 1, 6, 9, 10, 24, 25, 28, 29, 30, 31, 32, 34, 35, 36, 39, 40, 41, 43, 44], "otherwis": [0, 10], "global": 0, "usr": [0, 24, 31, 33, 34, 38, 39], "lib": [0, 33, 38], "local": [0, 1, 24, 28, 30, 31, 34, 35, 43], "still": [0, 10], "cannot": [0, 10, 30, 33, 39], "return": [0, 1, 9, 10, 29, 31, 43], "find_native_lib": 0, "register_hook": 0, "type": [0, 1, 9, 11, 23, 26, 28, 29, 30, 35, 37, 38, 41], "callabl": [0, 5], "__register_common_capi": 0, "interfac": [0, 2, 23, 26, 40], "embed": [0, 37], "after": [0, 5, 10, 27, 29, 34, 35, 38, 40], "load": [0, 10, 24, 31, 34, 37, 39], "call": [0, 9, 10, 33, 35, 36, 37, 43], "cifac": 0, "find_user_lib_path": 0, "asynchron": [0, 10, 28, 37], "f": [0, 10, 28, 38], "replac": [0, 7, 10, 36, 41], "sync": [0, 10, 35, 39], "async": [0, 1, 10, 29, 31], "us": [0, 5, 9, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 41, 43, 44], "aysncio": 0, "pool": [0, 31], "synchron": [0, 10, 29, 31], "run": [0, 10, 27, 28, 30, 31, 34, 35, 36, 37, 38, 39, 40], "place": 0, "result": [0, 10, 26, 29, 30, 31, 32, 34, 35, 39], "furiosaversioninfo": 0, "version": [0, 10, 11, 24, 27, 28, 34, 35, 37, 38, 40], "eprint": 0, "arg": [0, 34, 38], "kwarg": 0, "get_sdk_vers": 0, "git": [0, 39, 40, 43], "commit": [0, 28, 29, 30, 38], "hash": [0, 38], "repres": [0, 9, 10], "current": [0, 10, 25, 33, 34, 36, 37, 38, 40], "applic": [0, 23, 31, 33, 35, 37, 38, 39, 40, 43], "removeprefix": 0, "word": 0, "prefix": 0, "9": [0, 9, 10, 24, 28, 30, 31, 34, 36, 39, 43], "see": [0, 10, 24, 29, 32, 34, 35, 38, 40], "doc": 0, "org": 0, "stdtype": 0, "html": [0, 34], "sdk": [0, 10, 24, 26, 27, 28, 34, 35, 38, 39, 41, 42, 43, 44], "abc": 1, "read": [1, 10, 35, 38, 40], "download": [1, 36, 40], "variou": [1, 26, 30, 32, 36, 37, 38, 40, 41, 43], "sourc": [1, 35, 36, 38, 39, 40, 43], "properti": [1, 10, 30, 31, 36], "cache_directori": 1, "cach": 1, "directori": [1, 10, 30, 33, 35, 38, 39], "save": [1, 24, 28, 30, 34, 35, 39, 43], "abstract": [1, 30, 31, 32], "uri": 1, "destin": 1, "paramet": [1, 9, 10, 28, 30], "data": [1, 5, 9, 10, 30, 31, 32, 34, 35, 37, 38, 39, 41, 43], "name": [1, 5, 9, 10, 11, 27, 28, 30, 31, 32, 34, 35, 36, 38, 39, 40, 43], "locat": [1, 29, 31, 33, 35], "static": [1, 5, 24, 28, 33], "is_support": 1, "decid": [1, 32], "whether": [1, 9, 10, 30, 32, 34, 38, 40], "support": [1, 23, 25, 26, 29, 31, 33, 34, 35, 41, 43], "specifi": [1, 9, 10, 30, 31, 32, 34, 35, 37, 39, 40, 43], "model": [1, 2, 4, 5, 6, 7, 8, 9, 23, 24, 26, 27, 29, 30, 32, 39, 42, 44], "dispatch": 1, "method": [1, 9, 10, 37, 39, 40, 41, 43], "signatur": 1, "transporterror": 1, "msg": [1, 11], "transportnotfound": 1, "list": [1, 2, 4, 5, 6, 7, 9, 10, 25, 28, 29, 30, 31, 35, 36, 37], "filetransport": 1, "check": [1, 5, 7, 24, 28, 29, 34, 36, 38, 40, 43], "ha": [1, 5, 10, 26, 29, 30, 31, 32, 34, 38], "valid": [1, 10, 29, 34, 35], "scheme": [1, 26], "e": [1, 10, 29, 34, 36, 38], "g": [1, 10, 29, 34], "just": [1, 10, 30, 43], "copi": [1, 10, 43], "githubtransport": 1, "header": [1, 33], "dict": [1, 5, 9, 10, 29], "param": 1, "httptransport": 1, "repositori": [1, 20, 23, 29, 30, 31, 32, 40], "url": [1, 31, 35, 43], "com": [1, 6, 7, 28, 36, 38, 39, 40, 43], "pars": 1, "tupl": [1, 9, 10], "ai": [1, 10, 24, 25, 28, 36, 38, 39, 40, 43], "main": [1, 10, 11, 12, 23, 28], "s3transport": 1, "network": [1, 36, 41], "util": [1, 3, 4, 9, 11, 23, 26, 29, 31, 32, 34, 35, 37, 40], "fetch": 1, "which": [1, 10, 26, 31, 34, 35, 37, 38, 39], "rais": [1, 10], "all": [1, 5, 10, 26, 28, 31, 34, 35, 40, 43], "avail": [1, 10, 31, 34, 36, 37, 38, 40], "ar": [1, 5, 10, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 43], "is_rel": 1, "rel": [1, 33], "we": [1, 25, 28, 32, 33, 36, 37, 40], "assum": [1, 6, 10], "": [1, 5, 7, 9, 10, 26, 28, 32, 34, 35, 37, 38, 39, 40, 41], "you": [1, 10, 24, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "should": [1, 10, 24, 26, 30, 34], "iter": [1, 5, 6, 9, 10], "frontend": [2, 23], "onnx": [2, 3, 9, 10, 23, 24, 26, 27, 28, 29, 30, 31, 34, 35, 37, 41, 43], "submodul": [2, 3, 4, 10, 12, 23], "transform": [2, 3, 4, 7, 23], "furiosaai": [2, 9, 24, 27, 28, 29, 31, 33, 34, 35, 36, 38, 43], "optimize_model": [2, 4, 32], "modelproto": [2, 4, 5, 6, 7, 9], "input_shap": [2, 4, 5, 7, 31, 39], "map": [2, 4, 5, 7, 9, 41], "opset_vers": [2, 4, 7], "13": [2, 4, 5, 7, 28, 35, 41], "convert_negative_pads_to_slic": [3, 4], "convert_prelu_to_relu": [3, 4], "fuse_batchnorm": [3, 4], "fuse_gather_matmul": [3, 4], "infer_squeeze_ax": [3, 4], "polish_model": [3, 4], "check_model": [3, 4], "inference_shap": [3, 4], "version_check": [3, 4], "experiment": [4, 5, 30, 35], "eliminate_detection_postprocess": [4, 5], "embedding_bag_port": [4, 5], "fuse_div_for_bert": [4, 5], "reify_conv_for_bert": [4, 5], "convertnegativepadstoslic": 5, "pattern_1": 5, "name_nod": [5, 6], "true": [5, 6, 7, 10, 38, 43], "onnxtransform": [5, 6], "prev": 5, "pad": [5, 27, 30], "next": [5, 33, 38, 40], "slice": [5, 27, 29, 30], "input": [5, 6, 9, 23, 24, 28, 30, 31, 34, 35, 37, 39, 41, 43], "an": [5, 7, 9, 24, 26, 28, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 43], "initi": [5, 9, 10, 24, 30, 43], "least": [5, 35], "one": [5, 9, 10, 35, 37, 38, 39, 40], "neg": 5, "sum": 5, "axi": [5, 10, 27, 29, 30], "doe": [5, 10, 30, 40, 43], "exce": 5, "correspond": [5, 10], "invalid": [5, 35], "absolut": 5, "each": [5, 10, 26, 30, 31, 32, 34, 37, 38, 39], "less": 5, "than": [5, 10, 27, 30, 32, 33, 40], "lead": [5, 32, 35], "nan": 5, "output": [5, 9, 23, 24, 28, 30, 31, 32, 34, 35, 37, 38, 39, 41], "make_new_init_and_vi": 5, "matched_nod": 5, "nodeproto": [5, 6], "valueinfoproto": 5, "tensorproto": 5, "make_new_nod": 5, "pattern_condition_check": [5, 6], "nodes_to_check": [5, 6], "pattern_match": [5, 6], "base_nod": [5, 6], "pattern_to_match": 5, "pattern_2": 5, "zero": [5, 10], "convertprelutorelu": 5, "prelu": [5, 31], "x": [5, 10, 38], "slope": 5, "relu": [5, 27], "make_new_init": 5, "make_new_vi": 5, "fusebatchnorm": 5, "conv": [5, 6, 27, 30, 31], "batchnorm": [5, 27, 28, 30], "convtranspos": [5, 27, 28, 31], "pattern_3": 5, "mul": [5, 27, 31], "add": [5, 6, 27, 28, 29, 31, 34, 36, 37, 38], "onli": [5, 6, 10, 25, 27, 28, 29, 30, 33, 35, 37, 38], "pattern_4": 5, "op_typ": 5, "fusegathermatmul": 5, "gather": [5, 30, 31], "matmul": [5, 6, 27, 30, 31], "must": [5, 33, 36, 37, 38, 39], "have": [5, 9, 10, 24, 28, 30, 31, 33, 34, 35, 36, 38, 40], "exactli": 5, "defin": [5, 7, 10], "graph": [5, 6, 34, 35, 41], "weight": [5, 27, 30, 37, 41], "data_typ": 5, "float": [5, 9, 10, 26, 37, 41], "rank": 5, "infersqueezeax": 5, "squeez": [5, 30], "ax": [5, 10, 30], "attribut": 5, "fill": [5, 10], "opset": [5, 7, 28, 30, 35], "exist": [5, 10, 28, 32, 40], "shape": [5, 10, 24, 28, 30, 37, 39, 43], "infer": [5, 25, 26, 29, 30, 31, 34, 35, 37, 39, 40, 43, 44], "polishmodel": 5, "essenti": 5, "check_value_info": 5, "eliminate_initializer_from_graph_input": 5, "eliminate_unused_initi": 5, "elimin": [5, 26, 32], "everi": [5, 34], "node": [5, 6, 28, 37], "regardless": [5, 30], "ani": [5, 10, 30, 34, 40], "field": [5, 40], "thei": [5, 10, 27, 33, 37, 39], "eliminate_unused_input": 5, "eliminate_unused_output": 5, "eliminate_unused_proto": 5, "eliminate_unused_value_info": 5, "fix_batch_size_as_on": 5, "fix": [5, 28, 29, 31, 32], "batch_siz": [5, 6, 10, 24, 28, 30, 31, 35], "dim_param": 5, "given": [5, 10, 24, 30, 35, 36], "fixed_point": 5, "t": [5, 6, 8, 10, 30, 34], "get_attribut": 5, "attr": 5, "attributeproto": 5, "attr_nam": 5, "default": [5, 9, 10, 24, 28, 30, 33, 34, 35, 38, 39, 43], "get_node_attribut": 5, "get_node_input_nam": 5, "get_node_output_nam": 5, "is_op_typ": 5, "target_op_typ": 5, "make_initializer_name_uniqu": 5, "make_unhashables_uniqu": 5, "rebuild_model": 5, "new_nod": 5, "renam": [5, 29], "bridge_disconnected_nod": 5, "node_0": 5, "next_nod": 5, "new_input": 5, "For": [5, 10, 27, 32, 35, 36, 37, 38, 39, 40], "chang": [5, 10, 28, 34, 36, 41], "exampl": [5, 10, 24, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 43], "befor": [5, 28, 29, 30, 31, 32, 40], "node_1": 5, "bridg": 5, "assign": 5, "y": [5, 10, 30, 31, 32, 33, 34, 36, 38, 40], "build_optimized_model": 5, "check_runn": [5, 7], "copy_value_info": 5, "find_next_nod": 5, "find_prev_nod": 5, "node_input": 5, "get_data_node_input": 5, "get_init_node_input": 5, "get_initializer_arrai": 5, "get_map_valu": 5, "get_value_info_dtyp": 5, "tensor_nam": [5, 9], "get_value_info_shap": 5, "is_same_shap": 5, "input_1": 5, "input_2": 5, "pop_multiple_optimizer_map": 5, "pop_single_optimizer_map": 5, "transform_to_elimin": 5, "nodes_to_remov": 5, "design": [5, 28, 30, 31, 32, 34, 35, 36, 37, 38, 39], "previou": [5, 35], "them": [5, 10, 25, 26, 32, 37, 40], "b": [5, 34, 35, 38, 41], "remov": [5, 10, 29, 32], "A": [5, 9, 10, 30, 32, 34, 40], "d": [5, 10, 34, 36], "connect": [5, 10, 28, 33], "make": [5, 10, 38, 39, 40], "transform_to_fus": 5, "nodes_to_add": 5, "inits_to_add": 5, "vis_to_add": 5, "traverse_prev_nod": 5, "producer_map_kei": 5, "update_graph_field": 5, "update_multiple_initializer_map": 5, "update_multiple_optimizer_map": 5, "dest_nam": 5, "update_multiple_value_info_map": 5, "value_info": 5, "update_single_initializer_map": 5, "update_single_optimizer_map": 5, "update_single_value_info_map": 5, "eliminatessddetectionpostprocess": 6, "ssd_output": 6, "softmax": [6, 27, 30], "graph_output": 6, "nchw": [6, 10], "embeddingbagpattern": 6, "appli": [6, 10, 28, 38], "dlrm": 6, "export": [6, 10, 26, 35, 39], "embeddingbag": 6, "equival": [6, 10], "torch": 6, "simplifi": [6, 7], "doesn": [6, 10], "work": [6, 10, 34, 37], "becaus": [6, 10, 35], "can": [6, 9, 10, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "handl": [6, 10], "empti": [6, 10], "bag": 6, "pytorch": [6, 26, 29], "blob": 6, "55d479aca5e959c5f2fe3089e162db710bb7632d": 6, "symbolic_opset11": 6, "py": [6, 39], "l836": 6, "l902": 6, "check_condition_3": 6, "embeddingbagport": 6, "128": [6, 27, 29], "fusedivforbert": 6, "some": [6, 10, 24, 27, 28, 30, 35], "bert": 6, "transform_matmul_add": 6, "reifyconvforbert": 6, "well": [7, 10, 28, 29, 31], "execut": [7, 10, 29, 30, 31, 32, 34, 35, 37, 38, 39, 43], "onnxruntim": 7, "inferenceshap": 7, "former": 7, "onnx_simplifi": 7, "daquexian": 7, "checkvers": 7, "convert": [7, 9, 10, 11, 12, 23, 26, 29, 31, 32, 40, 41], "12": [7, 27, 28, 30, 31, 34, 38, 39], "opset_bound": 7, "calibrationmethod": [9, 32], "min_max_asym": [9, 32], "min": [9, 10, 32, 34, 39, 41], "max": [9, 10, 26, 32, 34, 39, 41], "asymmetr": [9, 32, 41], "min_max_sym": [9, 32], "symmetr": [9, 41], "entropy_asym": [9, 32], "entropi": [9, 32, 41], "aymmetr": 9, "entropy_sym": [9, 32], "percentile_asym": [9, 32], "percentil": [9, 10, 32, 34, 39, 41], "percentile_sym": [9, 32], "mse_asym": [9, 32], "mean": [9, 10, 27, 32, 34, 39], "squar": [9, 32], "error": [9, 23, 24, 32, 35, 38, 40, 43], "mse": [9, 41], "mse_sym": [9, 32], "sqnr_asym": [9, 32], "signal": [9, 10, 24, 32], "nois": [9, 32], "ratio": [9, 32], "sqnr": [9, 41], "sqnr_sym": [9, 32], "6": [9, 24, 27, 32, 34, 39], "7": [9, 32, 38], "4": [9, 10, 31, 32, 34, 39], "5": [9, 10, 24, 26, 29, 32, 34, 35, 39, 41], "8": [9, 10, 24, 26, 28, 30, 32, 34, 39, 40, 41, 43], "byte": [9, 10], "calibration_method": 9, "percentag": [9, 34], "99": [9, 10, 32, 34, 39, 41], "collect": [9, 10, 24, 34], "tensor": [9, 23, 24, 28, 29, 30, 31, 32, 37], "comput": [9, 26, 27, 34], "rang": [9, 10, 31, 32, 39], "collect_data": [9, 32], "calibration_dataset": 9, "sequenc": 9, "ndarrai": [9, 10, 29, 31], "multipl": [9, 10, 29, 31, 37, 38, 43], "time": [9, 10, 26, 29, 30, 31, 32, 34, 38, 39, 40], "numpi": [9, 10, 29, 31, 32, 39, 43], "provid": [9, 10, 26, 29, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 43], "compute_rang": [9, 32], "verbos": [9, 10, 34, 35], "fals": [9, 10], "estim": 9, "basi": 9, "show": [9, 30, 32, 35, 36, 38, 43], "progress": [9, 10], "bar": [9, 39], "dictionari": 9, "modeleditor": 9, "manipul": 9, "convert_input_typ": 9, "tensor_typ": 9, "tensortyp": 9, "element": [9, 10, 26, 32], "The": [9, 10, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 43], "desir": [9, 10, 35, 38], "convert_output_typ": 9, "tensor_rang": 9, "option": [9, 10, 30, 31, 32, 33, 34, 35, 36, 38, 40, 43], "new": [9, 10, 32, 33], "origin": [9, 10, 41], "retain": [9, 10, 26], "enumer": [9, 10], "convert_": 9, "_type": 9, "int8": [9, 10, 26, 37, 41], "uint8": [9, 10, 28, 32, 39, 43], "get_output_nam": 9, "get_pure_input_nam": 9, "associ": [9, 10], "qunat": 9, "tensor_name_to_rang": 9, "its": [9, 10, 25, 30, 38, 40, 43], "serial": [9, 34], "incorpor": 9, "inform": [9, 24, 28, 34, 37, 38, 40], "high": [10, 26, 31, 37, 39], "level": [10, 29, 33, 35, 37, 43], "npu": [10, 24, 29, 31, 32, 33, 34, 37, 39, 40, 41, 43], "divid": [10, 31], "three": [10, 41], "wa": [10, 28, 32, 34], "first": [10, 26, 31, 34, 35, 37, 41, 43], "introduc": [10, 33, 37], "furiosart": [10, 33], "10": [10, 24, 28, 30, 31, 32, 34, 35, 38, 39, 40], "backward": 10, "compat": [10, 29, 31, 38, 40, 43], "slate": 10, "futur": [10, 33], "releas": [10, 33], "more": [10, 28, 29, 31, 32, 33, 35, 37, 38, 40, 43], "further": [10, 41], "contain": [10, 24, 28, 30, 34, 37, 38, 40], "two": [10, 26, 34, 35, 38], "differ": 10, "singl": [10, 26, 28, 31, 38], "activ": [10, 26, 30, 31, 38], "either": [10, 34, 36], "possibl": [10, 35], "task": [10, 26, 29, 37, 38, 39], "thread": [10, 23, 29, 31, 34, 39], "separ": [10, 37, 40], "send": [10, 43], "recv": [10, 36], "respect": [10, 33, 37], "pair": 10, "context": [10, 30], "distinguish": 10, "while": [10, 26, 31, 37], "sent": 10, "deprec": [10, 33], "sinc": [10, 30, 35, 40], "strongli": 10, "discourag": 10, "mani": [10, 31, 32, 37, 38, 39], "histor": 10, "includ": [10, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 38], "full": [10, 26, 29, 32, 34, 38], "As": [10, 30, 31, 32, 33, 41], "report": [10, 25, 28, 34, 35, 39], "larg": [10, 26, 29, 30, 31, 39], "wrapper": [10, 34], "around": 10, "slight": 10, "incompat": 10, "most": [10, 26, 35, 40], "notabl": 10, "lack": 10, "subclass": 10, "those": 10, "intercompat": 10, "mark": [10, 36], "mai": [10, 26, 27, 31, 32, 34, 35, 36, 38, 40], "enabl": [10, 28, 31, 34, 35, 36, 37, 39], "extra": [10, 38], "instal": [10, 25, 28, 29, 30, 35, 37], "forc": 10, "old": 10, "implement": [10, 37], "disabl": [10, 35, 38], "pleas": [10, 24, 25, 28, 32, 34, 36, 43], "note": [10, 34], "With": [10, 26, 28, 31, 32, 37, 39, 40], "full_vers": 10, "nativeerror": 10, "modelsourc": 10, "how": [10, 24, 25, 32, 33, 34, 36, 44], "real": 10, "alia": 10, "like": [10, 43], "string": 10, "arrai": [10, 29], "__bytes__": 10, "raw": [10, 28, 38], "allow": [10, 29, 30, 31, 35, 37, 39, 40, 43], "addit": [10, 27, 31, 32, 34, 37, 38, 39, 40], "format": [10, 26, 30, 31, 32, 34, 35, 37, 38, 39], "restrict": [10, 36], "primari": 10, "represent": 10, "alias": 10, "document": [10, 34, 38], "purpos": [10, 35], "tensorarrai": 10, "itself": 10, "shouldn": 10, "requir": [10, 26, 29, 31, 32, 33, 37, 38, 41, 43], "alwai": [10, 30], "scalar": 10, "dimension": 10, "other": [10, 29, 30, 31, 33, 35, 38, 40, 44], "same": [10, 30, 31, 37, 43], "due": 10, "technic": 10, "reason": [10, 31], "concret": 10, "so": [10, 24, 26, 28, 30, 33, 34, 35, 37, 40, 41, 43], "item": [10, 30, 34, 38], "len": [10, 39], "is_empti": 10, "view": [10, 30, 32, 34, 39], "also": [10, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39], "intern": [10, 31], "were": 10, "here": [10, 33, 35, 36, 37, 40], "numpy_dtyp": 10, "dtype": [10, 28], "contrari": 10, "didn": 10, "mislead": 10, "thu": [10, 39], "instead": [10, 32], "copy_from": 10, "entir": [10, 28, 30], "what": 10, "refer": [10, 24, 27, 32, 33, 34, 35, 37, 38, 40, 41, 43], "memori": [10, 26, 27, 29, 32, 34, 35, 38], "region": 10, "independ": [10, 26, 28, 35], "creat": [10, 24, 28, 30, 31, 34, 35, 36, 37, 39, 43], "previous": [10, 28], "could": [10, 28, 32, 40], "directli": [10, 28, 35, 36, 37, 39, 40, 43], "via": [10, 32, 35, 37], "create_async": [10, 28, 30], "textual": 10, "section": [10, 24, 31, 34, 36, 37, 39, 40], "mainli": 10, "manag": [10, 30, 31, 32, 34, 35, 37, 38], "create_runn": [10, 35, 39], "await": [10, 29, 31], "act": 10, "scope": 10, "subsequ": [10, 34], "Its": [10, 43], "lifetim": [10, 35], "start": [10, 25, 32, 35, 38, 39, 40, 41, 43, 44], "successfulli": [10, 38], "end": [10, 25], "close": [10, 30], "fail": [10, 24, 28], "been": [10, 29, 30, 31, 32, 38, 40], "yet": [10, 30], "wait": [10, 31, 43], "until": [10, 31], "inde": 10, "take": [10, 30, 35, 36, 39], "too": [10, 30], "much": [10, 39], "case": [10, 28, 30], "open": [10, 38, 39], "timeout": [10, 28], "reach": 10, "made": 10, "identifi": [10, 28, 31, 37, 38, 39], "implicit": 10, "arch": [10, 36, 38], "denot": 10, "where": [10, 30], "target": [10, 34], "architectur": [10, 26, 38, 40], "warboi": [10, 25, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 41], "number": [10, 31, 34, 35, 37, 38], "pe": [10, 24, 26, 28, 35, 38, 43], "per": [10, 34], "omit": [10, 35, 38], "index": 10, "indic": 10, "z": 10, "fuse": [10, 26, 30, 35], "intermedi": 10, "occupi": [10, 31, 34], "worth": 10, "mode": [10, 27, 29, 30, 34], "determin": 10, "kernel": [10, 25, 29, 34, 36, 40], "driver": [10, 25, 29, 30, 31, 32, 33, 34, 38, 40, 43], "heavili": 10, "reli": 10, "upon": [10, 36], "especi": [10, 35], "npuxpei": 10, "dev": [10, 28, 31, 32, 33, 34, 36, 38], "aforement": 10, "alloc": [10, 32, 35, 37, 38], "greedi": 10, "manner": [10, 30, 37], "even": [10, 27, 30, 35, 39], "mix": 10, "recommend": [10, 38, 40], "explicit": 10, "furiosa_devic": 10, "preced": 10, "over": [10, 32, 34, 35], "npu_devnam": [10, 28, 38], "never": 10, "overrid": 10, "relev": 10, "affect": 10, "width": 10, "height": [10, 26, 27, 29], "channel": 10, "batch": [10, 26, 27, 30, 34], "unknown": 10, "constant": 10, "known": 10, "abbrevi": 10, "descript": [10, 34, 35, 38, 43], "w": [10, 28, 31, 32, 34, 38, 39], "h": [10, 34, 35], "depth": [10, 29], "convolut": [10, 26, 29], "n": [10, 32, 34, 35, 38, 39, 40], "privat": 10, "Their": 10, "stabl": [10, 29, 30, 36], "datatyp": [10, 43], "v": [10, 34, 35], "constructor": 10, "import": [10, 24, 28, 29, 31, 35, 39, 43], "np": [10, 28, 29, 31, 39, 43], "float32": [10, 30], "float16": 10, "bfloat16": 10, "int16": 10, "int32": 10, "int64": 10, "ieee": 10, "754": 10, "half": [10, 26], "precis": 10, "binary16": 10, "point": [10, 26, 37, 41], "binary32": 10, "bit": [10, 41], "sign": [10, 36], "integ": [10, 30, 43], "16": [10, 24, 26, 28, 32, 39], "32": [10, 24, 26, 28], "64": [10, 26], "unsign": 10, "valueerror": 10, "tensordesc": 10, "describ": [10, 37, 39, 40, 43], "ndim": 10, "dimens": [10, 24, 26, 28], "dim": 10, "idx": 10, "size": [10, 26, 30, 34, 39], "th": 10, "desc": [10, 32], "conceptu": 10, "stride": 10, "It": [10, 26, 29, 30, 33, 35, 37, 39, 40], "distanc": 10, "between": [10, 34, 35, 39, 40], "adjac": 10, "convent": 10, "length": [10, 26], "total": [10, 26, 31, 34, 39], "concaten": 10, "possibli": [10, 32], "input_num": 10, "output_num": 10, "summari": [10, 35, 39], "human": 10, "readabl": 10, "print": [10, 24, 30, 31, 34, 35, 38, 39, 43], "28": [10, 24, 28, 32, 34, 43], "3136": 10, "784": 10, "40": [10, 26, 32, 34, 38, 39], "print_summari": [10, 39], "simpl": [10, 30, 34], "worker_num": [10, 31], "worker": [10, 34, 38], "abov": [10, 24, 28, 33, 36, 37, 38, 39, 40, 43], "implicitli": 10, "couldn": 10, "get": [10, 29, 30, 31, 32, 33, 34, 36, 38, 40, 43], "argument": [10, 30, 32, 34, 35], "about": [10, 28, 31, 32, 34, 35, 37, 40, 43], "num_input": 10, "multipli": 10, "buffer": 10, "modif": [10, 38], "dure": [10, 28, 34, 35, 37, 39], "unexpect": 10, "ensur": [10, 29, 32, 40], "do": [10, 24, 37, 38, 39, 40, 43], "crash": 10, "top": [10, 26, 32, 39], "unspecifi": 10, "subject": 10, "garbag": 10, "through": [10, 28, 30, 31, 34, 35, 36, 37, 38, 39, 41, 43], "compiler_hint": 10, "need": [10, 26, 34, 35, 36, 37, 38, 40, 43], "hint": 10, "now": [10, 28, 30, 31], "keyword": 10, "posit": 10, "accept": [10, 43], "behav": 10, "ident": 10, "against": 10, "oper": [10, 25, 26, 28, 29, 30, 31, 32, 35, 36, 37, 38, 39, 41], "longer": [10, 31], "prefer": 10, "unlik": 10, "unless": 10, "alter": 10, "run_with": [10, 28], "order": [10, 28, 33, 36, 38, 40, 41], "partial": 10, "present": [10, 34, 38], "howev": [10, 35], "few": [10, 35], "submitt": 10, "These": [10, 36, 39], "actual": [10, 35, 37, 39], "process": [10, 26, 27, 29, 30, 32, 34, 35, 37, 38, 39, 40, 41, 43], "hold": 10, "complet": [10, 31, 33, 43], "both": [10, 34, 36], "configur": [10, 26, 28, 31, 33, 34, 35], "finit": 10, "quickli": [10, 31, 34], "block": [10, 31, 37], "create_queu": 10, "input_queue_s": 10, "output_queue_s": 10, "below": [10, 24, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41], "asyncio": [10, 28, 31], "taskgroup": 10, "tg": 10, "create_task": 10, "submit_task": 10, "recv_task": 10, "_": [10, 31, 32, 39], "fresh": 10, "suitabl": 10, "content": [10, 23, 38], "probabl": 10, "yourself": [10, 24, 34], "friendli": 10, "wai": [10, 30, 35], "whenev": 10, "appropri": [10, 30, 35], "immedi": [10, 28, 38, 39], "would": [10, 24, 27, 35], "later": 10, "uuid": [10, 31, 32, 34], "long": [10, 35, 37, 39], "logic": [10, 29], "leak": 10, "remain": [10, 39], "nevertheless": 10, "num_output": 10, "remark": 10, "finish": [10, 25, 32, 34, 41, 44], "handle_output": 10, "usag": [10, 29, 31, 32, 34, 35, 37, 38, 40, 43], "strictli": 10, "alreadi": [10, 30, 34, 36, 38], "loop": 10, "manual": 10, "guarante": 10, "onc": [10, 31, 33, 34, 35, 38, 39, 40, 43], "submiss": 10, "put": 10, "recov": 10, "parallel": 10, "fine": 10, "benefit": 10, "On": [10, 26], "hand": 10, "eventu": 10, "up": [10, 30, 31, 38, 39, 43], "idiomat": 10, "try": [10, 37, 38], "timeouterror": 10, "Not": 10, "built": [10, 24, 26, 28, 34, 43], "out": [10, 34, 36, 40, 43], "notifi": 10, "imposs": 10, "second": [10, 34, 35, 38], "completionqueu": [10, 28], "standard": [10, 26], "asyncsess": 10, "unbound": 10, "In": [10, 26, 28, 31, 32, 33, 35, 36, 37, 38, 41], "facili": 10, "migrat": 10, "continu": 10, "larger": [10, 26, 27, 32, 35], "though": [10, 39], "spawn_thread_to_send_input": 10, "millisecond": 10, "queuewaittimeout": 10, "prone": 10, "potenti": 10, "incompatibil": 10, "basic": [10, 25, 38, 44], "facil": 10, "recordformat": [10, 31, 39], "record": [10, 31, 38, 39], "chrometrac": 10, "pandasdatafram": [10, 31, 39], "resourc": [10, 26, 35, 37, 38], "cpu": [10, 25, 27, 29, 35, 37, 38, 39, 44], "json": [10, 34, 38, 39, 43], "span": 10, "incur": 10, "non": 10, "trivial": 10, "perform": [10, 25, 29, 31, 32, 33, 34, 35, 37, 38, 41], "overhead": [10, 39], "trace": [10, 31, 34, 43], "written": [10, 39], "temporari": 10, "writter": 10, "depend": [10, 26, 29, 30, 31, 35, 38, 40], "chrome": [10, 34, 39], "csv": 10, "profilererorr": 10, "config": [10, 30, 34, 38, 43], "At": [10, 37], "enter": [10, 30, 32, 34, 39], "exit": [10, 43], "within": [10, 36, 37, 39], "child": 10, "profilerrecordobject": 10, "paus": 10, "temporarili": [10, 39], "stop": [10, 32, 34], "minim": [10, 41], "resum": 10, "state": [10, 24, 26, 28, 30, 36, 43], "noth": 10, "arbitrari": 10, "measur": [10, 32, 39], "event": [10, 39], "occur": 10, "interv": [10, 32, 34, 38], "detail": [10, 27, 28, 29, 30, 31, 32, 34, 35, 38, 40, 43], "get_pandas_datafram": [10, 31, 39], "datafram": [10, 31], "look": [10, 33], "trace_id": [10, 31, 39], "parent_span_id": 10, "span_id": [10, 39], "cat": [10, 30, 31, 34, 38], "id": [10, 31, 34, 39], "dram_bas": 10, "pe_index": [10, 39], "execution_index": [10, 39], "instruction_index": 10, "operator_index": [10, 39], "dur": [10, 31, 39], "6ffe9ac3080814bc134ae4c5e58269e0": 10, "0000000000000000": 10, "a61dd01a47ce8de": 10, "1690798389820453606": 10, "1690798390204660478": 10, "35": 10, "na": 10, "384206872": 10, "079f8437488528d5768780162ed59374": 10, "2d18b0e17e760325": 10, "1690798390205840825": 10, "1690798390267819096": 10, "programbinari": 10, "26": 10, "61978271": 10, "fb4610c2fd1be67e63e01ca9169b6fef": 10, "2a092524d04a4077": 10, "1690798390267849007": 10, "1690798390267857471": 10, "allocatedram": 10, "8464": 10, "009b425f06ca0065a64f0586d1a999b0": 10, "cdac229f8d8569d7": 10, "1690798389793627190": 10, "1690798390268011450": 10, "regist": [10, 25, 36, 38], "474384260": 10, "348ee82fdf97fad9f782cc12a58d447d": 10, "59b5a5d06439f9f1": 10, "1690798390270474367": 10, "1690798390270526470": 10, "enqueu": 10, "52103": 10, "27efb2c82a5ac93bed911142e9187c45": 10, "174b38c90d1f7a10": 10, "ff7c4f8798d75b63": 10, "1690798390270558295": 10, "1690798390270570293": 10, "feedinput": 10, "11998": 10, "panda": [10, 31], "get_pandas_dataframe_with_filt": 10, "column": 10, "row": 10, "whose": 10, "filter": 10, "get_cpu_pandas_datafram": 10, "categori": [10, 39], "get_npu_pandas_datafram": 10, "print_npu_oper": [10, 39], "termin": [10, 24, 28, 30, 40], "average_elaps": [10, 39], "count": [10, 34, 38, 39], "lowlevelconv2d": [10, 39], "5119": 10, "9375": 10, "lowleveldepthwiseconv2d": [10, 39], "1091": 10, "56": [10, 32, 34, 39], "lowlevelpad": [10, 39], "561": 10, "482143": 10, "lowlevelexpand": [10, 39], "lowlevelslic": [10, 39], "lowlevelreshap": [10, 39], "232": 10, "print_npu_execut": [10, 39], "execution_ind": 10, "iowait": [10, 39], "39ffc55ef7b2177": 10, "555899badb3f8e5": 10, "116971": 10, "105186": 10, "11785": 10, "5338e9fa2d1fb70": 10, "f1": 10, "9c8aa64bbb878e3": 10, "4e9a13e698f4fa1": 10, "117011": 10, "11825": 10, "b62194f8dec670": 10, "3c": 10, "0ce2a8ce2c591e3": 10, "5cd8a081758f41c": 10, "116961": 10, "105185": 10, "11776": 10, "4e92e0c421f3946": 10, "14": [10, 43], "a941ace17a2c5e6": 10, "a3726d0ebb2705c": 10, "116909": 10, "11723": 10, "15a8f05d8872fa9": 10, "print_external_oper": [10, 39], "extern": [10, 31], "7d65ff7ae5587d3345d5df5a77": 10, "53e3fb9c02964361": 10, "quantiz": [10, 23, 25, 26, 34, 35, 39], "175246": 10, "ebfaad": 10, "33371e09f89cfa06c41286df13": 10, "8d5a00c6e4e8c2c0": 10, "lower": [10, 31, 32, 33, 34, 35, 39], "183803": 10, "11a30f": 10, "9f7df939abc20da11431c18024": 10, "064dacd9a108c4a0": 10, "unlow": [10, 32, 39], "60459": 10, "c41af1": 10, "1bda703f4ffc878a4294ec6253": 10, "cb2f103208d2fa45": 10, "dequant": [10, 32, 39], "19468": 10, "3ac8d0": 10, "9f769c8951f39d98e6ee216e34": 10, "91c0bdd8c5b81327": 10, "85724": 10, "6bc7e5": 10, "048e5cab6d4d676e4e6b10e827": 10, "714834cb8dc59f4b": 10, "306893": 10, "6b5489": 10, "6bb481ca3b1eab843b795a7865": 10, "46d538d7b4c72d25": 10, "73313": 10, "49558b": 10, "e0f13a5fb0bf2942ed16171844": 10, "71a432e3e3dc55f6": 10, "37079": 10, "ccb293": 10, "c3b2fdba80f16f781e4b313af3": 10, "066e3916590edf38": 10, "67805": 10, "a571b6": 10, "4bebe5f61e84d502f5b5dc7d22": 10, "9dfb32069b2b5a98": 10, "310303": 10, "1e4f5a": 10, "b8cabf53ae39a4ad18144af26c": 10, "cb767fbdd718da89": 10, "72378": 10, "e136c9": 10, "e40956dda5ecc0a1774e39377b": 10, "090d9cbd5e60032a": 10, "33951": 10, "1ef245": 10, "3d13f40c0966940439adcce4c1": 10, "4702a924e4b6d38b": 10, "76999": 10, "9981a4": 10, "53746b998038e994a5e378f9a2": 10, "522b7a9e354de2b3": 10, "339339": 10, "8caa5a": 10, "76a2080bc0917db26b7313e29a": 10, "4b1b0bf55f344258": 10, "74708": 10, "81def3": 10, "4c0a04dc669b04416f18e781d6": 10, "8eb55fb2b618933a": 10, "33661": 10, "afc3c6": 10, "print_infer": [10, 39], "b5edc4d40493df2028d186d4073d5487": 10, "a61af3b9ad70b956": 10, "4430749": 10, "983e136f80e1c070dca3ad854f37cf97": 10, "f2dd4e899d52531d": 10, "4181392": 10, "dada8a5830272b5d255fda801568fc5": 10, "cda7127619be5c33": 10, "4275757": 10, "6ad054709f76095c86fba6dcd9254ca0": 10, "9d7f199a445003aa": 10, "4215571": 10, "overal": 10, "latenc": [10, 26, 31, 33, 34, 37, 39], "4275867": 10, "median": [10, 39], "4245664": 10, "90": [10, 32, 34, 39], "4384251": 10, "95": [10, 32, 34, 39], "4407500": 10, "97": [10, 32, 34, 39, 41], "4416800": 10, "4426099": 10, "4430284": 10, "export_chrome_trac": 10, "filenam": [10, 35, 43], "write": [10, 33, 35, 37, 39, 40], "furiosaruntimeerror": 10, "nativeexcept": 10, "room": 10, "upcom": 10, "restructur": 10, "furiosaruntimewarn": 10, "dedic": 10, "log": [10, 24, 28, 30, 34, 36, 38, 43], "furiosa_log_level": 10, "decreas": 10, "code": [10, 29, 30, 37, 38, 39, 40, 43], "major": [10, 26, 30, 37, 38, 43], "incompatibilit": 10, "behavior": 10, "expens": 10, "generate_compiler_log_path": 10, "current_npu_devic": 10, "is_compile_log_en": 10, "log_dir": 10, "furiosa_log_dir": 10, "store": [10, 30], "profiler_output": 10, "furiosa_profiler_output_path": [10, 39], "nux_profiler_path": 10, "being": [10, 24, 32, 33, 37, 43], "incompatiblemodel": 10, "compilationfail": 10, "internalerror": 10, "unsupportedtensortyp": 10, "unsupporteddatatyp": 10, "incompatibleapiclienterror": 10, "invalidyamlexcept": 10, "apiclientinitfail": 10, "noapikeyexcept": 10, "invalidsessionopt": 10, "sessiontermin": 10, "devicebusi": 10, "invalidinput": 10, "tensornamenotfound": 10, "unsupportedfeatur": 10, "invalidcompilerconfig": 10, "sessionclos": 10, "mostli": 10, "typeerror": 10, "reexport": 10, "elig": 10, "match": [10, 32, 40], "rand": [10, 31, 39], "random": [10, 28, 30, 34, 39], "meant": 10, "quick": [10, 38], "test": [10, 31, 34], "qualiti": [10, 29], "correct": 10, "correctli": 10, "cli": [11, 12, 23, 30, 31, 32, 35], "grpc": [11, 12, 23, 28, 37, 40, 43], "servic": [11, 12, 23, 24, 28, 31, 34, 37, 38], "rest": [11, 12, 23, 28, 37, 40, 43], "app": [11, 12, 20, 23, 31, 34], "endpoint": [11, 12, 23], "handler": [11, 23], "predict": [11, 23, 29, 31, 37], "model_repositori": [11, 23], "namespac": [11, 23, 38], "loader": [11, 23], "modelnotfound": 11, "modelservererror": 11, "model_repository_pb2": [12, 14], "model_repository_pb2_grpc": [12, 14], "predict_pb2": [12, 14], "predict_pb2_grpc": [12, 14], "health": [20, 23, 43], "processor": [20, 23], "imagenet": [20, 23, 29, 31], "welcom": 23, "common": [23, 34, 35, 41], "packag": [23, 29, 30, 31, 32, 33, 34, 35, 37, 38], "subpackag": 23, "modul": [23, 40], "nativ": [23, 24, 34, 37], "variant": 23, "legaci": 23, "devic": [23, 28, 31, 32, 34, 36, 37, 43], "specif": [23, 25, 31, 32, 38, 39, 40, 41, 43], "metadata": [23, 34, 37, 38, 43], "runner": [23, 35, 39], "queue": [23, 28], "profil": [23, 24, 26, 34], "diagnost": 23, "compil": [23, 24, 25, 26, 31, 40, 43], "const": 23, "env": [23, 39], "session": [23, 30, 31, 35, 37], "optim": [23, 26, 28, 30, 31, 32, 34, 37, 39, 40, 41], "calibr": [23, 30, 34, 41], "editor": 23, "server": [23, 25, 29, 30, 31, 32, 33, 34, 38], "registri": [23, 28], "serv": [23, 30, 32, 35, 37, 40], "telemetri": 23, "encount": [24, 32, 40], "unresolv": 24, "issu": [24, 28, 34, 36, 39, 40], "custom": [24, 25, 28, 34, 43], "center": [24, 25, 34, 36], "reproduc": 24, "screenshot": 24, "By": [24, 28, 34, 35, 36, 38], "when": [24, 27, 28, 30, 31, 32, 33, 34, 35, 37, 38, 39, 40, 43], "happen": 24, "furiosa": [24, 25, 27, 30, 33, 37, 38, 39, 40, 41, 43, 44], "dump": [24, 28, 34], "home": [24, 25, 28, 30, 35], "20211121223028": [24, 28, 30], "l5w4g6": [24, 28, 30], "rev": [24, 28, 34, 43], "407c0c51f": [24, 28], "modifi": [24, 30, 38], "2021": [24, 28, 34], "11": [24, 28, 30, 32, 34, 43], "18": [24, 28], "22": [24, 28, 36], "34": [24, 28, 34], "22t06": [24, 28], "30": [24, 28, 35], "392114z": 24, "npu0pe0": [24, 28, 31, 34, 38, 43], "397757z": 24, "nuxinn": [24, 43], "peid": [24, 43], "dfg": [24, 34, 35], "423026z": 24, "receiv": [24, 30, 41, 43], "423371z": [24, 28], "unsupport": [24, 28], "dimparam": [24, 28], "sep": [24, 28], "42": [24, 28], "gcc": [24, 28, 33], "libnux": [24, 28, 29, 30, 31, 32, 33, 36], "problem": [24, 28, 34, 39], "atlassian": [24, 28], "net": [24, 28], "servicedesk": [24, 28], "portal": [24, 28], "shown": [24, 28, 29, 30, 31, 34, 35, 38, 39, 40, 41], "instruct": [24, 28, 29, 30, 31, 32, 33, 34, 37, 41], "necessari": [24, 34, 35, 36, 40], "__full_version__": 24, "lib64": 24, "explain": [25, 33, 34, 36, 37, 38, 40], "compon": [25, 36, 37, 38], "firmwar": [25, 29, 30, 31, 32, 33, 34, 40, 43], "command": [25, 30, 31, 32, 35, 36, 37, 38, 40, 41, 43], "line": [25, 30, 31, 32, 35, 37, 38, 39, 40, 41], "tool": [25, 26, 35, 37, 38, 39, 40, 41], "offer": 25, "user": [25, 26, 29, 30, 31, 34, 35, 37, 39], "who": [25, 36], "earli": 25, "access": [25, 31, 36, 37, 39], "program": [25, 31, 33, 34, 35, 36, 37, 40], "eap": 25, "agre": 25, "licenc": 25, "agreement": 25, "eula": 25, "contact": [25, 36], "learn": [25, 26, 43], "introduct": [25, 38], "hw": 25, "acceler": [25, 26, 28, 29, 31, 32, 35, 37, 40, 41], "sw": [25, 36], "stack": 25, "guid": [25, 37, 38], "zoo": [25, 42], "kubernet": [25, 30, 43], "advanc": [25, 43, 44], "compar": [25, 41, 44], "accuraci": [25, 26, 31, 32, 44], "imag": [25, 26, 29, 31, 32, 38, 39, 40, 44], "classif": [25, 26, 44], "ssd": [25, 31, 35, 37, 41, 43, 44], "detect": [25, 26, 31, 38, 44], "framework": [25, 35, 37, 40], "bug": [25, 28, 29, 31, 32, 34], "chip": 26, "deep": 26, "demonstr": [26, 37], "maintain": [26, 36], "cost": [26, 35], "effici": [26, 35, 37], "low": [26, 37, 39], "request": [26, 31, 36, 38, 43], "maxim": [26, 35], "achiev": 26, "abl": [26, 36, 39, 43], "cnn": 26, "therebi": [26, 29, 35, 36, 37, 39], "bottleneck": [26, 31, 39], "energi": 26, "kei": [26, 28, 36, 37], "vision": [26, 28, 31, 37], "ocr": 26, "super": 26, "resolut": 26, "particular": 26, "superior": 26, "depthwis": 26, "group": [26, 27, 29, 30], "drive": 26, "art": 26, "deliv": 26, "32mb": 26, "sram": 26, "consist": [26, 30, 40], "deploi": [26, 28, 41], "respons": [26, 29, 37, 43], "speed": 26, "undergo": [26, 37], "fusion": [26, 28, 38], "aggreg": 26, "throughput": [26, 31, 34, 37, 39], "softwar": [26, 34, 40], "tensorflow": [26, 27, 41], "post": [26, 29, 30, 41, 43], "train": [26, 31, 32, 40, 41], "commonli": [26, 37, 39], "tensorflowlit": 26, "billion": 26, "transistor": 26, "180mm": 26, "clock": 26, "2ghz": 26, "peak": 26, "maximum": 26, "4266": 26, "lpddr4x": 26, "dram": [26, 35], "bandwidth": 26, "66gb": 26, "pcie": 26, "gen4": 26, "8x": 26, "mb": 26, "host": [26, 35, 43], "lane": 26, "form": [26, 40], "factor": [26, 39], "fhhl": 26, "hhhl": 26, "thermal": 26, "solut": 26, "passiv": 26, "fan": 26, "tdp": 26, "60w": 26, "temperatur": [26, 34, 38], "50": 26, "ghz": 26, "ddr": 26, "mbp": 26, "gb": 26, "66": 26, "submit": 26, "mlcommon": 26, "mlperf": [26, 35], "edg": [26, 35], "v2": [26, 34, 35, 43], "v1": [26, 31, 35, 38, 41], "lite": [27, 41], "certain": [27, 30, 37, 39], "condit": [27, 30, 37], "met": [27, 34], "split": [27, 34, 37], "sever": [27, 31, 35, 38, 43], "suffici": 27, "averagepool": 27, "clip": 27, "concat": [27, 29, 30], "dilat": [27, 30], "depthtospac": [27, 30], "exp": [27, 30], "expand": [27, 31], "flatten": [27, 43], "gemm": [27, 31], "leakyrelu": [27, 28], "lpnormal": 27, "p": [27, 31], "maxpool": [27, 31], "reducel2": 27, "reducesum": 27, "reshap": [27, 43], "pow": [27, 28, 30], "spacetodepth": [27, 30], "crd": [27, 29], "higher": [27, 29, 33, 35, 36, 38, 40, 43], "sigmoid": 27, "softplu": 27, "sub": [27, 28, 30, 31, 34], "sqrt": [27, 28], "transpos": [27, 29], "unsqueez": 27, "approxim": [28, 29, 30], "87": 28, "ad": [28, 29, 30, 32, 34, 41], "addition": [28, 37, 43], "b1d2b74": 28, "wherea": 28, "explicitli": [28, 35], "done": [28, 31, 34, 36], "sess1": 28, "model1": 28, "sess2": 28, "model2": 28, "npu0pe1": [28, 31, 43], "async_sess": 28, "npu1pe2": 28, "4f1f114": 28, "limit": [28, 37, 38], "pass": [28, 30, 34, 35], "np1": 28, "randint": 28, "255": [28, 30], "session_input": 28, "np2": 28, "output1": 28, "input2": 28, "input1": 28, "debug": [28, 43], "easier": 28, "mnist": [28, 31, 34, 43], "duplic": 28, "01aaa40": 28, "21cba85": 28, "hang": 28, "interrupt": 28, "a0f4bd7": 28, "easili": [28, 29, 37, 38, 40], "pip": [28, 29, 30, 31, 32, 34, 36, 40, 43], "mnistnet_uint8_quant_without_softmax": [28, 31, 43], "tflite": [28, 31, 34, 35, 37, 39, 43], "mlcommonsresnet50": [28, 31], "resnet50": [28, 31, 35, 37, 41], "sess": [28, 30, 31], "apt": [28, 29, 30, 31, 32, 33, 34, 35, 38, 40], "toolkit": [28, 38], "statu": [28, 38], "idl": [28, 34], "temp": [28, 31, 32, 34, 38], "power": [28, 31, 32, 34, 38], "pci": [28, 31, 32, 34, 38], "bdf": [28, 31, 32, 34, 38], "npu0": [28, 31, 38], "92": 28, "0000": [28, 31, 32, 34, 38], "01": [28, 38], "00": [28, 31, 32, 34, 38], "510": 28, "devnam": [28, 38], "readi": [28, 38, 43], "plugin": [28, 31, 32, 37], "label": [28, 37, 39], "schedul": [28, 32, 37, 38], "togeth": [28, 34, 37], "pod": [28, 37], "kubectl": [28, 38], "githubusercont": [28, 38], "deploy": [28, 38], "yaml": [28, 34, 38, 43], "234": 29, "pr": 29, "improv": [29, 32], "900": 29, "updat": [29, 30, 31, 32, 33, 38, 40], "pdma": [29, 30, 31, 32], "libnpu": [29, 34], "uninstal": 29, "space": [29, 39], "significantli": [29, 31], "reduc": [29, 31, 39], "frequenc": 29, "idempot": 29, "relianc": 29, "pylint": 29, "21": 29, "pyyaml": 29, "project": [29, 31], "structur": [29, 31, 39, 41], "litmu": 29, "accordingli": 29, "checker": 29, "busi": 29, "pre": [29, 31, 32, 40], "postprocess": [29, 31], "uploadfil": 29, "serveapi": 29, "servemodel": [29, 31], "asset": [29, 31, 39, 43], "image_classif": [29, 31], "def": [29, 31], "your": [29, 30, 35, 38, 39, 40, 43], "preprocess": [29, 31], "classifi": 29, "400": [30, 36], "toward": 30, "enhanc": [30, 31, 32], "hal": [30, 31, 32], "hardwar": [30, 31, 32, 41], "layer": [30, 31, 32, 37], "featur": [30, 31, 32, 33, 37, 43], "discoveri": [30, 31, 32], "furiosactl": [30, 31, 32, 38], "adopt": 30, "linear": 30, "nearest": 30, "resiz": 30, "dcr": 30, "chw": 30, "tanh": 30, "increas": [30, 32, 35, 41], "x12": 30, "gelu": 30, "erf": 30, "elu": 30, "binari": [30, 32, 35, 37], "reus": [30, 35], "redi": [30, 35], "storag": [30, 31, 35], "help": [30, 34, 35, 37, 43], "understand": [30, 39], "give": 30, "opportun": 30, "grep": [30, 38], "why": 30, "2022": 30, "05": [30, 32, 34, 43], "24t02": 30, "44": [30, 32, 34], "399402z": 30, "19": [30, 32, 34, 43], "logsoftmax": 30, "399407z": 30, "bigger": 30, "399408z": 30, "17": [30, 39], "36": 30, "had": [30, 32], "beta": [30, 38], "taken": 30, "step": [30, 34, 38, 39, 41], "shell": [30, 35, 39, 43], "wasn": 30, "effect": [30, 32, 35, 36], "without_quant": [30, 31, 32], "input_min": 30, "input_max": 30, "permut": 30, "compile_config": 30, "publish": 30, "nxcxhxw": 30, "nxcxd": 30, "subgraph": 30, "bia": 30, "qdq": 30, "operand": 30, "prevent": [30, 40], "relat": [30, 32], "reinforc": 30, "inspect": 30, "miss": 30, "decim": 30, "adjust": 30, "scale": [30, 37], "small": [30, 31, 39], "becom": 30, "latest": [30, 37, 38, 40], "select": [30, 36, 37, 38, 39, 40], "core": [31, 34, 38], "develop": [31, 32, 34, 35, 36, 39, 40], "upgrad": [31, 32, 40], "simpler": [31, 32], "wish": [31, 32, 36, 40], "setup": [31, 32, 34, 38, 43], "libhal": [31, 32, 36], "fastapi": 31, "web": [31, 39], "npuservemodel": 31, "npu_devic": [31, 43], "npu1pe0": [31, 32, 34], "shift": 31, "queri": [31, 34, 39], "frequent": [31, 35], "better": [31, 32], "self": 31, "complex": 31, "scenario": 31, "openvino": 31, "cloud": 31, "auth": [31, 36], "aioboto": 31, "densenet": 31, "93d63f654f0f192cc4ff5691be60fb9379e9d7fd": 31, "opentelemetri": 31, "collector": 31, "track": 31, "To": [31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 43], "get_trac": 31, "reset": [31, 35, 38], "tracer": 31, "start_as_current_span": 31, "__name__": 31, "input_tensor": 31, "output_tensor": 31, "furiosa_serving_otlp_endpoint": 31, "diagram": [31, 37, 41], "visual": [31, 38, 39], "grafana": [31, 38], "share": [31, 35, 39], "analyz": [31, 35, 37, 39], "analysi": [31, 34, 41], "df": [31, 39], "silu": 31, "usabl": 31, "div": 31, "auto_pad": 31, "newli": [31, 32, 40], "o": [31, 32, 34, 35, 38], "pid": [31, 32, 34], "cmd": [31, 34], "132529": [31, 34], "bin": [31, 34, 38, 39], "python3": [31, 34, 40], "uvicorn": [31, 34, 43], "image_classifi": [31, 34, 39], "uniqu": 31, "72212674": 31, "61be": 31, "4fca": 31, "a2c9": 31, "555e4ee67ab5": 31, "12180b0": 31, "49": [31, 32, 34, 43], "24": 31, "235": 31, "npu1": [31, 32, 34, 38], "df80fb54": 31, "8190": 31, "44bc": 31, "b9fb": 31, "664fa36c754a": 31, "54": [31, 32, 34], "53": 31, "6d": 31, "511": [31, 32, 34], "furioa": 31, "public": [31, 36], "yolov5": [31, 41], "medium": 31, "yolov5l": 31, "yolov5m": 31, "sota": 31, "explan": [31, 34, 36, 38], "nonblock": 31, "load_async": 31, "ouput": 31, "mobilenet": [31, 35, 37, 41], "resnet34": [31, 35, 41], "benchmark": [31, 35], "70": 31, "nativepostprocessor": 31, "postprocessor": 31, "jpg": [31, 39], "eval": 31, "changelog": 31, "partcular": 32, "signific": [32, 35], "setuptool": [32, 40], "wheel": [32, 40], "without": [32, 34, 35, 39, 40], "satisfi": [32, 37, 40], "impl": [32, 40], "No": [32, 40], "distribut": [32, 35, 36, 37, 38, 40], "flexibl": 32, "solid": 32, "optimized_onnx_model": 32, "source_onnx_model": 32, "calibration_data": 32, "tqdm": 32, "calibration_dataload": 32, "unit": [32, 34], "mininterv": 32, "quantizated_graph": 32, "begin": [32, 38], "with_quant": 32, "normalized_pixel_output": 32, "fp32": [32, 41], "pixel": 32, "quasisymmetr": 32, "popular": [32, 37], "among": [32, 40], "efficientnet": [32, 37, 41], "b0": [32, 41], "57": [32, 34], "452": 32, "104": 32, "contrast": 32, "73": [32, 41], "556": [32, 41], "experi": 32, "tutori": [32, 37, 40], "unnecessari": 32, "200": [32, 34, 43], "under": [32, 34, 38], "ctrl": [32, 34, 43], "datetim": [32, 34], "comp": [32, 34], "2023": [32, 34, 35, 43], "03": [32, 34], "21t09": [32, 34], "45": [32, 34], "699483936z": [32, 34], "152616": [32, 34], "06": [32, 34], "100": [32, 34, 41], "npu_runtime_test": [32, 34], "10000": [32, 34], "resnet": [32, 34], "ctc_kor1_200_nightly3_128dpes_8batch": [32, 34], "enf": [32, 34], "906443888z": [32, 34], "51": [32, 34], "09": [32, 34], "93": [32, 34, 41], "110489333z": [32, 34], "46": [32, 34], "98": [32, 34], "02": [32, 34], "316060982z": [32, 34], "43": [32, 34], "521140588z": [32, 34], "94": [32, 34, 41], "725910558z": [32, 34], "48": [32, 34], "07": [32, 34, 35], "935041998z": [32, 34], "47": [32, 34], "91": [32, 34], "58": [32, 34], "13929122z": [32, 34], "displai": 32, "concis": 32, "want": [32, 35, 36, 38, 39, 40, 43], "3c10fd3": [32, 34], "00000000": [32, 34], "000000000000": [32, 34], "wbyb0000000000000": [32, 34], "languag": [33, 37, 42], "abi": 33, "substitut": 33, "minimum": [33, 34], "ubuntu": [33, 36, 38, 40, 43], "20": [33, 34, 36, 38, 40, 41, 43], "04": [33, 34, 36, 38, 40, 43], "lt": [33, 36, 40, 43], "debian": [33, 36, 40, 43], "bullsey": [33, 36, 40, 43], "system": [33, 35, 36, 37, 38, 40], "administr": [33, 36], "privileg": [33, 36], "root": [33, 36, 38, 39, 43], "authent": 33, "x86_64": [33, 40], "linux": [33, 36, 37, 38], "gnu": 33, "simpli": [33, 43], "lnux": 33, "monitor": 34, "sudo": [34, 36, 40], "varieti": [34, 41], "subcommand": 34, "abil": 34, "obtain": [34, 41], "control": [34, 35, 37, 38], "recogn": [34, 37, 38], "consumpt": 34, "visibl": 34, "mount": [34, 37], "machin": [34, 35, 36, 38], "devfil": 34, "npu1pe1": 34, "calcul": 34, "m": [34, 41], "observ": 34, "carri": 34, "workload": [34, 37, 38], "tail": 34, "qp": 34, "expon": 34, "io": [34, 38], "durat": [34, 39], "min_query_count": 34, "min_dur": 34, "min_duration_m": 34, "orient": 34, "l": [34, 39, 41], "model_path": [34, 39], "produc": [34, 39], "1000": 34, "priorit": 34, "individu": 34, "stat": 34, "u": 34, "8399": 34, "307568": 34, "29040": 34, "50th": 34, "19329": 34, "95th": 34, "62797": 34, "99th": 34, "79874": 34, "model_data": 34, "md5": 34, "d7cd24a0a76cd492f31065301d468c3d": 34, "compiler_vers": 34, "2d862de8a": 34, "built_at": 34, "13t20": 34, "04z": 34, "hal_vers": 34, "nightli": 34, "230716": 34, "git_revis": 34, "fe6f77a": 34, "30025": 34, "num": 34, "33": 34, "31": 34, "8840": 34, "113254": 34, "29989": 34, "18861": 34, "64927": 34, "87052": 34, "9th": 34, "meta": 34, "pythonsdk": [34, 35], "extens": [34, 40], "prepar": 34, "verifi": 34, "fe1fca3": 34, "49b97492a": 34, "08": 34, "page": [34, 37], "normal": [34, 38], "goe": 34, "archiv": [34, 36], "step1": 34, "step2": 34, "step3": 34, "step4": 34, "skip": [34, 36, 40], "output_prefix": 34, "target_npu": [34, 35], "libfuriosa_h": [34, 43], "v0": [34, 35, 38, 43], "43c901f": [34, 43], "libfuriosa_compil": 34, "d7548b7f6": 34, "9ecebb6": 34, "famili": [34, 35, 38], "2pe": [34, 35], "09272794": 34, "ldfg": 34, "la": 34, "034934": 34, "140083": 34, "019548794": 34, "29": 34, "196825": 34, "cdfg": [34, 35], "001701888": 34, "gir": [34, 35], "015205072": 34, "lir": [34, 35], "0038304": 34, "020943863": 34, "331545": 34, "000001198": 34, "125": 34, "7448": 34, "unix_epoch": 34, "zip": 34, "zipinfo": 34, "1690438803": 34, "16904388032l4hoi3h": 34, "dot": [34, 35], "etc": [35, 36, 38], "analys": 35, "automat": [35, 37, 38], "But": 35, "f8f05c8ea": 35, "31t19": 35, "30z": 35, "ir": 35, "target_ir": 35, "dot_graph": 35, "analyze_memori": 35, "accler": 35, "choos": 35, "stand": 35, "foo": 35, "let": 35, "npu\ub294": 35, "sampl": [35, 43], "across": 35, "exceed": 35, "degrad": 35, "usual": 35, "trial": 35, "final": 35, "minut": 35, "Then": 35, "instantli": 35, "veri": 35, "coupl": 35, "remot": [35, 37], "fc_cache_en": 35, "fc_cache_store_url": 35, "rediss": 35, "ssl": 35, "tmp": 35, "cluster": [35, 37, 38], "password": [35, 36], "127": 35, "6379": 35, "25945": 35, "dai": 35, "hour": 35, "7200": 35, "sec": 35, "aliv": 35, "forev": 35, "expir": 35, "site": 36, "registr": 36, "evalu": 36, "focal": [36, 38], "fossa": 36, "internet": 36, "delin": 36, "ca": 36, "certif": 36, "gnupg": 36, "adv": 36, "keyserv": 36, "5f03afa423a751913f249259814f888b20b09a7": 36, "tee": 36, "conf": [36, 38], "null": 36, "eot": 36, "login": 36, "secret": 36, "chmod": 36, "tab": 36, "bookworm": 36, "deb": 36, "amd64": [36, 38], "jammi": 36, "multi": [36, 37], "owner": 36, "exclus": 36, "member": 36, "usermod": 36, "ag": [36, 38], "usernam": 36, "back": 36, "membership": 36, "libonnxruntim": 36, "held": 36, "showhold": 36, "outlin": 37, "role": 37, "guidelin": 37, "lowest": 37, "acknowledg": 37, "reinstal": [37, 40], "There": 37, "plai": 37, "dnn": [37, 39, 40, 43], "research": 37, "area": 37, "smaller": 37, "balanc": 37, "facilit": 37, "expos": 37, "protocol": 37, "horizont": 37, "platform": [37, 38, 43], "container": [37, 38], "enterpris": 37, "tenant": 37, "physic": 37, "particip": 37, "spec": [37, 38], "nodeselector": [37, 38], "nodeaffin": 37, "kuberent": 38, "awar": [38, 39], "simultan": 38, "equip": 38, "nfd": 38, "intel": 38, "recognit": 38, "again": [38, 40], "reboot": 38, "37": 38, "509": 38, "daemonset": 38, "jq": 38, "cpuid": 38, "adx": 38, "aesni": 38, "master": 38, "vendor": 38, "devicelabelfield": 38, "deviceclasswhitelist": 38, "mandatori": 38, "kube": 38, "alpha": 38, "1ed2": 38, "attach": 38, "TO": 38, "date": 38, "selector": 38, "128m": 38, "162m": 38, "tabl": [38, 41, 43], "product": 38, "warboyb0": 38, "renegad": 38, "manufactur": 38, "nmae": 38, "minor": 38, "patch": 38, "57ac7b0": 38, "build": 38, "node01": 38, "65d": 38, "node02": 38, "12d": 38, "apivers": 38, "kind": 38, "ghcr": 38, "search": 38, "comma": 38, "dir": 38, "kubelet": 38, "var": 38, "socket": 38, "permit": 38, "defaultp": 38, "disableddevic": 38, "nodenam": 38, "plugindir": 38, "socketnam": 38, "resourcenam": 38, "text": [38, 43], "configmap": 38, "reflect": 38, "edit": 38, "15": 38, "npu2": 38, "001": 38, "creation": 38, "eol": 38, "8gi": 38, "exec": 38, "bash": [38, 43], "echo": 38, "daemon": 38, "express": [38, 39], "prometheu": 38, "review": 38, "furiosa_npu_al": 38, "furiosa_npu_uptim": 38, "furiosa_npu_error": 38, "furiosa_npu_hw_temperatur": 38, "mc": 38, "furiosa_npu_hw_pow": 38, "instantan": 38, "\u00b5w": 38, "furiosa_npu_hw_voltag": 38, "instantaen": 38, "voltag": 38, "mv": 38, "furiosa_npu_hw_curr": 38, "ma": 38, "critic": 39, "ml": [39, 40], "engin": 39, "assist": 39, "browser": 39, "tempor": 39, "advantag": 39, "unchang": 39, "downsid": 39, "clone": [39, 40, 43], "cd": [39, 40, 43], "pwd": 39, "car": 39, "rw": 39, "r": 39, "456493": 39, "jul": 39, "27": 39, "address": [39, 43], "viewer": 39, "click": 39, "button": 39, "upper": 39, "left": 39, "corner": 39, "comparison": 39, "interact": 39, "interpret": 39, "jupyt": 39, "notebook": 39, "tree": 39, "quantized_model": [39, 43], "imagenet_224x224_mobilenet_v1_uint8_quant": 39, "trained_dm_1": 39, "0_without_softmax": 39, "mobilenet_v1_trac": 39, "warm": 39, "mnist_trac": 39, "job": 39, "huge": 39, "disk": 39, "difficult": 39, "hard": 39, "interest": 39, "avoid": 39, "exclud": 39, "phase": 39, "warmup": 39, "assert": 39, "1584494": 39, "3027309": 39, "2136984": 39, "1968066": 39, "2752525": 39, "2889917": 39, "2944874": 39, "2999831": 39, "3024561": 39, "7cf3d3b7439cf4c3fac1a47998783102": 39, "403ada67f1d8220": 39, "16d65f6f8f1db256d0f39953855dea72": 39, "78b065c19c3675ef": 39, "2111363": 39, "d0534e3a9f19edadab81954ad28ab44f": 39, "9a7addaf0f28c9f": 39, "1824769": 39, "70512188522f45b87cfe4f545de3cf2c": 39, "c75f697f8e72d333": 39, "elaps": 39, "8f6fce6c0e52b4735cae3379732a0943": 39, "3e1e4a76523cbf89": 39, "119145": 39, "108134": 39, "18446744073709540605": 39, "195366613b1da9b0350c0a3c2a608f42": 39, "07dff2e92172fabd": 39, "119363": 39, "18446744073709540387": 39, "3b65b8fa3eabfaf8f815ec9f41fcc7d9": 39, "639a366a7f932a23": 39, "119157": 39, "18446744073709540593": 39, "e48825df32a07e5559f7f50048c08e1f": 39, "ecaab4915bfda725": 39, "119219": 39, "18446744073709540531": 39, "5327": 39, "60": 39, "1412": 39, "285714": 39, "575": 39, "785714": 39, "lowleveltranspos": 39, "250": 39, "240": 39, "e7ab6656cc090a8d05992a9e4683b8b7": 39, "206a1d6f351ca4b1": 39, "136285": 39, "03636fd6c7dbc42f0a9dd29a7283d3fc": 39, "f636740983e095a6": 39, "133350": 39, "c9a0858f7e0885a976f51c6cb57d3e0f": 39, "bb6c84f88e453055": 39, "44775": 39, "8777c67ad9fe597139bbd6970362c2fc": 39, "63bac982c7b98aba": 39, "14682": 39, "98aeba2a25b0525166b6a4065ab01774": 39, "34ccd560571d733f": 39, "45465": 39, "420525dc13ba9624083e0a276f7ee718": 39, "9f6d342da5eb86bc": 39, "152748": 39, "cb67393f6949bbbb396053c1e00931ff": 39, "2d724fa6ab8ca024": 39, "67140": 39, "00424b4f02039ae0ca98388a964062b0": 39, "a5fb9fbd5bffe6a6": 39, "32388": 39, "d7412c59d360067e8b7a2508a30d1079": 39, "8e426d778fa95722": 39, "71736": 39, "6820acf9345c5b373c512f6cd5edcbc7": 39, "2d787c2df381f010": 39, "311310": 39, "84d24b02a95c63c3e40f7682384749e4": 39, "1236a974a619ff1a": 39, "51930": 39, "8d25dff1cfd6624509cbf95503e93382": 39, "673efb3bfb8deac6": 39, "12362": 39, "4cc60ec1eee7d9f3cdd290d07b303a18": 39, "e7903b0a584d6388": 39, "56736": 39, "c5f04d9fea26e5b52c6ec5e5406775fc": 39, "701118dabd065e6f": 39, "265447": 39, "c5fdfb9cf454da130148e8e364eeee93": 39, "5cf3750def19c6e8": 39, "35869": 39, "e1e650d23061140404915f1df36daf9c": 39, "ddd76ff19b5cd713": 39, "14688": 39, "direct": 39, "487": 39, "f3b158734e3684f2e043ed41309c4c2d": 39, "11204385": 39, "kit": 40, "ecosystem": 40, "wide": 40, "familiar": 40, "therefor": 40, "often": 40, "concern": 40, "conflict": 40, "easi": 40, "ye": 40, "question": 40, "sh": 40, "miniconda3": 40, "wget": 40, "repo": 40, "anaconda": 40, "miniconda": 40, "bashrc": 40, "deactiv": 40, "proper": 40, "upload": 40, "pypi": 40, "subdirectori": 40, "rather": 40, "adot": 40, "simul": 40, "fp16": 41, "techniqu": 41, "provi": 41, "greater": 41, "topolog": 41, "drop": 41, "fp": 41, "convnext": 41, "85": 41, "80": 41, "376": 41, "678": 41, "77": 41, "698": 41, "669": 41, "efficientnetv2": 41, "84": 41, "228": 41, "83": 41, "566": 41, "214": 41, "76": 41, "456": 41, "702": 41, "retinanet": 41, "3757": 41, "37373": 41, "476": 41, "23": 41, "23215": 41, "21626": 41, "108": 41, "yolox": 41, "497": 41, "48524": 41, "634": 41, "490": 41, "47443": 41, "96": 41, "822": 41, "454": 41, "43963": 41, "835": 41, "apu": 41, "8bit": 41, "link": 41, "bind": 43, "port": 43, "ip": 43, "listen": 43, "8080": 43, "zsh": 43, "fish": 43, "powershel": 43, "pwsh": 43, "model_config_list": 43, "mnistnet_uint8_qu": 43, "compiler_config": 43, "keep_unsigned": 43, "split_unit": 43, "ssd512_mobilenet_v2_bdd_int_without_reshap": 43, "model_config_exampl": 43, "20230509151914": 43, "axpfej": 43, "e626c458c": 43, "19t13": 43, "26z": 43, "09t06": 43, "560585z": 43, "565216z": 43, "d063sw": 43, "591795z": 43, "595298z": 43, "1184080": 43, "startup": 43, "press": 43, "quit": 43, "curl": 43, "mnist_imag": 43, "train_imag": 43, "60000": 43, "astyp": 43, "localhost": 43, "tolist": 43, "live": 43, "ok": 43, "respond": 43, "livenessprob": 43, "inferenc": 43, "readinessprob": 43, "model_nam": 43, "model_vers": 43}, "objects": {"furiosa": [[0, 0, 0, "-", "common"], [2, 0, 0, "-", "optimizer"], [9, 0, 0, "-", "quantizer"], [10, 0, 0, "-", "runtime"]], "furiosa.common": [[0, 0, 0, "-", "error"], [0, 2, 1, "", "get_sdk_version"], [0, 0, 0, "-", "native"], [0, 0, 0, "-", "thread"], [1, 0, 0, "-", "transport"], [0, 0, 0, "-", "utils"]], "furiosa.common.error": [[0, 1, 1, "", "FuriosaError"], [0, 2, 1, "", "is_err"], [0, 2, 1, "", "is_ok"]], "furiosa.common.native": [[0, 3, 1, "", "LogLevel"], [0, 2, 1, "", "find_global_lib_path"], [0, 2, 1, "", "find_native_lib_path"], [0, 2, 1, "", "find_native_libs"], [0, 2, 1, "", "find_user_lib_path"]], "furiosa.common.native.LogLevel": [[0, 4, 1, "", "ERROR"], [0, 4, 1, "", "INFO"], [0, 4, 1, "", "OFF"], [0, 4, 1, "", "WARN"]], "furiosa.common.thread": [[0, 2, 1, "", "asynchronous"], [0, 2, 1, "", "synchronous"]], "furiosa.common.transport": [[1, 3, 1, "", "FileTransport"], [1, 3, 1, "", "GithubTransport"], [1, 3, 1, "", "HTTPTransport"], [1, 3, 1, "", "S3Transport"], [1, 3, 1, "", "Transport"], [1, 0, 0, "-", "base"], [1, 2, 1, "", "download"], [1, 0, 0, "-", "error"], [1, 0, 0, "-", "file"], [1, 0, 0, "-", "github"], [1, 0, 0, "-", "http"], [1, 2, 1, "", "is_relative"], [1, 0, 0, "-", "s3"], [1, 2, 1, "", "supported"]], "furiosa.common.transport.FileTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.GithubTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.HTTPTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.S3Transport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.Transport": [[1, 6, 1, "", "cache_directory"], [1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"]], "furiosa.common.transport.base": [[1, 3, 1, "", "Transport"]], "furiosa.common.transport.base.Transport": [[1, 6, 1, "", "cache_directory"], [1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"]], "furiosa.common.transport.error": [[1, 1, 1, "", "TransportError"], [1, 1, 1, "", "TransportNotFound"]], "furiosa.common.transport.file": [[1, 3, 1, "", "FileTransport"]], "furiosa.common.transport.file.FileTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.transport.github": [[1, 3, 1, "", "GithubTransport"]], "furiosa.common.transport.github.GithubTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.http": [[1, 3, 1, "", "HTTPTransport"]], "furiosa.common.transport.http.HTTPTransport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 4, 1, "", "read"], [1, 4, 1, "", "schemes"]], "furiosa.common.transport.s3": [[1, 3, 1, "", "S3Transport"]], "furiosa.common.transport.s3.S3Transport": [[1, 5, 1, "", "download"], [1, 5, 1, "", "is_supported"], [1, 5, 1, "", "parse"], [1, 4, 1, "", "read"], [1, 4, 1, "", "scheme"]], "furiosa.common.utils": [[0, 3, 1, "", "FuriosaVersionInfo"], [0, 2, 1, "", "eprint"], [0, 2, 1, "", "get_sdk_version"], [0, 2, 1, "", "removeprefix"]], "furiosa.optimizer": [[3, 0, 0, "-", "frontend"], [8, 0, 0, "-", "interfaces"], [2, 2, 1, "", "optimize_model"]], "furiosa.optimizer.frontend": [[4, 0, 0, "-", "onnx"]], "furiosa.optimizer.frontend.onnx": [[4, 2, 1, "", "optimize_model"], [5, 0, 0, "-", "transformer"], [7, 0, 0, "-", "utils"]], "furiosa.optimizer.frontend.onnx.transformer": [[5, 3, 1, "", "ONNXTransformer"], [5, 0, 0, "-", "convert_negative_pads_to_slice"], [5, 0, 0, "-", "convert_prelu_to_relu"], [6, 0, 0, "-", "experimental"], [5, 0, 0, "-", "fuse_batchnorm"], [5, 0, 0, "-", "fuse_gather_matmul"], [5, 0, 0, "-", "infer_squeeze_axes"], [5, 0, 0, "-", "polish_model"], [5, 0, 0, "-", "utils"]], "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer": [[5, 5, 1, "", "bridge_disconnected_nodes"], [5, 5, 1, "", "build_optimized_model"], [5, 4, 1, "", "check_runnable"], [5, 5, 1, "", "copy_value_info"], [5, 5, 1, "", "find_next_node"], [5, 5, 1, "", "find_prev_node"], [5, 5, 1, "", "get_data_node_input"], [5, 5, 1, "", "get_init_node_input"], [5, 5, 1, "", "get_initializer_array"], [5, 5, 1, "", "get_map_values"], [5, 5, 1, "", "get_value_info_dtype"], [5, 5, 1, "", "get_value_info_shape"], [5, 5, 1, "", "is_same_shape"], [5, 5, 1, "", "pattern_matcher"], [5, 5, 1, "", "pattern_matching"], [5, 5, 1, "", "pop_multiple_optimizer_map"], [5, 5, 1, "", "pop_single_optimizer_map"], [5, 5, 1, "", "transform"], [5, 5, 1, "", "transform_to_eliminate"], [5, 5, 1, "", "transform_to_fuse"], [5, 5, 1, "", "traverse_prev_node"], [5, 5, 1, "", "update_graph_fields"], [5, 5, 1, "", "update_multiple_initializer_map"], [5, 5, 1, "", "update_multiple_optimizer_map"], [5, 5, 1, "", "update_multiple_value_info_map"], [5, 5, 1, "", "update_single_initializer_map"], [5, 5, 1, "", "update_single_optimizer_map"], [5, 5, 1, "", "update_single_value_info_map"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice": [[5, 3, 1, "", "ConvertNegativePadsToSlice"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1": [[5, 5, 1, "", "make_new_init_and_vi"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2": [[5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu": [[5, 3, 1, "", "ConvertPReluToRelu"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.experimental": [[6, 0, 0, "-", "eliminate_detection_postprocess"], [6, 0, 0, "-", "embedding_bag_porting"], [6, 0, 0, "-", "fuse_div_for_bert"], [6, 0, 0, "-", "reify_conv_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess": [[6, 3, 1, "", "EliminateSSDDetectionPostprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess": [[6, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting": [[6, 3, 1, "", "EmbeddingBagPattern"], [6, 3, 1, "", "EmbeddingBagPorting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern": [[6, 5, 1, "", "check_condition_3"], [6, 5, 1, "", "pattern_condition_checker"], [6, 5, 1, "", "pattern_matching"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting": [[6, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert": [[6, 3, 1, "", "FuseDivForBert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert": [[6, 5, 1, "", "transform"], [6, 5, 1, "", "transform_matmul_add"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert": [[6, 3, 1, "", "ReifyConvForBert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert": [[6, 5, 1, "", "transform"], [6, 5, 1, "", "transform_matmul_add"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm": [[5, 3, 1, "", "FuseBatchNorm"], [5, 3, 1, "", "Pattern_1"], [5, 3, 1, "", "Pattern_2"], [5, 3, 1, "", "Pattern_3"], [5, 3, 1, "", "Pattern_4"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1": [[5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2": [[5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3": [[5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_vi"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul": [[5, 3, 1, "", "FuseGatherMatMul"], [5, 3, 1, "", "Pattern_1"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1": [[5, 5, 1, "", "make_new_init"], [5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes": [[5, 3, 1, "", "InferSqueezeAxes"], [5, 3, 1, "", "Pattern_1"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1": [[5, 5, 1, "", "make_new_node"], [5, 5, 1, "", "pattern_condition_checker"], [5, 5, 1, "", "pattern_matching"], [5, 4, 1, "", "pattern_to_match"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model": [[5, 3, 1, "", "PolishModel"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel": [[5, 5, 1, "", "transform"]], "furiosa.optimizer.frontend.onnx.transformer.utils": [[5, 2, 1, "", "check_value_info"], [5, 2, 1, "", "eliminate_initializer_from_graph_input"], [5, 2, 1, "", "eliminate_unused_initializer"], [5, 2, 1, "", "eliminate_unused_input"], [5, 2, 1, "", "eliminate_unused_output"], [5, 2, 1, "", "eliminate_unused_protos"], [5, 2, 1, "", "eliminate_unused_value_info"], [5, 2, 1, "", "fix_batch_size_as_one"], [5, 2, 1, "", "fixed_point"], [5, 2, 1, "", "get_attribute"], [5, 2, 1, "", "get_node_attributes"], [5, 2, 1, "", "get_node_input_names"], [5, 2, 1, "", "get_node_output_names"], [5, 2, 1, "", "is_op_type"], [5, 2, 1, "", "make_initializer_name_unique"], [5, 2, 1, "", "make_unhashables_unique"], [5, 2, 1, "", "name_nodes"], [5, 2, 1, "", "rebuild_model"]], "furiosa.optimizer.frontend.onnx.utils": [[7, 0, 0, "-", "check_model"], [7, 0, 0, "-", "inference_shape"], [7, 0, 0, "-", "version_checker"]], "furiosa.optimizer.frontend.onnx.utils.check_model": [[7, 2, 1, "", "check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape": [[7, 3, 1, "", "InferenceShape"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape": [[7, 5, 1, "", "inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker": [[7, 3, 1, "", "CheckVersion"]], "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion": [[7, 4, 1, "", "opset_bound"], [7, 5, 1, "", "transform"]], "furiosa.optimizer.interfaces": [[8, 0, 0, "-", "transformer"]], "furiosa.optimizer.interfaces.transformer": [[8, 3, 1, "", "Transformer"]], "furiosa.optimizer.interfaces.transformer.Transformer": [[8, 5, 1, "", "transform"]], "furiosa.quantizer": [[9, 3, 1, "", "CalibrationMethod"], [9, 3, 1, "", "Calibrator"], [9, 3, 1, "", "ModelEditor"], [9, 3, 1, "", "TensorType"], [9, 0, 0, "-", "calibrator"], [9, 0, 0, "-", "editor"], [9, 2, 1, "", "get_output_names"], [9, 2, 1, "", "get_pure_input_names"], [9, 2, 1, "", "quantize"]], "furiosa.quantizer.CalibrationMethod": [[9, 4, 1, "id10", "ENTROPY_ASYM"], [9, 4, 1, "id11", "ENTROPY_SYM"], [9, 4, 1, "id12", "MIN_MAX_ASYM"], [9, 4, 1, "id13", "MIN_MAX_SYM"], [9, 4, 1, "id14", "MSE_ASYM"], [9, 4, 1, "id15", "MSE_SYM"], [9, 4, 1, "id16", "PERCENTILE_ASYM"], [9, 4, 1, "id17", "PERCENTILE_SYM"], [9, 4, 1, "id18", "SQNR_ASYM"], [9, 4, 1, "id19", "SQNR_SYM"]], "furiosa.quantizer.Calibrator": [[9, 5, 1, "", "collect_data"], [9, 5, 1, "", "compute_range"]], "furiosa.quantizer.ModelEditor": [[9, 5, 1, "", "convert_input_type"], [9, 5, 1, "", "convert_output_type"]], "furiosa.quantizer.TensorType": [[9, 4, 1, "", "INT8"], [9, 4, 1, "", "UINT8"]], "furiosa.quantizer.calibrator": [[9, 3, 1, "", "CalibrationMethod"], [9, 3, 1, "", "Calibrator"]], "furiosa.quantizer.calibrator.CalibrationMethod": [[9, 4, 1, "id0", "ENTROPY_ASYM"], [9, 4, 1, "id1", "ENTROPY_SYM"], [9, 4, 1, "id2", "MIN_MAX_ASYM"], [9, 4, 1, "id3", "MIN_MAX_SYM"], [9, 4, 1, "id4", "MSE_ASYM"], [9, 4, 1, "id5", "MSE_SYM"], [9, 4, 1, "id6", "PERCENTILE_ASYM"], [9, 4, 1, "id7", "PERCENTILE_SYM"], [9, 4, 1, "id8", "SQNR_ASYM"], [9, 4, 1, "id9", "SQNR_SYM"]], "furiosa.quantizer.calibrator.Calibrator": [[9, 5, 1, "", "collect_data"], [9, 5, 1, "", "compute_range"]], "furiosa.quantizer.editor": [[9, 3, 1, "", "ModelEditor"], [9, 3, 1, "", "TensorType"], [9, 2, 1, "", "get_output_names"], [9, 2, 1, "", "get_pure_input_names"]], "furiosa.quantizer.editor.ModelEditor": [[9, 5, 1, "", "convert_input_type"], [9, 5, 1, "", "convert_output_type"]], "furiosa.quantizer.editor.TensorType": [[9, 4, 1, "", "INT8"], [9, 4, 1, "", "UINT8"]], "furiosa.runtime": [[10, 3, 1, "", "Axis"], [10, 3, 1, "", "DataType"], [10, 1, 1, "", "FuriosaRuntimeError"], [10, 1, 1, "", "FuriosaRuntimeWarning"], [10, 3, 1, "", "Model"], [10, 3, 1, "", "ModelSource"], [10, 3, 1, "", "Receiver"], [10, 3, 1, "", "Runner"], [10, 3, 1, "", "Runtime"], [10, 3, 1, "", "Submitter"], [10, 3, 1, "", "Tensor"], [10, 3, 1, "", "TensorArray"], [10, 3, 1, "", "TensorDesc"], [10, 0, 0, "-", "compiler"], [10, 0, 0, "-", "consts"], [10, 2, 1, "", "create_queue"], [10, 2, 1, "", "create_runner"], [10, 0, 0, "-", "envs"], [10, 0, 0, "-", "errors"], [10, 2, 1, "", "full_version"], [10, 0, 0, "-", "model"], [10, 0, 0, "-", "profiler"], [10, 0, 0, "-", "session"], [10, 0, 0, "-", "tensor"]], "furiosa.runtime.Axis": [[10, 6, 1, "", "BATCH"], [10, 6, 1, "", "CHANNEL"], [10, 6, 1, "", "HEIGHT"], [10, 6, 1, "", "UNKNOWN"], [10, 6, 1, "", "WIDTH"]], "furiosa.runtime.DataType": [[10, 6, 1, "", "BFLOAT16"], [10, 6, 1, "", "FLOAT16"], [10, 6, 1, "", "FLOAT32"], [10, 6, 1, "", "INT16"], [10, 6, 1, "", "INT32"], [10, 6, 1, "", "INT64"], [10, 6, 1, "", "INT8"], [10, 6, 1, "", "UINT8"], [10, 6, 1, "", "numpy"], [10, 6, 1, "", "numpy_dtype"]], "furiosa.runtime.Model": [[10, 5, 1, "", "input"], [10, 6, 1, "", "input_num"], [10, 5, 1, "", "inputs"], [10, 5, 1, "", "output"], [10, 6, 1, "", "output_num"], [10, 5, 1, "", "outputs"], [10, 5, 1, "", "print_summary"], [10, 5, 1, "", "summary"]], "furiosa.runtime.Receiver": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "recv"]], "furiosa.runtime.Runner": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "run"]], "furiosa.runtime.Runtime": [[10, 5, 1, "", "close"]], "furiosa.runtime.Submitter": [[10, 5, 1, "", "allocate"], [10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "submit"]], "furiosa.runtime.TensorDesc": [[10, 5, 1, "", "axis"], [10, 5, 1, "", "dim"], [10, 6, 1, "", "dtype"], [10, 6, 1, "", "format"], [10, 6, 1, "", "length"], [10, 6, 1, "", "name"], [10, 6, 1, "", "ndim"], [10, 6, 1, "", "numpy_dtype"], [10, 6, 1, "", "shape"], [10, 6, 1, "", "size"], [10, 5, 1, "", "stride"]], "furiosa.runtime.compiler": [[10, 2, 1, "", "generate_compiler_log_path"]], "furiosa.runtime.envs": [[10, 2, 1, "", "current_npu_device"], [10, 2, 1, "", "is_compile_log_enabled"], [10, 2, 1, "", "log_dir"], [10, 2, 1, "", "profiler_output"]], "furiosa.runtime.errors": [[10, 1, 1, "", "ApiClientInitFailed"], [10, 1, 1, "", "CompilationFailed"], [10, 1, 1, "", "DeviceBusy"], [10, 1, 1, "", "IncompatibleApiClientError"], [10, 1, 1, "", "IncompatibleModel"], [10, 1, 1, "", "InternalError"], [10, 1, 1, "", "InvalidCompilerConfig"], [10, 1, 1, "", "InvalidInput"], [10, 1, 1, "", "InvalidSessionOption"], [10, 1, 1, "", "InvalidYamlException"], [10, 1, 1, "", "NoApiKeyException"], [10, 1, 1, "", "QueueWaitTimeout"], [10, 1, 1, "", "SessionClosed"], [10, 1, 1, "", "SessionTerminated"], [10, 1, 1, "", "TensorNameNotFound"], [10, 1, 1, "", "UnsupportedDataType"], [10, 1, 1, "", "UnsupportedFeature"], [10, 1, 1, "", "UnsupportedTensorType"]], "furiosa.runtime.furiosa.runtime.Runtime": [[10, 5, 1, "", "create_queue"], [10, 5, 1, "", "create_runner"]], "furiosa.runtime.profiler": [[10, 3, 1, "", "RecordFormat"], [10, 3, 1, "", "Resource"], [10, 3, 1, "", "profile"]], "furiosa.runtime.profiler.RecordFormat": [[10, 6, 1, "", "ChromeTrace"], [10, 6, 1, "", "PandasDataFrame"]], "furiosa.runtime.profiler.Resource": [[10, 6, 1, "", "All"], [10, 6, 1, "", "Cpu"], [10, 6, 1, "", "Npu"]], "furiosa.runtime.profiler.profile": [[10, 5, 1, "", "export_chrome_trace"], [10, 5, 1, "", "get_cpu_pandas_dataframe"], [10, 5, 1, "", "get_npu_pandas_dataframe"], [10, 5, 1, "", "get_pandas_dataframe"], [10, 5, 1, "", "get_pandas_dataframe_with_filter"], [10, 5, 1, "", "pause"], [10, 5, 1, "", "print_external_operators"], [10, 5, 1, "", "print_inferences"], [10, 5, 1, "", "print_npu_executions"], [10, 5, 1, "", "print_npu_operators"], [10, 5, 1, "", "print_summary"], [10, 5, 1, "", "record"], [10, 5, 1, "", "resume"]], "furiosa.runtime.session": [[10, 3, 1, "", "AsyncSession"], [10, 3, 1, "", "CompletionQueue"], [10, 3, 1, "", "Session"], [10, 2, 1, "", "create"], [10, 2, 1, "", "create_async"]], "furiosa.runtime.session.AsyncSession": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "submit"]], "furiosa.runtime.session.CompletionQueue": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "recv"]], "furiosa.runtime.session.Session": [[10, 5, 1, "", "close"], [10, 6, 1, "", "model"], [10, 5, 1, "", "run"], [10, 5, 1, "", "run_with"]], "furiosa.runtime.sync": [[10, 3, 1, "", "Receiver"], [10, 3, 1, "", "Runner"], [10, 3, 1, "", "Runtime"], [10, 3, 1, "", "Submitter"], [10, 2, 1, "", "create_queue"], [10, 2, 1, "", "create_runner"]], "furiosa.runtime.sync.Receiver": [[10, 5, 1, "", "recv"]], "furiosa.runtime.sync.furiosa.runtime.sync.Runtime": [[10, 5, 1, "", "create_queue"], [10, 5, 1, "", "create_runner"]], "furiosa.runtime.tensor": [[10, 3, 1, "", "Tensor"], [10, 3, 1, "", "TensorArray"], [10, 2, 1, "", "numpy_dtype"], [10, 2, 1, "", "rand"], [10, 2, 1, "", "zeros"]], "furiosa.runtime.tensor.Tensor": [[10, 5, 1, "", "copy_from"], [10, 5, 1, "", "numpy"], [10, 6, 1, "", "numpy_dtype"], [10, 6, 1, "", "shape"], [10, 5, 1, "", "view"]], "furiosa.runtime.tensor.TensorArray": [[10, 5, 1, "", "is_empty"], [10, 5, 1, "", "numpy"], [10, 5, 1, "", "view"]], "furiosa.server": [[11, 0, 0, "-", "errors"], [19, 0, 0, "-", "utils"]], "furiosa.server.errors": [[11, 1, 1, "", "ModelNotFound"], [11, 1, 1, "", "ModelServerError"]], "": [[10, 7, 1, "-", "FURIOSA_DEVICES"], [10, 7, 1, "-", "FURIOSA_LOG_LEVEL"], [10, 7, 1, "-", "NPU_DEVNAME"]]}, "objtypes": {"0": "py:module", "1": "py:exception", "2": "py:function", "3": "py:class", "4": "py:attribute", "5": "py:method", "6": "py:property", "7": "std:envvar"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "exception", "Python exception"], "2": ["py", "function", "Python function"], "3": ["py", "class", "Python class"], "4": ["py", "attribute", "Python attribute"], "5": ["py", "method", "Python method"], "6": ["py", "property", "Python property"], "7": ["std", "envvar", "environment variable"]}, "titleterms": {"furiosa": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 28, 29, 31, 32, 34, 35, 36], "common": [0, 1], "packag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 28, 36, 40], "subpackag": [0, 2, 3, 4, 5, 11, 12, 14, 20], "submodul": [0, 1, 5, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], "error": [0, 1, 10, 11, 28], "modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], "nativ": [0, 31], "thread": 0, "util": [0, 5, 7, 19], "content": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22], "transport": 1, "base": [1, 22, 29], "file": [1, 30, 35, 43], "github": 1, "http": 1, "s3": 1, "optim": [2, 3, 4, 5, 6, 7, 8, 35], "frontend": [3, 4, 5, 6, 7], "onnx": [4, 5, 6, 7], "transform": [5, 6, 8], "convert_negative_pads_to_slic": 5, "convert_prelu_to_relu": 5, "fuse_batchnorm": 5, "fuse_gather_matmul": 5, "infer_squeeze_ax": 5, "polish_model": 5, "experiment": [6, 34], "eliminate_detection_postprocess": 6, "embedding_bag_port": 6, "fuse_div_for_bert": 6, "reify_conv_for_bert": 6, "check_model": 7, "inference_shap": 7, "version_check": 7, "interfac": [8, 10], "quantiz": [9, 29, 30, 31, 32, 37, 40, 41], "calibr": [9, 32], "editor": 9, "runtim": [10, 36, 37, 40], "variant": 10, "us": [10, 35, 39, 40], "legaci": [10, 40], "model": [10, 11, 20, 21, 25, 28, 31, 34, 35, 37, 40, 41, 43], "input": 10, "tensor": 10, "output": 10, "when": 10, "an": 10, "type": [10, 18], "object": 10, "devic": [10, 30, 38], "specif": [10, 26, 36], "metadata": 10, "runner": 10, "api": [10, 12, 13, 14, 15, 16, 23, 28, 31, 37, 40, 41], "inform": [10, 30, 31, 32], "infer": 10, "miscellan": 10, "queue": 10, "submit": 10, "receiv": 10, "profil": [10, 30, 31, 39], "diagnost": 10, "support": [10, 27, 28, 30, 32, 37, 38], "compil": [10, 28, 29, 30, 32, 33, 34, 35, 37], "const": 10, "env": 10, "session": [10, 28], "server": [11, 12, 13, 14, 15, 16, 17, 18, 19, 28, 36, 37, 40, 43], "registri": 11, "repositori": [11, 17, 21], "set": 11, "cli": 13, "main": 13, "grpc": [14, 15], "convert": 14, "servic": 14, "gener": 15, "model_repository_pb2": 15, "model_repository_pb2_grpc": 15, "predict_pb2": 15, "predict_pb2_grpc": 15, "rest": 16, "app": [16, 21], "endpoint": [16, 43], "handler": 17, "predict": [17, 18, 43], "model_repositori": 18, "namespac": 19, "loader": 19, "serv": [20, 21, 22, 25, 28, 29, 31, 43], "applic": 20, "telemetri": 20, "health": 21, "processor": 22, "imagenet": 22, "sdk": [23, 25, 29, 30, 31, 32, 33, 36, 37, 40], "0": [23, 25, 28, 29, 30, 31, 32], "10": [23, 25], "document": [23, 25], "bug": [24, 30], "report": 24, "furiosaai": [25, 26, 37, 40], "npu": [25, 26, 27, 28, 30, 35, 36, 38], "softwar": 25, "tutori": [25, 44], "exampl": [25, 44], "deploy": 25, "mlop": 25, "refer": [25, 42], "other": 25, "link": 25, "warboi": 26, "hw": 26, "hardwar": 26, "perform": [26, 30, 39], "see": 26, "also": 26, "list": [27, 34, 38], "oper": 27, "acceler": [27, 30], "releas": [28, 29, 30, 31, 32], "note": [28, 29, 30, 31, 32], "5": [28, 38], "improv": [28, 30, 31], "diagnosi": 28, "messag": 28, "handl": 28, "introduc": 28, "framework": [28, 31, 43], "command": [28, 34], "line": [28, 34], "manag": 28, "tool": [28, 30, 31, 32, 34], "furiosactl": [28, 34], "kubernet": [28, 37, 38], "6": 29, "how": [29, 30, 41], "upgrad": [29, 30], "major": [29, 31, 32], "chang": [29, 30, 31, 32], "python": [29, 30, 37, 40], "addit": 29, "fastapi": 29, "advanc": 29, "librari": 29, "7": 30, "compon": [30, 31, 32], "version": [30, 31, 32, 36, 43], "kei": 30, "more": 30, "cach": [30, 35], "hint": 30, "fix": 30, "plugin": [30, 38], "configur": [30, 36, 38, 40, 43], "8": 31, "instal": [31, 32, 33, 34, 36, 38, 40, 43], "latest": [31, 32], "toolkit": [31, 32, 34], "zoo": 31, "ad": [31, 36], "post": 31, "process": 31, "code": [31, 44], "9": 32, "method": 32, "c": [33, 37], "user": [33, 36, 40], "guid": [33, 40], "info": 34, "p": 34, "top": 34, "field": 34, "bench": 34, "benchmark": 34, "litmu": [34, 40], "compat": 34, "checker": 34, "target": 35, "batch": 35, "size": 35, "well": 35, "known": 35, "enf": 35, "behavior": 35, "accord": 35, "fc_cache_lifetim": 35, "driver": [36, 37], "firmwar": [36, 37], "minimum": 36, "requir": [36, 40], "apt": 36, "group": 36, "hold": 36, "unhold": 36, "updat": 36, "sw": 37, "stack": 37, "introduct": 37, "kernel": 37, "1": 38, "prepar": 38, "node": 38, "2": [38, 43], "featur": 38, "discoveri": 38, "3": 38, "label": 38, "argument": 38, "k8": 38, "4": 38, "creat": [38, 40], "pod": 38, "monitor": 38, "metric": 38, "export": 38, "collect": 38, "categori": 38, "trace": 39, "analysi": 39, "via": 39, "environ": [39, 40], "variabl": 39, "context": 39, "paus": 39, "resum": 39, "panda": 39, "datafram": 39, "execut": 40, "setup": 40, "conda": 40, "activ": 40, "isol": 40, "linux": 40, "extra": 40, "It": 41, "work": 41, "accuraci": 41, "run": 43, "kserv": 43, "protocol": 43}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinx": 57}, "alltitles": {"furiosa.common package": [[0, "furiosa-common-package"]], "Subpackages": [[0, "subpackages"], [2, "subpackages"], [3, "subpackages"], [4, "subpackages"], [5, "subpackages"], [11, "subpackages"], [12, "subpackages"], [14, "subpackages"], [20, "subpackages"]], "Submodules": [[0, "submodules"], [1, "submodules"], [5, "submodules"], [6, "submodules"], [7, "submodules"], [8, "submodules"], [9, "submodules"], [11, "submodules"], [13, "submodules"], [14, "submodules"], [15, "submodules"], [16, "submodules"], [17, "submodules"], [18, "submodules"], [19, "submodules"], [20, "submodules"], [21, "submodules"], [22, "submodules"]], "furiosa.common.error module": [[0, "module-furiosa.common.error"]], "furiosa.common.native module": [[0, "module-furiosa.common.native"]], "furiosa.common.thread module": [[0, "module-furiosa.common.thread"]], "furiosa.common.utils module": [[0, "module-furiosa.common.utils"]], "Module contents": [[0, "module-furiosa.common"], [1, "module-furiosa.common.transport"], [2, "module-furiosa.optimizer"], [3, "module-furiosa.optimizer.frontend"], [4, "module-furiosa.optimizer.frontend.onnx"], [5, "module-furiosa.optimizer.frontend.onnx.transformer"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"], [7, "module-furiosa.optimizer.frontend.onnx.utils"], [8, "module-furiosa.optimizer.interfaces"], [9, "module-furiosa.quantizer"], [11, "module-contents"], [12, "module-contents"], [13, "module-contents"], [14, "module-contents"], [15, "module-contents"], [16, "module-contents"], [17, "module-contents"], [18, "module-contents"], [20, "module-contents"], [21, "module-contents"], [22, "module-contents"]], "furiosa.common.transport package": [[1, "furiosa-common-transport-package"]], "furiosa.common.transport.base module": [[1, "module-furiosa.common.transport.base"]], "furiosa.common.transport.error module": [[1, "module-furiosa.common.transport.error"]], "furiosa.common.transport.file module": [[1, "module-furiosa.common.transport.file"]], "furiosa.common.transport.github module": [[1, "module-furiosa.common.transport.github"]], "furiosa.common.transport.http module": [[1, "module-furiosa.common.transport.http"]], "furiosa.common.transport.s3 module": [[1, "module-furiosa.common.transport.s3"]], "furiosa.optimizer package": [[2, "furiosa-optimizer-package"]], "furiosa.optimizer.frontend package": [[3, "furiosa-optimizer-frontend-package"]], "furiosa.optimizer.frontend.onnx package": [[4, "furiosa-optimizer-frontend-onnx-package"]], "furiosa.optimizer.frontend.onnx.transformer package": [[5, "furiosa-optimizer-frontend-onnx-transformer-package"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"]], "furiosa.optimizer.frontend.onnx.transformer.utils module": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"]], "furiosa.optimizer.frontend.onnx.transformer.experimental package": [[6, "furiosa-optimizer-frontend-onnx-transformer-experimental-package"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert module": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"]], "furiosa.optimizer.frontend.onnx.utils package": [[7, "furiosa-optimizer-frontend-onnx-utils-package"]], "furiosa.optimizer.frontend.onnx.utils.check_model module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker module": [[7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"]], "furiosa.optimizer.interfaces package": [[8, "furiosa-optimizer-interfaces-package"]], "furiosa.optimizer.interfaces.transformer module": [[8, "module-furiosa.optimizer.interfaces.transformer"]], "furiosa.quantizer package": [[9, "furiosa-quantizer-package"]], "furiosa.quantizer.calibrator module": [[9, "module-furiosa.quantizer.calibrator"]], "furiosa.quantizer.editor module": [[9, "module-furiosa.quantizer.editor"]], "furiosa.runtime package": [[10, "module-furiosa.runtime"]], "Runtime Variants": [[10, "runtime-variants"]], "Use of legacy modules": [[10, "use-of-legacy-modules"]], "Model Inputs": [[10, "model-inputs"]], "Tensor Inputs and Outputs": [[10, "tensor-inputs-and-outputs"]], "Legacy Interface": [[10, "legacy-interface"], [10, "legacy-runner-api"], [10, "legacy-queue-api"]], "When used as an input type": [[10, "legacy-tensorarray-as-input"], [10, "legacy-tensor-as-input"]], "Runtime Object": [[10, "runtime-object"]], "Device Specification": [[10, "device-specification"]], "Model Metadata": [[10, "model-metadata"]], "Runner API": [[10, "runner-api"]], "Model informations": [[10, null], [10, null], [10, null]], "Inference": [[10, null]], "Miscellaneous": [[10, null], [10, null], [10, null]], "Queue API": [[10, "queue-api"]], "Submitting": [[10, null]], "Receiving": [[10, null]], "Profiler": [[10, "profiler"], [31, "profiler"]], "Diagnostics": [[10, "diagnostics"]], "Legacy Supports": [[10, "legacy-supports"]], "furiosa.runtime.compiler": [[10, "module-furiosa.runtime.compiler"]], "furiosa.runtime.consts": [[10, "module-furiosa.runtime.consts"]], "furiosa.runtime.envs": [[10, "module-furiosa.runtime.envs"]], "furiosa.runtime.errors": [[10, "module-furiosa.runtime.errors"]], "furiosa.runtime.model": [[10, "module-furiosa.runtime.model"]], "furiosa.runtime.session": [[10, "module-furiosa.runtime.session"]], "furiosa.runtime.tensor": [[10, "module-furiosa.runtime.tensor"]], "furiosa.server package": [[11, "furiosa-server-package"]], "furiosa.server.errors module": [[11, "module-furiosa.server.errors"]], "furiosa.server.model module": [[11, "furiosa-server-model-module"]], "furiosa.server.registry module": [[11, "furiosa-server-registry-module"]], "furiosa.server.repository module": [[11, "furiosa-server-repository-module"]], "furiosa.server.server module": [[11, "furiosa-server-server-module"]], "furiosa.server.settings module": [[11, "furiosa-server-settings-module"]], "furiosa.server.api package": [[12, "furiosa-server-api-package"]], "furiosa.server.api.cli package": [[13, "furiosa-server-api-cli-package"]], "furiosa.server.api.cli.main module": [[13, "furiosa-server-api-cli-main-module"]], "furiosa.server.api.grpc package": [[14, "furiosa-server-api-grpc-package"]], "furiosa.server.api.grpc.converters module": [[14, "furiosa-server-api-grpc-converters-module"]], "furiosa.server.api.grpc.server module": [[14, "furiosa-server-api-grpc-server-module"]], "furiosa.server.api.grpc.servicers module": [[14, "furiosa-server-api-grpc-servicers-module"]], "furiosa.server.api.grpc.generated package": [[15, "furiosa-server-api-grpc-generated-package"]], "furiosa.server.api.grpc.generated.model_repository_pb2 module": [[15, "furiosa-server-api-grpc-generated-model-repository-pb2-module"]], "furiosa.server.api.grpc.generated.model_repository_pb2_grpc module": [[15, "furiosa-server-api-grpc-generated-model-repository-pb2-grpc-module"]], "furiosa.server.api.grpc.generated.predict_pb2 module": [[15, "furiosa-server-api-grpc-generated-predict-pb2-module"]], "furiosa.server.api.grpc.generated.predict_pb2_grpc module": [[15, "furiosa-server-api-grpc-generated-predict-pb2-grpc-module"]], "furiosa.server.api.rest package": [[16, "furiosa-server-api-rest-package"]], "furiosa.server.api.rest.app module": [[16, "furiosa-server-api-rest-app-module"]], "furiosa.server.api.rest.endpoints module": [[16, "furiosa-server-api-rest-endpoints-module"]], "furiosa.server.api.rest.server module": [[16, "furiosa-server-api-rest-server-module"]], "furiosa.server.handlers package": [[17, "furiosa-server-handlers-package"]], "furiosa.server.handlers.predict module": [[17, "furiosa-server-handlers-predict-module"]], "furiosa.server.handlers.repository module": [[17, "furiosa-server-handlers-repository-module"]], "furiosa.server.types package": [[18, "furiosa-server-types-package"]], "furiosa.server.types.model_repository module": [[18, "furiosa-server-types-model-repository-module"]], "furiosa.server.types.predict module": [[18, "furiosa-server-types-predict-module"]], "furiosa.server.utils namespace": [[19, "module-furiosa.server.utils"]], "furiosa.server.utils.loader module": [[19, "furiosa-server-utils-loader-module"]], "furiosa.serving package": [[20, "furiosa-serving-package"]], "furiosa.serving.application module": [[20, "furiosa-serving-application-module"]], "furiosa.serving.model module": [[20, "furiosa-serving-model-module"]], "furiosa.serving.telemetry module": [[20, "furiosa-serving-telemetry-module"]], "furiosa.serving.apps package": [[21, "furiosa-serving-apps-package"]], "furiosa.serving.apps.health module": [[21, "furiosa-serving-apps-health-module"]], "furiosa.serving.apps.model module": [[21, "furiosa-serving-apps-model-module"]], "furiosa.serving.apps.repository module": [[21, "furiosa-serving-apps-repository-module"]], "furiosa.serving.processors package": [[22, "furiosa-serving-processors-package"]], "furiosa.serving.processors.base module": [[22, "furiosa-serving-processors-base-module"]], "furiosa.serving.processors.imagenet module": [[22, "furiosa-serving-processors-imagenet-module"]], "Furiosa SDK 0.10.0 API Documentation": [[23, "furiosa-sdk-release-api-documentation"]], "Bug Report": [[24, "bug-report"]], "FuriosaAI NPU & SDK 0.10.0 Documents": [[25, "furiosaai-npu-sdk-release-documents"]], "FuriosaAI NPU": [[25, "furiosaai-npu"], [26, "furiosaai-npu"]], "FuriosaAI Software": [[25, "furiosaai-software"]], "FuriosaAI SDK Tutorial and Examples": [[25, "furiosaai-sdk-tutorial-and-examples"]], "Serving, Model Deployment, MLOps": [[25, "serving-model-deployment-mlops"]], "References": [[25, "references"], [42, "references"]], "Other Links": [[25, "other-links"]], "FuriosaAI Warboy": [[26, "furiosaai-warboy"]], "FuriosaAI Warboy HW Specifications": [[26, "furiosaai-warboy-hw-specifications"]], "Warboy Hardware Specification": [[26, "id2"]], "FuriosaAI Warboy Performance": [[26, "furiosaai-warboy-performance"]], "See Also": [[26, "see-also"]], "List of Supported Operators for NPU Acceleration": [[27, "list-of-supported-operators-for-npu-acceleration"]], "Operators Accelerated on NPU": [[27, "id1"]], "Release Notes - 0.5.0": [[28, "release-notes-0-5-0"]], "Compiler Improvement": [[28, "compiler-improvement"]], "Session API Improvement": [[28, "session-api-improvement"]], "Error Diagnosis Message & Error Handling Improvements": [[28, "error-diagnosis-message-error-handling-improvements"]], "Introducing Furiosa Server (serving framework)": [[28, "introducing-furiosa-server-serving-framework"]], "Introducing Furiosa Model package": [[28, "introducing-furiosa-model-package"]], "Command line NPU management tool: furiosactl": [[28, "command-line-npu-management-tool-furiosactl"]], "Kubernetes support": [[28, "kubernetes-support"]], "Release Notes - 0.6.0": [[29, "release-notes-0-6-0"]], "How to upgrade": [[29, "how-to-upgrade"], [30, "how-to-upgrade"]], "Major changes": [[29, "major-changes"], [31, "major-changes"], [32, "major-changes"]], "Compiler": [[29, "compiler"], [32, "compiler"], [35, "compiler"], [37, "compiler"]], "Quantizer": [[29, "quantizer"], [40, "quantizer"]], "Python SDK": [[29, "python-sdk"]], "Furiosa Serving: Addition of FastAPI-based advanced serving library": [[29, "furiosa-serving-addition-of-fastapi-based-advanced-serving-library"]], "Release Notes - 0.7.0": [[30, "release-notes-0-7-0"]], "component version information": [[30, "id1"]], "Key changes": [[30, "key-changes"]], "Compiler - More NPU acceleration supports": [[30, "compiler-more-npu-acceleration-supports"]], "Compiler - Compiler Cache": [[30, "compiler-compiler-cache"]], "Compiler - Compiler Hint": [[30, "compiler-compiler-hint"]], "Performance Profiling Tools": [[30, "performance-profiling-tools"]], "Improvements/Bug fixes of Python SDK": [[30, "improvements-bug-fixes-of-python-sdk"]], "Improvements/Bug fixes of Quantization tools": [[30, "improvements-bug-fixes-of-quantization-tools"]], "Device Plugin - Configuration file support": [[30, "device-plugin-configuration-file-support"]], "Release Notes - 0.8.0": [[31, "release-notes-0-8-0"]], "Component Version Information": [[31, "id3"], [32, "id1"]], "Installing the latest SDK": [[31, "installing-the-latest-sdk"], [32, "installing-the-latest-sdk"]], "Improvements to serving framework API": [[31, "improvements-to-serving-framework-api"]], "Quantization tool": [[31, "quantization-tool"], [32, "quantization-tool"]], "furiosa-toolkit": [[31, "furiosa-toolkit"], [32, "furiosa-toolkit"], [34, "furiosa-toolkit"]], "Model Zoo API improvements, added models, and added native post-processing code": [[31, "model-zoo-api-improvements-added-models-and-added-native-post-processing-code"]], "Release Notes - 0.9.0": [[32, "release-notes-0-9-0"]], "Supported Calibration Methods": [[32, "id2"]], "C SDK installation and user guide": [[33, "c-sdk-installation-and-user-guide"]], "C SDK installation": [[33, "c-sdk-installation"]], "Compiling with C SDK": [[33, "compiling-with-c-sdk"]], "Command Line Tools": [[34, "command-line-tools"]], "furiosa-toolkit installation": [[34, "furiosa-toolkit-installation"]], "furiosactl": [[34, "furiosactl"]], "furiosactl info": [[34, "furiosactl-info"]], "furiosactl list": [[34, "furiosactl-list"]], "furiosactl ps": [[34, "furiosactl-ps"]], "furiosactl top (experimental)": [[34, "furiosactl-top-experimental"]], "furiosa top fields": [[34, "id1"]], "furiosa-bench (Benchmark Tool)": [[34, "furiosa-bench-benchmark-tool"]], "furiosa": [[34, "furiosa"]], "furiosa compile": [[34, "furiosa-compile"]], "furiosa litmus (Model Compatibility Checker)": [[34, "furiosa-litmus-model-compatibility-checker"]], "furiosa-compiler": [[35, "furiosa-compiler"]], "Target NPUs": [[35, "id4"]], "Optimal Batch Size for Well-known Models": [[35, "id5"]], "Using ENF files": [[35, "using-enf-files"]], "Compiler Cache": [[35, "compiler-cache"]], "Cache behaviors according to FC_CACHE_LIFETIME": [[35, "id6"]], "Driver, Firmware, and Runtime Installation": [[36, "driver-firmware-and-runtime-installation"]], "Minimum requirements for SDK installation": [[36, "minimum-requirements-for-sdk-installation"]], "APT server configuration": [[36, "apt-server-configuration"]], "Installing required packages.": [[36, "installing-required-packages"]], "Adding a user to the furiosa Group": [[36, "adding-a-user-to-the-furiosa-group"]], "Holding/unholding installed version": [[36, "holding-unholding-installed-version"]], "Installing a specific version": [[36, "installing-a-specific-version"]], "NPU Firmware Update": [[36, "npu-firmware-update"]], "FuriosaAI SW Stack Introduction": [[37, "furiosaai-sw-stack-introduction"]], "Kernel Driver and Firmware": [[37, "kernel-driver-and-firmware"]], "Runtime": [[37, "runtime"]], "Python SDK and C SDK": [[37, "python-sdk-and-c-sdk"]], "Model quantizer API": [[37, "model-quantizer-api"]], "Model Server": [[37, "model-server"], [40, "model-server"]], "Kubernetes Support": [[37, "kubernetes-support"], [38, "kubernetes-support"]], "1. Preparing NPU nodes": [[38, "preparing-npu-nodes"]], "2. Installing Node Feature Discovery": [[38, "installing-node-feature-discovery"]], "3. Installing Device Plugin and NPU Feature Discovery": [[38, "installing-device-plugin-and-npu-feature-discovery"]], "NPU Node Labels": [[38, "id1"]], "Device Plugin Configuration": [[38, "device-plugin-configuration"]], "arguments of k8s-device-plugin": [[38, "id2"]], "4. Creating a Pod with NPUs": [[38, "creating-a-pod-with-npus"]], "5. NPU monitoring": [[38, "npu-monitoring"]], "npu-metrics-exporter collection category list": [[38, "id5"]], "Performance Profiling": [[39, "performance-profiling"]], "Trace Analysis": [[39, "trace-analysis"]], "Tracing via Environment Variable": [[39, "tracing-via-environment-variable"]], "Tracing via Profiler Context": [[39, "tracing-via-profiler-context"]], "Pause/Resume of Profiler Context": [[39, "pause-resume-of-profiler-context"]], "Trace analysis using Pandas DataFrame": [[39, "trace-analysis-using-pandas-dataframe"]], "Python SDK installation and user guide": [[40, "python-sdk-installation-and-user-guide"]], "Requirements": [[40, "requirements"]], "Python execution environment setup": [[40, "python-execution-environment-setup"]], "Python environment configuration with Conda": [[40, "python-environment-configuration-with-conda"]], "Creating and activating isolated Python execution environment": [[40, "creating-and-activating-isolated-python-execution-environment"]], "Configuring Python environment using Linux packages": [[40, "configuring-python-environment-using-linux-packages"]], "Installing Python SDK package": [[40, "installing-python-sdk-package"]], "Extra packages": [[40, "extra-packages"]], "Legacy Runtime/API": [[40, "legacy-runtime-api"]], "FuriosaAI Models": [[40, "furiosaai-models"]], "Litmus": [[40, "litmus"]], "Model Quantization": [[41, "model-quantization"]], "How It Works": [[41, "how-it-works"]], "Accuracy of Quantized Models": [[41, "accuracy-of-quantized-models"]], "Quantization Accuracy": [[41, "id1"]], "Model Quantization APIs": [[41, "model-quantization-apis"]], "Model Server (Serving Framework)": [[43, "model-server-serving-framework"]], "Installation": [[43, "installation"]], "Running a Model Server": [[43, "running-a-model-server"]], "Running a Model Server with a Configuration File": [[43, "running-a-model-server-with-a-configuration-file"]], "Endpoints": [[43, "endpoints"]], "Endpoints of KServe Predict Protocol Version 2": [[43, "id1"]], "Tutorial and Code Examples": [[44, "tutorial-and-code-examples"]], "Tutorial": [[44, "id1"]], "Code Examples": [[44, "code-examples"]]}, "indexentries": {"error (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.ERROR"]], "furiosaerror": [[0, "furiosa.common.error.FuriosaError"]], "furiosaversioninfo (class in furiosa.common.utils)": [[0, "furiosa.common.utils.FuriosaVersionInfo"]], "info (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.INFO"]], "loglevel (class in furiosa.common.native)": [[0, "furiosa.common.native.LogLevel"]], "off (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.OFF"]], "warn (furiosa.common.native.loglevel attribute)": [[0, "furiosa.common.native.LogLevel.WARN"]], "asynchronous() (in module furiosa.common.thread)": [[0, "furiosa.common.thread.asynchronous"]], "eprint() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.eprint"]], "find_global_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_global_lib_path"]], "find_native_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_native_lib_path"]], "find_native_libs() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_native_libs"]], "find_user_lib_path() (in module furiosa.common.native)": [[0, "furiosa.common.native.find_user_lib_path"]], "furiosa.common": [[0, "module-furiosa.common"]], "furiosa.common.error": [[0, "module-furiosa.common.error"]], "furiosa.common.native": [[0, "module-furiosa.common.native"]], "furiosa.common.thread": [[0, "module-furiosa.common.thread"]], "furiosa.common.utils": [[0, "module-furiosa.common.utils"]], "get_sdk_version() (in module furiosa.common)": [[0, "furiosa.common.get_sdk_version"]], "get_sdk_version() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.get_sdk_version"]], "is_err() (in module furiosa.common.error)": [[0, "furiosa.common.error.is_err"]], "is_ok() (in module furiosa.common.error)": [[0, "furiosa.common.error.is_ok"]], "module": [[0, "module-furiosa.common"], [0, "module-furiosa.common.error"], [0, "module-furiosa.common.native"], [0, "module-furiosa.common.thread"], [0, "module-furiosa.common.utils"], [1, "module-furiosa.common.transport"], [1, "module-furiosa.common.transport.base"], [1, "module-furiosa.common.transport.error"], [1, "module-furiosa.common.transport.file"], [1, "module-furiosa.common.transport.github"], [1, "module-furiosa.common.transport.http"], [1, "module-furiosa.common.transport.s3"], [2, "module-furiosa.optimizer"], [3, "module-furiosa.optimizer.frontend"], [4, "module-furiosa.optimizer.frontend.onnx"], [5, "module-furiosa.optimizer.frontend.onnx.transformer"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"], [5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"], [6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"], [7, "module-furiosa.optimizer.frontend.onnx.utils"], [7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"], [7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"], [7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"], [8, "module-furiosa.optimizer.interfaces"], [8, "module-furiosa.optimizer.interfaces.transformer"], [9, "module-furiosa.quantizer"], [9, "module-furiosa.quantizer.calibrator"], [9, "module-furiosa.quantizer.editor"], [10, "module-furiosa.runtime"], [10, "module-furiosa.runtime.compiler"], [10, "module-furiosa.runtime.consts"], [10, "module-furiosa.runtime.envs"], [10, "module-furiosa.runtime.errors"], [10, "module-furiosa.runtime.model"], [10, "module-furiosa.runtime.profiler"], [10, "module-furiosa.runtime.session"], [10, "module-furiosa.runtime.tensor"], [11, "module-furiosa.server.errors"], [19, "module-furiosa.server.utils"]], "removeprefix() (in module furiosa.common.utils)": [[0, "furiosa.common.utils.removeprefix"]], "synchronous() (in module furiosa.common.thread)": [[0, "furiosa.common.thread.synchronous"]], "filetransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.FileTransport"]], "filetransport (class in furiosa.common.transport.file)": [[1, "furiosa.common.transport.file.FileTransport"]], "githubtransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.GithubTransport"]], "githubtransport (class in furiosa.common.transport.github)": [[1, "furiosa.common.transport.github.GithubTransport"]], "httptransport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.HTTPTransport"]], "httptransport (class in furiosa.common.transport.http)": [[1, "furiosa.common.transport.http.HTTPTransport"]], "s3transport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.S3Transport"]], "s3transport (class in furiosa.common.transport.s3)": [[1, "furiosa.common.transport.s3.S3Transport"]], "transport (class in furiosa.common.transport)": [[1, "furiosa.common.transport.Transport"]], "transport (class in furiosa.common.transport.base)": [[1, "furiosa.common.transport.base.Transport"]], "transporterror": [[1, "furiosa.common.transport.error.TransportError"]], "transportnotfound": [[1, "furiosa.common.transport.error.TransportNotFound"]], "cache_directory (furiosa.common.transport.transport property)": [[1, "furiosa.common.transport.Transport.cache_directory"]], "cache_directory (furiosa.common.transport.base.transport property)": [[1, "furiosa.common.transport.base.Transport.cache_directory"]], "download() (furiosa.common.transport.filetransport method)": [[1, "furiosa.common.transport.FileTransport.download"]], "download() (furiosa.common.transport.githubtransport method)": [[1, "furiosa.common.transport.GithubTransport.download"]], "download() (furiosa.common.transport.httptransport method)": [[1, "furiosa.common.transport.HTTPTransport.download"]], "download() (furiosa.common.transport.s3transport method)": [[1, "furiosa.common.transport.S3Transport.download"]], "download() (furiosa.common.transport.transport method)": [[1, "furiosa.common.transport.Transport.download"]], "download() (furiosa.common.transport.base.transport method)": [[1, "furiosa.common.transport.base.Transport.download"]], "download() (furiosa.common.transport.file.filetransport method)": [[1, "furiosa.common.transport.file.FileTransport.download"]], "download() (furiosa.common.transport.github.githubtransport method)": [[1, "furiosa.common.transport.github.GithubTransport.download"]], "download() (furiosa.common.transport.http.httptransport method)": [[1, "furiosa.common.transport.http.HTTPTransport.download"]], "download() (furiosa.common.transport.s3.s3transport method)": [[1, "furiosa.common.transport.s3.S3Transport.download"]], "download() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.download"]], "furiosa.common.transport": [[1, "module-furiosa.common.transport"]], "furiosa.common.transport.base": [[1, "module-furiosa.common.transport.base"]], "furiosa.common.transport.error": [[1, "module-furiosa.common.transport.error"]], "furiosa.common.transport.file": [[1, "module-furiosa.common.transport.file"]], "furiosa.common.transport.github": [[1, "module-furiosa.common.transport.github"]], "furiosa.common.transport.http": [[1, "module-furiosa.common.transport.http"]], "furiosa.common.transport.s3": [[1, "module-furiosa.common.transport.s3"]], "is_relative() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.is_relative"]], "is_supported() (furiosa.common.transport.filetransport static method)": [[1, "furiosa.common.transport.FileTransport.is_supported"]], "is_supported() (furiosa.common.transport.githubtransport static method)": [[1, "furiosa.common.transport.GithubTransport.is_supported"]], "is_supported() (furiosa.common.transport.httptransport static method)": [[1, "furiosa.common.transport.HTTPTransport.is_supported"]], "is_supported() (furiosa.common.transport.s3transport static method)": [[1, "furiosa.common.transport.S3Transport.is_supported"]], "is_supported() (furiosa.common.transport.transport static method)": [[1, "furiosa.common.transport.Transport.is_supported"]], "is_supported() (furiosa.common.transport.base.transport static method)": [[1, "furiosa.common.transport.base.Transport.is_supported"]], "is_supported() (furiosa.common.transport.file.filetransport static method)": [[1, "furiosa.common.transport.file.FileTransport.is_supported"]], "is_supported() (furiosa.common.transport.github.githubtransport static method)": [[1, "furiosa.common.transport.github.GithubTransport.is_supported"]], "is_supported() (furiosa.common.transport.http.httptransport static method)": [[1, "furiosa.common.transport.http.HTTPTransport.is_supported"]], "is_supported() (furiosa.common.transport.s3.s3transport static method)": [[1, "furiosa.common.transport.s3.S3Transport.is_supported"]], "parse() (furiosa.common.transport.githubtransport method)": [[1, "furiosa.common.transport.GithubTransport.parse"]], "parse() (furiosa.common.transport.s3transport static method)": [[1, "furiosa.common.transport.S3Transport.parse"]], "parse() (furiosa.common.transport.github.githubtransport method)": [[1, "furiosa.common.transport.github.GithubTransport.parse"]], "parse() (furiosa.common.transport.s3.s3transport static method)": [[1, "furiosa.common.transport.s3.S3Transport.parse"]], "read (furiosa.common.transport.filetransport attribute)": [[1, "furiosa.common.transport.FileTransport.read"]], "read (furiosa.common.transport.githubtransport attribute)": [[1, "furiosa.common.transport.GithubTransport.read"]], "read (furiosa.common.transport.httptransport attribute)": [[1, "furiosa.common.transport.HTTPTransport.read"]], "read (furiosa.common.transport.s3transport attribute)": [[1, "furiosa.common.transport.S3Transport.read"]], "read (furiosa.common.transport.transport attribute)": [[1, "furiosa.common.transport.Transport.read"]], "read (furiosa.common.transport.base.transport attribute)": [[1, "furiosa.common.transport.base.Transport.read"]], "read (furiosa.common.transport.file.filetransport attribute)": [[1, "furiosa.common.transport.file.FileTransport.read"]], "read (furiosa.common.transport.github.githubtransport attribute)": [[1, "furiosa.common.transport.github.GithubTransport.read"]], "read (furiosa.common.transport.http.httptransport attribute)": [[1, "furiosa.common.transport.http.HTTPTransport.read"]], "read (furiosa.common.transport.s3.s3transport attribute)": [[1, "furiosa.common.transport.s3.S3Transport.read"]], "scheme (furiosa.common.transport.filetransport attribute)": [[1, "furiosa.common.transport.FileTransport.scheme"]], "scheme (furiosa.common.transport.s3transport attribute)": [[1, "furiosa.common.transport.S3Transport.scheme"]], "scheme (furiosa.common.transport.file.filetransport attribute)": [[1, "furiosa.common.transport.file.FileTransport.scheme"]], "scheme (furiosa.common.transport.s3.s3transport attribute)": [[1, "furiosa.common.transport.s3.S3Transport.scheme"]], "schemes (furiosa.common.transport.githubtransport attribute)": [[1, "furiosa.common.transport.GithubTransport.schemes"]], "schemes (furiosa.common.transport.httptransport attribute)": [[1, "furiosa.common.transport.HTTPTransport.schemes"]], "schemes (furiosa.common.transport.github.githubtransport attribute)": [[1, "furiosa.common.transport.github.GithubTransport.schemes"]], "schemes (furiosa.common.transport.http.httptransport attribute)": [[1, "furiosa.common.transport.http.HTTPTransport.schemes"]], "supported() (in module furiosa.common.transport)": [[1, "furiosa.common.transport.supported"]], "furiosa.optimizer": [[2, "module-furiosa.optimizer"]], "optimize_model() (in module furiosa.optimizer)": [[2, "furiosa.optimizer.optimize_model"]], "furiosa.optimizer.frontend": [[3, "module-furiosa.optimizer.frontend"]], "furiosa.optimizer.frontend.onnx": [[4, "module-furiosa.optimizer.frontend.onnx"]], "optimize_model() (in module furiosa.optimizer.frontend.onnx)": [[4, "furiosa.optimizer.frontend.onnx.optimize_model"]], "convertnegativepadstoslice (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice"]], "convertprelutorelu (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu"]], "fusebatchnorm (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm"]], "fusegathermatmul (class in furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul"]], "infersqueezeaxes (class in furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes"]], "onnxtransformer (class in furiosa.optimizer.frontend.onnx.transformer)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1"]], "pattern_1 (class in furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2"]], "pattern_2 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2"]], "pattern_3 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3"]], "pattern_4 (class in furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4"]], "polishmodel (class in furiosa.optimizer.frontend.onnx.transformer.polish_model)": [[5, "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel"]], "bridge_disconnected_nodes() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.bridge_disconnected_nodes"]], "build_optimized_model() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.build_optimized_model"]], "check_runnable (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.check_runnable"]], "check_value_info() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.check_value_info"]], "copy_value_info() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.copy_value_info"]], "eliminate_initializer_from_graph_input() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_initializer_from_graph_input"]], "eliminate_unused_initializer() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_initializer"]], "eliminate_unused_input() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_input"]], "eliminate_unused_output() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_output"]], "eliminate_unused_protos() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_protos"]], "eliminate_unused_value_info() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.eliminate_unused_value_info"]], "find_next_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.find_next_node"]], "find_prev_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.find_prev_node"]], "fix_batch_size_as_one() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.fix_batch_size_as_one"]], "fixed_point() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.fixed_point"]], "furiosa.optimizer.frontend.onnx.transformer": [[5, "module-furiosa.optimizer.frontend.onnx.transformer"]], "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice"]], "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm"]], "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul"]], "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes"]], "furiosa.optimizer.frontend.onnx.transformer.polish_model": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.polish_model"]], "furiosa.optimizer.frontend.onnx.transformer.utils": [[5, "module-furiosa.optimizer.frontend.onnx.transformer.utils"]], "get_attribute() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_attribute"]], "get_data_node_input() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_data_node_input"]], "get_init_node_input() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_init_node_input"]], "get_initializer_array() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_initializer_array"]], "get_map_values() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_map_values"]], "get_node_attributes() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_attributes"]], "get_node_input_names() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_input_names"]], "get_node_output_names() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.get_node_output_names"]], "get_value_info_dtype() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_value_info_dtype"]], "get_value_info_shape() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.get_value_info_shape"]], "is_op_type() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.is_op_type"]], "is_same_shape() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.is_same_shape"]], "make_initializer_name_unique() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.make_initializer_name_unique"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.make_new_init"]], "make_new_init() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.make_new_init"]], "make_new_init_and_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.make_new_init_and_vi"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 static method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.make_new_node"]], "make_new_node() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.make_new_node"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.make_new_vi"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.make_new_vi"]], "make_new_vi() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.make_new_vi"]], "make_unhashables_unique() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.make_unhashables_unique"]], "name_nodes() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.name_nodes"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_condition_checker"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_condition_checker"]], "pattern_matcher() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pattern_matcher"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_2 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_matching"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_matching"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_2 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_2.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_3 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_3.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.pattern_4 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.Pattern_4.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.Pattern_1.pattern_to_match"]], "pattern_to_match (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.pattern_1 attribute)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.Pattern_1.pattern_to_match"]], "pop_multiple_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pop_multiple_optimizer_map"]], "pop_single_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.pop_single_optimizer_map"]], "rebuild_model() (in module furiosa.optimizer.frontend.onnx.transformer.utils)": [[5, "furiosa.optimizer.frontend.onnx.transformer.utils.rebuild_model"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.convertnegativepadstoslice method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_negative_pads_to_slice.ConvertNegativePadsToSlice.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.convertprelutorelu method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.convert_prelu_to_relu.ConvertPReluToRelu.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.fusebatchnorm method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_batchnorm.FuseBatchNorm.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.fusegathermatmul method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.fuse_gather_matmul.FuseGatherMatMul.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.infersqueezeaxes method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.infer_squeeze_axes.InferSqueezeAxes.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.polish_model.polishmodel method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.polish_model.PolishModel.transform"]], "transform_to_eliminate() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform_to_eliminate"]], "transform_to_fuse() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.transform_to_fuse"]], "traverse_prev_node() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.traverse_prev_node"]], "update_graph_fields() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_graph_fields"]], "update_multiple_initializer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_initializer_map"]], "update_multiple_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_optimizer_map"]], "update_multiple_value_info_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_multiple_value_info_map"]], "update_single_initializer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_initializer_map"]], "update_single_optimizer_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_optimizer_map"]], "update_single_value_info_map() (furiosa.optimizer.frontend.onnx.transformer.onnxtransformer method)": [[5, "furiosa.optimizer.frontend.onnx.transformer.ONNXTransformer.update_single_value_info_map"]], "eliminatessddetectionpostprocess (class in furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess"]], "embeddingbagpattern (class in furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern"]], "embeddingbagporting (class in furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting"]], "fusedivforbert (class in furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert"]], "reifyconvforbert (class in furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert"]], "check_condition_3() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.check_condition_3"]], "furiosa.optimizer.frontend.onnx.transformer.experimental": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert"]], "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert": [[6, "module-furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert"]], "pattern_condition_checker() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.pattern_condition_checker"]], "pattern_matching() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagpattern method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPattern.pattern_matching"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.eliminatessddetectionpostprocess method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.eliminate_detection_postprocess.EliminateSSDDetectionPostprocess.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.embeddingbagporting method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.embedding_bag_porting.EmbeddingBagPorting.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.fusedivforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert.transform"]], "transform() (furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.reifyconvforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert.transform"]], "transform_matmul_add() (furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.fusedivforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.fuse_div_for_bert.FuseDivForBert.transform_matmul_add"]], "transform_matmul_add() (furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.reifyconvforbert method)": [[6, "furiosa.optimizer.frontend.onnx.transformer.experimental.reify_conv_for_bert.ReifyConvForBert.transform_matmul_add"]], "checkversion (class in furiosa.optimizer.frontend.onnx.utils.version_checker)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion"]], "inferenceshape (class in furiosa.optimizer.frontend.onnx.utils.inference_shape)": [[7, "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape"]], "check_model() (in module furiosa.optimizer.frontend.onnx.utils.check_model)": [[7, "furiosa.optimizer.frontend.onnx.utils.check_model.check_model"]], "furiosa.optimizer.frontend.onnx.utils": [[7, "module-furiosa.optimizer.frontend.onnx.utils"]], "furiosa.optimizer.frontend.onnx.utils.check_model": [[7, "module-furiosa.optimizer.frontend.onnx.utils.check_model"]], "furiosa.optimizer.frontend.onnx.utils.inference_shape": [[7, "module-furiosa.optimizer.frontend.onnx.utils.inference_shape"]], "furiosa.optimizer.frontend.onnx.utils.version_checker": [[7, "module-furiosa.optimizer.frontend.onnx.utils.version_checker"]], "inference_shape() (furiosa.optimizer.frontend.onnx.utils.inference_shape.inferenceshape method)": [[7, "furiosa.optimizer.frontend.onnx.utils.inference_shape.InferenceShape.inference_shape"]], "opset_bound (furiosa.optimizer.frontend.onnx.utils.version_checker.checkversion attribute)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion.opset_bound"]], "transform() (furiosa.optimizer.frontend.onnx.utils.version_checker.checkversion method)": [[7, "furiosa.optimizer.frontend.onnx.utils.version_checker.CheckVersion.transform"]], "transformer (class in furiosa.optimizer.interfaces.transformer)": [[8, "furiosa.optimizer.interfaces.transformer.Transformer"]], "furiosa.optimizer.interfaces": [[8, "module-furiosa.optimizer.interfaces"]], "furiosa.optimizer.interfaces.transformer": [[8, "module-furiosa.optimizer.interfaces.transformer"]], "transform() (furiosa.optimizer.interfaces.transformer.transformer method)": [[8, "furiosa.optimizer.interfaces.transformer.Transformer.transform"]], "calibrationmethod (class in furiosa.quantizer)": [[9, "furiosa.quantizer.CalibrationMethod"]], "calibrationmethod (class in furiosa.quantizer.calibrator)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod"]], "calibrator (class in furiosa.quantizer)": [[9, "furiosa.quantizer.Calibrator"]], "calibrator (class in furiosa.quantizer.calibrator)": [[9, "furiosa.quantizer.calibrator.Calibrator"]], "entropy_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.ENTROPY_ASYM"], [9, "id10"]], "entropy_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.ENTROPY_ASYM"], [9, "id0"]], "entropy_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.ENTROPY_SYM"], [9, "id11"]], "entropy_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.ENTROPY_SYM"], [9, "id1"]], "int8 (furiosa.quantizer.tensortype attribute)": [[9, "furiosa.quantizer.TensorType.INT8"]], "int8 (furiosa.quantizer.editor.tensortype attribute)": [[9, "furiosa.quantizer.editor.TensorType.INT8"]], "min_max_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MIN_MAX_ASYM"], [9, "id12"]], "min_max_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MIN_MAX_ASYM"], [9, "id2"]], "min_max_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MIN_MAX_SYM"], [9, "id13"]], "min_max_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MIN_MAX_SYM"], [9, "id3"]], "mse_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MSE_ASYM"], [9, "id14"]], "mse_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MSE_ASYM"], [9, "id4"]], "mse_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.MSE_SYM"], [9, "id15"]], "mse_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.MSE_SYM"], [9, "id5"]], "modeleditor (class in furiosa.quantizer)": [[9, "furiosa.quantizer.ModelEditor"]], "modeleditor (class in furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.ModelEditor"]], "percentile_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.PERCENTILE_ASYM"], [9, "id16"]], "percentile_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.PERCENTILE_ASYM"], [9, "id6"]], "percentile_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.PERCENTILE_SYM"], [9, "id17"]], "percentile_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.PERCENTILE_SYM"], [9, "id7"]], "sqnr_asym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.SQNR_ASYM"], [9, "id18"]], "sqnr_asym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.SQNR_ASYM"], [9, "id8"]], "sqnr_sym (furiosa.quantizer.calibrationmethod attribute)": [[9, "furiosa.quantizer.CalibrationMethod.SQNR_SYM"], [9, "id19"]], "sqnr_sym (furiosa.quantizer.calibrator.calibrationmethod attribute)": [[9, "furiosa.quantizer.calibrator.CalibrationMethod.SQNR_SYM"], [9, "id9"]], "tensortype (class in furiosa.quantizer)": [[9, "furiosa.quantizer.TensorType"]], "tensortype (class in furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.TensorType"]], "uint8 (furiosa.quantizer.tensortype attribute)": [[9, "furiosa.quantizer.TensorType.UINT8"]], "uint8 (furiosa.quantizer.editor.tensortype attribute)": [[9, "furiosa.quantizer.editor.TensorType.UINT8"]], "collect_data() (furiosa.quantizer.calibrator method)": [[9, "furiosa.quantizer.Calibrator.collect_data"]], "collect_data() (furiosa.quantizer.calibrator.calibrator method)": [[9, "furiosa.quantizer.calibrator.Calibrator.collect_data"]], "compute_range() (furiosa.quantizer.calibrator method)": [[9, "furiosa.quantizer.Calibrator.compute_range"]], "compute_range() (furiosa.quantizer.calibrator.calibrator method)": [[9, "furiosa.quantizer.calibrator.Calibrator.compute_range"]], "convert_input_type() (furiosa.quantizer.modeleditor method)": [[9, "furiosa.quantizer.ModelEditor.convert_input_type"]], "convert_input_type() (furiosa.quantizer.editor.modeleditor method)": [[9, "furiosa.quantizer.editor.ModelEditor.convert_input_type"]], "convert_output_type() (furiosa.quantizer.modeleditor method)": [[9, "furiosa.quantizer.ModelEditor.convert_output_type"]], "convert_output_type() (furiosa.quantizer.editor.modeleditor method)": [[9, "furiosa.quantizer.editor.ModelEditor.convert_output_type"]], "furiosa.quantizer": [[9, "module-furiosa.quantizer"]], "furiosa.quantizer.calibrator": [[9, "module-furiosa.quantizer.calibrator"]], "furiosa.quantizer.editor": [[9, "module-furiosa.quantizer.editor"]], "get_output_names() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.get_output_names"]], "get_output_names() (in module furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.get_output_names"]], "get_pure_input_names() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.get_pure_input_names"]], "get_pure_input_names() (in module furiosa.quantizer.editor)": [[9, "furiosa.quantizer.editor.get_pure_input_names"]], "quantize() (in module furiosa.quantizer)": [[9, "furiosa.quantizer.quantize"]], "all (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.All"]], "apiclientinitfailed": [[10, "furiosa.runtime.errors.ApiClientInitFailed"]], "asyncsession (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.AsyncSession"]], "axis (class in furiosa.runtime)": [[10, "furiosa.runtime.Axis"]], "batch (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.BATCH"]], "bfloat16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.BFLOAT16"]], "channel (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.CHANNEL"]], "chrometrace (furiosa.runtime.profiler.recordformat property)": [[10, "furiosa.runtime.profiler.RecordFormat.ChromeTrace"]], "compilationfailed": [[10, "furiosa.runtime.errors.CompilationFailed"]], "completionqueue (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.CompletionQueue"]], "cpu (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.Cpu"]], "datatype (class in furiosa.runtime)": [[10, "furiosa.runtime.DataType"]], "devicebusy": [[10, "furiosa.runtime.errors.DeviceBusy"]], "float16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.FLOAT16"]], "float32 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.FLOAT32"]], "furiosa_devices": [[10, "envvar-FURIOSA_DEVICES"], [10, "index-1"]], "furiosa_log_level": [[10, "envvar-FURIOSA_LOG_LEVEL"]], "furiosaruntimeerror": [[10, "furiosa.runtime.FuriosaRuntimeError"]], "furiosaruntimewarning": [[10, "furiosa.runtime.FuriosaRuntimeWarning"]], "height (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.HEIGHT"]], "int16 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT16"]], "int32 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT32"]], "int64 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT64"]], "int8 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.INT8"]], "incompatibleapiclienterror": [[10, "furiosa.runtime.errors.IncompatibleApiClientError"]], "incompatiblemodel": [[10, "furiosa.runtime.errors.IncompatibleModel"]], "input queue": [[10, "term-Input-queue"]], "internalerror": [[10, "furiosa.runtime.errors.InternalError"]], "invalidcompilerconfig": [[10, "furiosa.runtime.errors.InvalidCompilerConfig"]], "invalidinput": [[10, "furiosa.runtime.errors.InvalidInput"]], "invalidsessionoption": [[10, "furiosa.runtime.errors.InvalidSessionOption"]], "invalidyamlexception": [[10, "furiosa.runtime.errors.InvalidYamlException"]], "model (class in furiosa.runtime)": [[10, "furiosa.runtime.Model"]], "modelsource (class in furiosa.runtime)": [[10, "furiosa.runtime.ModelSource"]], "npu_devname": [[10, "envvar-NPU_DEVNAME"], [10, "index-0"]], "noapikeyexception": [[10, "furiosa.runtime.errors.NoApiKeyException"]], "npu (furiosa.runtime.profiler.resource property)": [[10, "furiosa.runtime.profiler.Resource.Npu"]], "output queue": [[10, "term-Output-queue"]], "pandasdataframe (furiosa.runtime.profiler.recordformat property)": [[10, "furiosa.runtime.profiler.RecordFormat.PandasDataFrame"]], "queuewaittimeout": [[10, "furiosa.runtime.errors.QueueWaitTimeout"]], "queues": [[10, "term-Queues"]], "receiver (class in furiosa.runtime)": [[10, "furiosa.runtime.Receiver"]], "receiver (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Receiver"]], "recordformat (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.RecordFormat"]], "resource (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.Resource"]], "runner (class in furiosa.runtime)": [[10, "furiosa.runtime.Runner"]], "runner (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Runner"]], "runners": [[10, "term-Runners"]], "runtime (class in furiosa.runtime)": [[10, "furiosa.runtime.Runtime"]], "runtime (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Runtime"]], "session (class in furiosa.runtime.session)": [[10, "furiosa.runtime.session.Session"]], "sessionclosed": [[10, "furiosa.runtime.errors.SessionClosed"]], "sessionterminated": [[10, "furiosa.runtime.errors.SessionTerminated"]], "submitter (class in furiosa.runtime)": [[10, "furiosa.runtime.Submitter"]], "submitter (class in furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.Submitter"]], "tensor (class in furiosa.runtime)": [[10, "furiosa.runtime.Tensor"]], "tensor (class in furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.Tensor"]], "tensorarray (class in furiosa.runtime)": [[10, "furiosa.runtime.TensorArray"]], "tensorarray (class in furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.TensorArray"]], "tensordesc (class in furiosa.runtime)": [[10, "furiosa.runtime.TensorDesc"]], "tensornamenotfound": [[10, "furiosa.runtime.errors.TensorNameNotFound"]], "uint8 (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.UINT8"]], "unknown (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.UNKNOWN"]], "unsupporteddatatype": [[10, "furiosa.runtime.errors.UnsupportedDataType"]], "unsupportedfeature": [[10, "furiosa.runtime.errors.UnsupportedFeature"]], "unsupportedtensortype": [[10, "furiosa.runtime.errors.UnsupportedTensorType"]], "width (furiosa.runtime.axis property)": [[10, "furiosa.runtime.Axis.WIDTH"]], "allocate() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.allocate"]], "axis() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.axis"]], "close() (furiosa.runtime.receiver method)": [[10, "furiosa.runtime.Receiver.close"]], "close() (furiosa.runtime.runner method)": [[10, "furiosa.runtime.Runner.close"]], "close() (furiosa.runtime.runtime method)": [[10, "furiosa.runtime.Runtime.close"]], "close() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.close"]], "close() (furiosa.runtime.session.asyncsession method)": [[10, "furiosa.runtime.session.AsyncSession.close"]], "close() (furiosa.runtime.session.completionqueue method)": [[10, "furiosa.runtime.session.CompletionQueue.close"]], "close() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.close"]], "copy_from() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.copy_from"]], "create() (in module furiosa.runtime.session)": [[10, "furiosa.runtime.session.create"]], "create_async() (in module furiosa.runtime.session)": [[10, "furiosa.runtime.session.create_async"]], "create_queue() (furiosa.runtime.furiosa.runtime.runtime method)": [[10, "furiosa.runtime.furiosa.runtime.Runtime.create_queue"]], "create_queue() (furiosa.runtime.sync.furiosa.runtime.sync.runtime method)": [[10, "furiosa.runtime.sync.furiosa.runtime.sync.Runtime.create_queue"]], "create_queue() (in module furiosa.runtime)": [[10, "furiosa.runtime.create_queue"]], "create_queue() (in module furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.create_queue"]], "create_runner() (furiosa.runtime.furiosa.runtime.runtime method)": [[10, "furiosa.runtime.furiosa.runtime.Runtime.create_runner"]], "create_runner() (furiosa.runtime.sync.furiosa.runtime.sync.runtime method)": [[10, "furiosa.runtime.sync.furiosa.runtime.sync.Runtime.create_runner"]], "create_runner() (in module furiosa.runtime)": [[10, "furiosa.runtime.create_runner"]], "create_runner() (in module furiosa.runtime.sync)": [[10, "furiosa.runtime.sync.create_runner"]], "current_npu_device() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.current_npu_device"]], "dim() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.dim"]], "dtype (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.dtype"]], "environment variable": [[10, "envvar-FURIOSA_DEVICES"], [10, "envvar-FURIOSA_LOG_LEVEL"], [10, "envvar-NPU_DEVNAME"], [10, "index-0"], [10, "index-1"]], "export_chrome_trace() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.export_chrome_trace"]], "format (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.format"]], "full_version() (in module furiosa.runtime)": [[10, "furiosa.runtime.full_version"]], "furiosa.runtime": [[10, "module-furiosa.runtime"]], "furiosa.runtime.compiler": [[10, "module-furiosa.runtime.compiler"]], "furiosa.runtime.consts": [[10, "module-furiosa.runtime.consts"]], "furiosa.runtime.envs": [[10, "module-furiosa.runtime.envs"]], "furiosa.runtime.errors": [[10, "module-furiosa.runtime.errors"]], "furiosa.runtime.model": [[10, "module-furiosa.runtime.model"]], "furiosa.runtime.profiler": [[10, "module-furiosa.runtime.profiler"]], "furiosa.runtime.session": [[10, "module-furiosa.runtime.session"]], "furiosa.runtime.tensor": [[10, "module-furiosa.runtime.tensor"]], "generate_compiler_log_path() (in module furiosa.runtime.compiler)": [[10, "furiosa.runtime.compiler.generate_compiler_log_path"]], "get_cpu_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_cpu_pandas_dataframe"]], "get_npu_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_npu_pandas_dataframe"]], "get_pandas_dataframe() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_pandas_dataframe"]], "get_pandas_dataframe_with_filter() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.get_pandas_dataframe_with_filter"]], "input() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.input"]], "input_num (furiosa.runtime.model property)": [[10, "furiosa.runtime.Model.input_num"]], "inputs() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.inputs"]], "is_compile_log_enabled() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.is_compile_log_enabled"]], "is_empty() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.is_empty"]], "length (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.length"]], "log_dir() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.log_dir"]], "model (furiosa.runtime.receiver property)": [[10, "furiosa.runtime.Receiver.model"]], "model (furiosa.runtime.runner property)": [[10, "furiosa.runtime.Runner.model"]], "model (furiosa.runtime.submitter property)": [[10, "furiosa.runtime.Submitter.model"]], "model (furiosa.runtime.session.asyncsession property)": [[10, "furiosa.runtime.session.AsyncSession.model"]], "model (furiosa.runtime.session.completionqueue property)": [[10, "furiosa.runtime.session.CompletionQueue.model"]], "model (furiosa.runtime.session.session property)": [[10, "furiosa.runtime.session.Session.model"]], "name (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.name"]], "ndim (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.ndim"]], "numpy (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.numpy"]], "numpy() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.numpy"]], "numpy() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.numpy"]], "numpy_dtype (furiosa.runtime.datatype property)": [[10, "furiosa.runtime.DataType.numpy_dtype"]], "numpy_dtype (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.numpy_dtype"]], "numpy_dtype (furiosa.runtime.tensor.tensor property)": [[10, "furiosa.runtime.tensor.Tensor.numpy_dtype"]], "numpy_dtype() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.numpy_dtype"]], "output() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.output"]], "output_num (furiosa.runtime.model property)": [[10, "furiosa.runtime.Model.output_num"]], "outputs() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.outputs"]], "pause() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.pause"]], "print_external_operators() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_external_operators"]], "print_inferences() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_inferences"]], "print_npu_executions() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_npu_executions"]], "print_npu_operators() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_npu_operators"]], "print_summary() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.print_summary"]], "print_summary() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.print_summary"]], "profile (class in furiosa.runtime.profiler)": [[10, "furiosa.runtime.profiler.profile"]], "profiler_output() (in module furiosa.runtime.envs)": [[10, "furiosa.runtime.envs.profiler_output"]], "rand() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.rand"]], "record() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.record"]], "recv() (furiosa.runtime.receiver method)": [[10, "furiosa.runtime.Receiver.recv"]], "recv() (furiosa.runtime.session.completionqueue method)": [[10, "furiosa.runtime.session.CompletionQueue.recv"]], "recv() (furiosa.runtime.sync.receiver method)": [[10, "furiosa.runtime.sync.Receiver.recv"]], "resume() (furiosa.runtime.profiler.profile method)": [[10, "furiosa.runtime.profiler.profile.resume"]], "run() (furiosa.runtime.runner method)": [[10, "furiosa.runtime.Runner.run"]], "run() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.run"]], "run_with() (furiosa.runtime.session.session method)": [[10, "furiosa.runtime.session.Session.run_with"]], "shape (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.shape"]], "shape (furiosa.runtime.tensor.tensor property)": [[10, "furiosa.runtime.tensor.Tensor.shape"]], "size (furiosa.runtime.tensordesc property)": [[10, "furiosa.runtime.TensorDesc.size"]], "stride() (furiosa.runtime.tensordesc method)": [[10, "furiosa.runtime.TensorDesc.stride"]], "submit() (furiosa.runtime.submitter method)": [[10, "furiosa.runtime.Submitter.submit"]], "submit() (furiosa.runtime.session.asyncsession method)": [[10, "furiosa.runtime.session.AsyncSession.submit"]], "summary() (furiosa.runtime.model method)": [[10, "furiosa.runtime.Model.summary"]], "view() (furiosa.runtime.tensor.tensor method)": [[10, "furiosa.runtime.tensor.Tensor.view"]], "view() (furiosa.runtime.tensor.tensorarray method)": [[10, "furiosa.runtime.tensor.TensorArray.view"]], "zeros() (in module furiosa.runtime.tensor)": [[10, "furiosa.runtime.tensor.zeros"]], "modelnotfound": [[11, "furiosa.server.errors.ModelNotFound"]], "modelservererror": [[11, "furiosa.server.errors.ModelServerError"]], "furiosa.server.errors": [[11, "module-furiosa.server.errors"]], "furiosa.server.utils": [[19, "module-furiosa.server.utils"]]}})
\ No newline at end of file
diff --git a/v0.10.0/en/software/c-sdk.html b/v0.10.0/en/software/c-sdk.html
index d986676..ba8b309 100644
--- a/v0.10.0/en/software/c-sdk.html
+++ b/v0.10.0/en/software/c-sdk.html
@@ -67,7 +67,7 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/software/cli.html b/v0.10.0/en/software/cli.html
index bf1b286..a39f902 100644
--- a/v0.10.0/en/software/cli.html
+++ b/v0.10.0/en/software/cli.html
@@ -76,7 +76,7 @@
Compiler
Model Quantization
-Performance profiling
+Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
@@ -303,8 +303,9 @@ furiosactl 99th percentile ( us) : 307568
-If an output path is specified, furiosa-bench will save a json document as the following:
- $ furiosa-bench mnist-8.onnx --workload L -n 1000 -w 8 -t 2 -o mnist.json | cat mnist.json
+If an output path is specified, furiosa-bench will save a JSON document as the following:
+
SOURCE
is the file path of
-TFLite or Onnx .
+TFLite or ONNX .
You have to use quantized models through Model Quantization for NPU accleration.
You can omit the option -o OUTPUT , and you can also choose to designate the output file name.
When omitted, the default output file name is output.enf
. Here, enf stands for Executable NPU Format.
diff --git a/v0.10.0/en/software/installation.html b/v0.10.0/en/software/installation.html
index 3650109..28d2f23 100644
--- a/v0.10.0/en/software/installation.html
+++ b/v0.10.0/en/software/installation.html
@@ -74,7 +74,7 @@
Command Line Tools
Compiler
Model Quantization
-
Performance profiling
+
Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/software/intro.html b/v0.10.0/en/software/intro.html
index e7b444c..e855d8e 100644
--- a/v0.10.0/en/software/intro.html
+++ b/v0.10.0/en/software/intro.html
@@ -71,7 +71,7 @@
Command Line Tools
Compiler
Model Quantization
-
Performance profiling
+
Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/software/kubernetes_support.html b/v0.10.0/en/software/kubernetes_support.html
index 2986210..dffca85 100644
--- a/v0.10.0/en/software/kubernetes_support.html
+++ b/v0.10.0/en/software/kubernetes_support.html
@@ -62,7 +62,7 @@
Command Line Tools
Compiler
Model Quantization
-
Performance profiling
+
Performance Profiling
FuriosaAI Model Zoo
Model Server (Serving Framework)
Kubernetes Support
diff --git a/v0.10.0/en/software/profiler.html b/v0.10.0/en/software/profiler.html
index 96a4c11..c247a34 100644
--- a/v0.10.0/en/software/profiler.html
+++ b/v0.10.0/en/software/profiler.html
@@ -4,7 +4,7 @@
- Performance profiling — Furiosa SDK Documentation 0.10.0 documentation
+ Performance Profiling — Furiosa SDK Documentation 0.10.0 documentation
@@ -62,10 +62,10 @@
Command Line Tools
Compiler
Model Quantization
-Performance profiling