[pass] Implement checker pass and refactor shape inference

[justinchuby]

• Refactor shape_inference pass to extract logic for handling large models for onnx c-api.
• Implement an onnx checker pass leveraging the refactored logic.

[Copilot]

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

onnxscript/ir/passes/common/shape_inference.py:23

• [nitpick] Consider renaming _merge_func to a more descriptive name such as merge_inferred_model to better reflect its purpose.

def _merge_func(model: ir.Model, inferred_proto: onnx.ModelProto) -> tuple[ir.Model, bool]:

[codecov]

❌ 19 Tests Failed:

Tests completed	Failed	Passed	Skipped
14672	19	14653	2387

View the top 2 failed test(s) by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0399_test_gather_elements_1

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_gather_elements_1'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_gather_elements_1' (e=No module named 'tests.onnx_backend_test_code.test_gather_elements_1') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gather_elements_1.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gather_elements_1.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_gather_elements_1(data: FLOAT[3,3], indices: INT64[2,3]) -> (FLOAT[2,3]):
E       y = opset13.GatherElements(data, indices, axis=0)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0926_test_reduce_sum_square_keepdims_random

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.12.9\x64\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_reduce_sum_square_keepdims_random'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_reduce_sum_square_keepdims_random' (e=No module named 'tests.onnx_backend_test_code.test_reduce_sum_square_keepdims_random') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_sum_square_keepdims_random.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_sum_square_keepdims_random.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_reduce_sum_square_keepdims_random(data: FLOAT[3,2,2], axes: INT64[1]) -> (FLOAT[3,1,2]):
E       reduced = opset18.ReduceSumSquare(data, axes, keepdims=1)
E       return reduced

View the full list of 1 ❄️ flaky tests

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0755_test_optional_has_element_empty_no_input_optional_input

Flake rate in main: 10.00% (Passed 27 times, Failed 3 times)

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_optional_has_element_empty_no_input_optional_input'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_optional_has_element_empty_no_input_optional_input' (e=No module named 'tests.onnx_backend_test_code.test_optional_has_element_empty_no_input_optional_input') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_optional_has_element_empty_no_input_optional_input.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_optional_has_element_empty_no_input_optional_input.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import BOOL
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_optional_has_element_empty_no_input_optional_input() -> (BOOL):
E       output = opset18.OptionalHasElement()
E       return output

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[gramalingam]

nit: Looks like the check in line 73 is not really being used effectively? Wouldn't it be better to ensure that we create only a list of the temporarily removed initializers in the loop below?

[justinchuby]

I updated the function to use a dictionary and removed the check. Does it look like what you had in mind?

[Copilot]

Pull Request Overview

This PR implements a new CheckerPass to perform ONNX model checks and refactors the shape inference pass to better handle large models by offloading initializer management to a shared utility. Key changes include:

• Adding CheckerPass and its corresponding tests in onnx_checker.py and onnx_checker_test.py.
• Refactoring shape inference in shape_inference.py to use a helper (partial_infer_shapes) and common initializer management via _c_api_utils.
• Updating tests in shape_inference_test.py to validate the in-place nature of the pass and adjust initializer handling.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
onnxscript/ir/passes/common/shape_inference_test.py	Added test for in-place property and updated initializer tests reflecting the refactor.
onnxscript/ir/passes/common/shape_inference.py	Refactored the inference pass and merged inferred shapes using a dedicated helper.
onnxscript/ir/passes/common/onnx_checker_test.py	Introduced tests for the new CheckerPass ensuring proper model checking without side effects.
onnxscript/ir/passes/common/onnx_checker.py	Added a CheckerPass to run ONNX model checks using the C API utilities.
onnxscript/ir/passes/common/_c_api_utils.py	Extracted common functionality for managing large initializer values during API calls.

Comments suppressed due to low confidence (1)

onnxscript/ir/passes/common/shape_inference_test.py:129

• Consider adding or reintroducing some assertions to validate that initializers and input lists are correctly restored after performing shape inference. This will help ensure that the refactored initializer handling via _c_api_utils does not inadvertently modify the model.

Removed assertions checking that the original model is not modified after shape inference