Canonicalize Subtensor slices

[Dhruvanshu-Joshi]

Description

Herein I aim to convert all:

• X[0:-1:1] -> X[:-1]
• X[0:-1] -> X[:-1]
• X[:-1] -> X[:-1]

Related Issue

• Closes Canonicalize Subtensor slices #58
• Related to #

Checklist

• Checked that the pre-commit linting/style checks pass
• Included tests that prove the fix is effective or that the new feature works
• Added necessary documentation (docstrings and/or example notebooks)
• If you are a pro: each commit corresponds to a relevant logical change

Type of change

• New feature / enhancement
• Bug fix
• Documentation
• Maintenance
• Other (please specify):

[Dhruvanshu-Joshi]

For all the tests that are failing ( eg when I do pytest tests/sparse/sandbox/test_sp.py -k TestSP -k test_maxpool ), I am getting the error:

TypeError: Cannot convert Type Vector(float64, shape=(2,)) (of Variable Cast{float64}.0) into Type Scalar(float64, shape=()). You can try to manually convert Cast{float64}.0 into a Scalar(float64, shape=()).

However this occurs because of @register_canonicalize. It vanishes if I comment it out.

Similarly for pytest tests/compile/function/test_pfunc.py -k TestAliasingRules -k test_input_aliasing_aff, the error vanishes when I comment out @register_specialize.

What do these registers do and why do the changes that I have made force them to fail?

[Dhruvanshu-Joshi]

Assume x = tensor(shape=(3, 5, None, 9))
The changes made above works when no other rewrite is to be applied (eg: x[0:None, 0:5, 0:7, 0:9:1]).
However when we have other rewrites in picture (for eg:x[0:-1, 0:5, 0:7, 0:-1:-1]), we face the error :

TypeError: The type of the replacement (Tensor4(float64, shape=(3, 5, ?, 9))) must be compatible with the type of the original Variable (Tensor4(float64, shape=(2, 5, ?, 0)))

This happens because the shape of input x was modified by previous rewrites to be (2, 5, ?, 0) (owing to stop at index 0 being -1 and step at index 3 being -1 in slice: x[0:-1, 0:5, 0:7, 0:-1:-1].

How can I make this rewrite: local_replace_slice be aware of the previous rewrites?

[ricardoV94]

The error message tells you there's likely a bug in your new rewrite. The returned variable should have the same type as the original one. You can print node.outputs[0].type to see the original type and print the type of your proposed replacement.

Perhaps you lost track of some non slice indexes during the rewrite?

[Dhruvanshu-Joshi]

The error message tells you there's likely a bug in your new rewrite. The returned variable should have the same type as the original one. You can print node.outputs[0].type to see the original type and print the type of your proposed replacement.

Perhaps you lost track of some non slice indexes during the rewrite?

node.outputs[0].type and x[tuple(new_idxs)].type comes out to be the same.

Tensor4(float64, shape=(2, 5, ?, 0))
Tensor4(float64, shape=(2, 5, ?, 0))

[ricardoV94]

How can I make this rewrite: local_replace_slice be aware of the previous rewrites?

Your /other rewrite shouldn't need to be aware of each other, because each should leave the graph consistent with how it was before.

[ricardoV94]

Do you have a test that is failing that we can look at?

[Dhruvanshu-Joshi]

Do you have a test that is failing that we can look at?

def test_slice_canonicalize():
    x = tensor(shape=(3, 5, None, 9))
    y = x[0:-1, 0:5, 0:7, 0:-1:-1]
    f = pytensor.function([x], y)

    test_y = f(np.random.normal(size=(3, 5, 6, 9)))

    y1 = x[None:None:None, None:None:None, None:7:None, None:None:-1]

    if lstop == -1 and lstep == -1:
        y1 = x[None:None:None, None:None:None, None:7:None, None:0:-1]

    f1 = pytensor.function([x], y1)

    expected_y = f1(np.random.normal(size=(3, 5, 6, 9)))

    # TODO: test using equal_computations
    assert test_y.all() == expected_y.all()

This is a dummy test. The assert will be replaced with equal_computations.
This fails and also the tests in the CI fails due to this. One such being: pytest tests/compile/function/test_pfunc.py -k TestAliasingRules -k test_input_aliasing_aff

[ricardoV94]

The first failing test in the CI, prints:

<<!! BUG IN FGRAPH.REPLACE OR A LISTENER !!>> <class 'TypeError'> Cannot convert Type Vector(int64, shape=(1,)) (of Variable Subtensor{:stop}.0) into Type Scalar(int64, shape=()). You can try to manually convert Subtensor{:stop}.0 into a Scalar(int64, shape=()). local_replace_slice

The rewrite is trying to replace a length 1 vector by a scalar. Stepping into the debugger with a breakpoint on the new rewrite may help you figure out where the bug is

[Dhruvanshu-Joshi]

So for the recent changes, tests like tests/tensor/rewriting/test_subtensor.py::TestLocalSubtensorLift::test_basic_3 fails locally.
They do so because of assert check_stack_trace(f, ops_to_check=Subtensor) where:

x = matrix("x")
        y = scalar("y")
        z = matrix("z")
        f = function([x, y, z], exp(x + y + z)[0:2], mode=mode_opt)

I tried printing the output.tag.trace in check_stack_trace and here's what I get:
Without this rewrite:

[[('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_callers.py', 102, '_multicall', 'res = hook_impl.function(*args)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\runner.py', 172, 'pytest_runtest_call', 'item.runtest()'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\python.py', 1772, 'runtest', 'self.ihook.pytest_pyfunc_call(pyfuncitem=self)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_hooks.py', 501, '__call__', 'return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_manager.py', 119, '_hookexec', 'return self._inner_hookexec(hook_name, methods, kwargs, firstresult)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_callers.py', 102, '_multicall', 'res = hook_impl.function(*args)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\python.py', 195, 'pytest_pyfunc_call', 'result = testfunction(**testargs)'), ('C:\\Users\\jmahe\\pymc\\new1\\pytensor\\tests\\tensor\\rewriting\\test_subtensor.py', 836, 'test_basic_3', 'f = function([x, y, z], exp(x + y + z)[0:2], mode=mode_opt)')]]
[[('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_callers.py', 102, '_multicall', 'res = hook_impl.function(*args)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\runner.py', 172, 'pytest_runtest_call', 'item.runtest()'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\python.py', 1772, 'runtest', 'self.ihook.pytest_pyfunc_call(pyfuncitem=self)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_hooks.py', 501, '__call__', 'return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_manager.py', 119, '_hookexec', 'return self._inner_hookexec(hook_name, methods, kwargs, firstresult)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\pluggy\\_callers.py', 102, '_multicall', 'res = hook_impl.function(*args)'), ('C:\\Users\\jmahe\\miniconda3\\envs\\pytensor-dev\\Lib\\site-packages\\_pytest\\python.py', 195, 'pytest_pyfunc_call', 'result = testfunction(**testargs)'), ('C:\\Users\\jmahe\\pymc\\new1\\pytensor\\tests\\tensor\\rewriting\\test_subtensor.py', 836, 'test_basic_3', 'f = function([x, y, z], exp(x + y + z)[0:2], mode=mode_opt)')]]

With this rewrite:

[]

So how can I preserve this trace in my rewrite?

[ricardoV94]

Check the utility copy_stack_trace

[Dhruvanshu-Joshi]

The error message tells you there's likely a bug in your new rewrite. The returned variable should have the same type as the original one. You can print node.outputs[0].type to see the original type and print the type of your proposed replacement.

Perhaps you lost track of some non slice indexes during the rewrite?

Is it always necessary to have node.outputs[0].type == x[tuple(new_idxs)].type?
This is not the case when setting config.mode to FastCompile or when jax runs this rewrite. I have added an assert and corresponding assertion error whenever it is not true because of which the remaining tests fail in the CI.
The tests pass even with config.mode set to FastCompile when the assertion is absent meaning that node.outputs[0].type != x[tuple(new_idxs)].type

Should I proceed to drop this assert in the rewrite as it seems to be flexible?

https://github.com/Dhruvanshu-Joshi/pytensor/blob/863105a689f96b262e520010aa34e9dc1a0ff723/pytensor/tensor/rewriting/subtensor.py#L430-L435

[ricardoV94]

Is it always necessary to have node.outputs[0].type == x[tuple(new_idxs)].type?

The returned type must be compatible with the original type that it is substituting. That means the same number of dimensions and compatible shape info. You can replace something of type.shape=(5, None) by something with type.shape (None, 3) and vice-versa, but not something of type.shape=(5, None) by something with type.shape=(3, None) or (5,) (less dims)

[ricardoV94]

I've noticed Scan rewrites being even more strict about this, would be good to check which line is being too strict in the failing tests.

[Dhruvanshu-Joshi]

Is it always necessary to have node.outputs[0].type == x[tuple(new_idxs)].type?

The returned type must be compatible with the original type that it is substituting. That means the same number of dimensions and compatible shape info. You can replace something of type.shape=(5, None) by something with type.shape (None, 3) and vice-versa, but not something of type.shape=(5, None) by something with type.shape=(3, None) or (5,) (less dims)

So according to that, (3, None) and (5, None) will also not be compatible right? This can be done simple by checking if it is preserved or is now none, right?

[ricardoV94]

Sorry, I missed your not in the comment. Yes you're correct. In practice you shouldn'tt have to check anything, because the PyTensor graph machinery will raise if the replacement is incompatible.

[Dhruvanshu-Joshi]

Sorry, I missed your not in the comment. Yes you're correct. In practice you shouldn'tt have to check anything, because the PyTensor graph machinery will raise if the replacement is incompatible.

That's great! Then I can drop the assert.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.75%. Comparing base (8a6e407) to head (0f6a7a0).
Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #761   +/-   ##
=======================================
  Coverage   81.74%   81.75%           
=======================================
  Files         183      183           
  Lines       47733    47749   +16     
  Branches    11616    11620    +4     
=======================================
+ Hits        39020    39037   +17     
+ Misses       6518     6517    -1     
  Partials     2195     2195           

Files with missing lines	Coverage Δ
pytensor/tensor/rewriting/subtensor.py	90.12% <100.00%> (+0.06%)	⬆️

... and 1 file with indirect coverage changes

[ricardoV94]

Looking great, some more suggestions for optimization and combining with the existing rewerite

[ricardoV94]

Why not the output directly, there's a deepcopy?

[HangenYuu]

@ricardoV94 @Dhruvanshu-Joshi Is there anything left preventing this PR from being merge?

[ricardoV94]

I had a small question above, why we are checking the input of the output instead of the output directly

[Dhruvanshu-Joshi]

I had a small question above, why we are checking the input of the output instead of the output directly

Hi @ricardoV94 @HangenYuu , you mean test_y = f.maker.fgraph.outputs[0].owner.inputs[0] right?
This is because f.maker.fgraph.outputs[0] is a Deepcopy. Its input is what we want.

[ricardoV94]

I had a small question above, why we are checking the input of the output instead of the output directly

Hi @ricardoV94 @HangenYuu , you mean test_y = f.maker.fgraph.outputs[0].owner.inputs[0] right?
This is because f.maker.fgraph.outputs[0] is a Deepcopy. Its input is what we want.

Then it's good to assert that as well so everything is explicit

[Dhruvanshu-Joshi]

I had a small question above, why we are checking the input of the output instead of the output directly

Hi @ricardoV94 @HangenYuu , you mean test_y = f.maker.fgraph.outputs[0].owner.inputs[0] right?
This is because f.maker.fgraph.outputs[0] is a Deepcopy. Its input is what we want.

Then it's good to assert that as well so everything is explicit

Can we eliminate the deepcopy using assert? Sounds great, how can we do so?

[ricardoV94]

Can we eliminate the deepcopy using assert? Sounds great, how can we do so?

I didn't mean to eliminate, I meant asserting that the Op of that fist node is a DeepCopy. That will make the test less mysterious. Alternative you can say that you expect a deepcopy around the slice in assert_equal_computations

[Dhruvanshu-Joshi]

Hi @ricardoV94
I added the assert within the tests and pushed the changes as usual. However, the tests fail with, "Blas flags are empty" error.

Here's my log of running python -c "import pytensor; print(pytensor.config)":

$ python -c "import pytensor; print(pytensor.config)
> "
floatX ({'float16', 'float32', 'float64'})
    Doc:  Default floating-point precision for python casts.

Note: float16 support is experimental, use at your own risk.
    Value:  float64

warn_float64 ({'warn', 'pdb', 'raise', 'ignore'})
    Doc:  Do an action when a tensor variable with float64 dtype is created.
    Value:  ignore

pickle_test_value (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57435B550>>)
    Doc:  Dump test values while pickling model. If True, test values will be dumped with model.
    Value:  True

cast_policy ({'custom', 'numpy+floatX'})
    Doc:  Rules for implicit type casting
    Value:  custom

device (cpu)
    Doc:  Default device for computations. only cpu is supported for now
    Value:  cpu

conv__assert_shape (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57EF23F10>>)
    Doc:  If True, AbstractConv* ops will verify that user-provided shapes match the runtime shapes (debugging option, may slow down compilation)
    Value:  False

print_global_stats (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E5743D5AD0>>)
    Doc:  Print some global statistics (time spent) at the end
    Value:  False

unpickle_function (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57E893E90>>)
    Doc:  Replace unpickled PyTensor functions with None. This is useful to unpickle old graphs that pickled them when it shouldn't
    Value:  True

<pytensor.configparser.ConfigParam object at 0x000001E57EED0190>
    Doc:  Default compilation mode
    Value:  Mode

cxx (<class 'str'>)
    Doc:  The C++ compiler to use. Currently only g++ is supported, but supporting additional compilers should not be too difficult. If it is empty, no C++ code is compiled.
    Value:  "C:\msys64\ucrt64\bin\g++.exe"

linker ({'cvm_nogc', 'cvm', 'c|py_nogc', 'c|py', 'py', 'c', 'vm_nogc', 'vm'})
    Doc:  Default linker used if the pytensor flags mode is Mode
    Value:  cvm

allow_gc (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF80450>>)
    Doc:  Do we default to delete intermediate results during PyTensor function calls? Doing so lowers the memory requirement, but asks that we reallocate memory at the next function call. This is implemented for the default linker, but may not work for all linkers.
    Value:  True

optimizer ({'o1', 'fast_compile', 'unsafe', 'merge', 'o2', 'o4', 'fast_run', 'None', 'o3'})
    Doc:  Default optimizer. If not None, will use this optimizer with the Mode
    Value:  o4

optimizer_verbose (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF81E50>>)
    Doc:  If True, we print all optimization being applied
    Value:  False

on_opt_error ({'warn', 'pdb', 'ignore', 'raise'})
    Doc:  What to do when an optimization crashes: warn and skip it, raise the exception, or fall into the pdb debugger.
    Value:  warn

nocleanup (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF81A50>>)
    Doc:  Suppress the deletion of code files that did not compile cleanly
    Value:  False

on_unused_input ({'warn', 'ignore', 'raise'})
    Doc:  What to do if a variable in the 'inputs' list of  pytensor.function() is not used in the graph.
    Value:  raise

gcc__cxxflags (<class 'str'>)
    Doc:  Extra compiler flags for gcc
    Value:

cmodule__warn_no_version (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF81690>>)
    Doc:  If True, will print a warning when compiling one or more Op with C code that can't be cached because there is no c_code_cache_version() function associated to at least one of those Ops.
    Value:  False

cmodule__remove_gxx_opt (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF81750>>)
    Doc:  If True, will remove the -O* parameter passed to g++.This is useful to debug in gdb modules compiled by PyTensor.The parameter -g is passed by default to g++
    Value:  False

cmodule__compilation_warning (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57EF23F50>>)
    Doc:  If True, will print compilation warnings.
    Value:  False

cmodule__preload_cache (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF80590>>)
    Doc:  If set to True, will preload the C module cache at import time
    Value:  False

cmodule__age_thresh_use (<class 'int'>)
    Doc:  In seconds. The time after which PyTensor won't reuse a compile c module.
    Value:  2073600

cmodule__debug (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57F9C2950>>)
    Doc:  If True, define a DEBUG macro (if not exists) for any compiled C code.
    Value:  False

compile__wait (<class 'int'>)
    Doc:  Time to wait before retrying to acquire the compile lock.
    Value:  5

compile__timeout (<class 'int'>)
    Doc:  In seconds, time that a process will wait before deciding to
    override an existing lock. An override only happens when the existing
    lock is held by the same owner *and* has not been 'refreshed' by this
    owner for more than this period. Refreshes are done every half timeout
    period for running processes.
    Value:  120

tensor__cmp_sloppy (<class 'int'>)
    Doc:  Relax pytensor.tensor.math._allclose (0) not at all, (1) a bit, (2) more
    Value:  0

lib__amdlibm (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57E90CB10>>)
    Doc:  Use amd's amdlibm numerical library
    Value:  False

tensor__insert_inplace_optimizer_validate_nb (<class 'int'>)
    Doc:  -1: auto, if graph have less then 500 nodes 1, else 10
    Value:  -1

traceback__limit (<class 'int'>)
    Doc:  The number of stack to trace. -1 mean all.
    Value:  8

traceback__compile_limit (<class 'int'>)
    Doc:  The number of stack to trace to keep during compilation. -1 mean all. If greater then 0, will also make us save PyTensor internal stack trace.
    Value:  0

warn__ignore_bug_before ({'0.9', '0.7', '1.0', '1.0.3', '0.3', '1.0.1', 'all', '0.6', '0.4.1', 'None', '0.5', '0.8.2', '0.4', '1.0.2', '1.0.4', '0.8.1', '0.10', '1.0.5', '0.8'})
    Doc:  If 'None', we warn about all PyTensor bugs found by default. If 'all', we don't warn about PyTensor bugs found by default. If a version, we print only the warnings relative to PyTensor bugs found after that version. Warning for specific bugs can be configured with specific [warn] flags.
    Value:  0.9

exception_verbosity ({'high', 'low'})
    Doc:  If 'low', the text of exceptions will generally refer to apply nodes with short names such as Elemwise{add_no_inplace}. If 'high', some exceptions will also refer to apply nodes with long descriptions  like:
        A. Elemwise{add_no_inplace}
                B. log_likelihood_v_given_h
                C. log_likelihood_h
    Value:  low

print_test_value (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF654D0>>)
    Doc:  If 'True', the __eval__ of an PyTensor variable will return its test_value when this is available. This has the practical consequence that, e.g., in debugging `my_var` will print the same as `my_var.tag.test_value` when a test value is defined.
    Value:  False

compute_test_value ({'off', 'warn', 'raise', 'pdb', 'ignore'})
    Doc:  If 'True', PyTensor will run each op at graph build time, using Constants, SharedVariables and the tag 'test_value' as inputs to the function. This helps the user track down problems in the graph before it gets optimized.
    Value:  off

compute_test_value_opt ({'off', 'warn', 'raise', 'pdb', 'ignore'})
    Doc:  For debugging PyTensor optimization only. Same as compute_test_value, but is used during PyTensor optimization
    Value:  off

check_input (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF655D0>>)
    Doc:  Specify if types should check their input in their C code. It can be used to speed up compilation, reduce overhead (particularly for scalars) and reduce the number of generated C files.
    Value:  True

NanGuardMode__nan_is_error (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65690>>)
    Doc:  Default value for nan_is_error
    Value:  True

NanGuardMode__inf_is_error (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF657D0>>)
    Doc:  Default value for inf_is_error
    Value:  True

NanGuardMode__big_is_error (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65850>>)
    Doc:  Default value for big_is_error
    Value:  True

NanGuardMode__action ({'warn', 'pdb', 'raise'})
    Doc:  What NanGuardMode does when it finds a problem
    Value:  raise

DebugMode__patience (<class 'int'>)
    Doc:  Optimize graph this many times to detect inconsistency
    Value:  10

DebugMode__check_c (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65A50>>)
    Doc:  Run C implementations where possible
    Value:  True

DebugMode__check_py (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65950>>)
    Doc:  Run Python implementations where possible
    Value:  True

DebugMode__check_finite (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65B50>>)
    Doc:  True -> complain about NaN/Inf results
    Value:  True

DebugMode__check_strides (<class 'int'>)
    Doc:  Check that Python- and C-produced ndarrays have same strides. On difference: (0) - ignore, (1) warn, or (2) raise error
    Value:  0

DebugMode__warn_input_not_reused (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65C10>>)
    Doc:  Generate a warning when destroy_map or view_map says that an op works inplace, but the op did not reuse the input for its output.
    Value:  True

DebugMode__check_preallocated_output (<class 'str'>)
    Doc:  Test thunks with pre-allocated memory as output storage. This is a list of strings separated by ":". Valid values are: "initial" (initial storage in storage map, happens with Scan),"previous" (previously-returned memory), "c_contiguous", "f_contiguous", "strided" (positive and negative strides), "wrong_size" (larger and smaller dimensions), and "ALL" (all of the above).
    Value:

DebugMode__check_preallocated_output_ndim (<class 'int'>)
    Doc:  When testing with "strided" preallocated output memory, test all combinations of strides over that number of (inner-most) dimensions. You may want to reduce that number to reduce memory or time usage, but it is advised to keep a minimum of 2.
    Value:  4

profiling__time_thunks (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65D10>>)
    Doc:  Time individual thunks when profiling
    Value:  True

profiling__n_apply (<class 'int'>)
    Doc:  Number of Apply instances to print by default
    Value:  20

profiling__n_ops (<class 'int'>)
    Doc:  Number of Ops to print by default
    Value:  20

profiling__output_line_width (<class 'int'>)
    Doc:  Max line width for the profiling output
    Value:  512

profiling__min_memory_size (<class 'int'>)
    Doc:  For the memory profile, do not print Apply nodes if the size
                 of their outputs (in bytes) is lower than this threshold
    Value:  1024

profiling__min_peak_memory (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65E90>>)
    Doc:  The min peak memory usage of the order
    Value:  False

profiling__destination (<class 'str'>)
    Doc:  File destination of the profiling output
    Value:  stderr

profiling__debugprint (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E574335ED0>>)
    Doc:  Do a debugprint of the profiled functions
    Value:  False

profiling__ignore_first_call (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF65DD0>>)
    Doc:  Do we ignore the first call of an PyTensor function.
    Value:  False

on_shape_error ({'warn', 'raise'})
    Doc:  warn: print a warning and use the default value. raise: raise an error
    Value:  warn

openmp (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E573B35350>>)
    Doc:  Allow (or not) parallel computation on the CPU with OpenMP. This is the default value used when creating an Op that supports OpenMP parallelization. It is preferable to define it via the PyTensor configuration file ~/.pytensorrc or with the environment variable PYTENSOR_FLAGS. Parallelization is only done for some operations that implement it, and even for operations that implement parallelism, each operation is free to respect this flag or not. You can control the number of threads used with the environment variable OMP_NUM_THREADS. If it is set to 1, we disable openmp in PyTensor by default.
    Value:  False

openmp_elemwise_minsize (<class 'int'>)
    Doc:  If OpenMP is enabled, this is the minimum size of vectors for which the openmp parallelization is enabled in element wise ops.
    Value:  200000

optimizer_excluding (<class 'str'>)
    Doc:  When using the default mode, we will remove optimizer with these tags. Separate tags with ':'.
    Value:

optimizer_including (<class 'str'>)
    Doc:  When using the default mode, we will add optimizer with these tags. Separate tags with ':'.
    Value:

optimizer_requiring (<class 'str'>)
    Doc:  When using the default mode, we will require optimizer with these tags. Separate tags with ':'.
    Value:

optdb__position_cutoff (<class 'float'>)
    Doc:  Where to stop earlier during optimization. It represent the position of the optimizer where to stop.
    Value:  inf

optdb__max_use_ratio (<class 'float'>)
    Doc:  A ratio that prevent infinite loop in EquilibriumGraphRewriter.
    Value:  8.0

cycle_detection ({'fast', 'regular'})
    Doc:  If cycle_detection is set to regular, most inplaces are allowed,but it is slower. If cycle_detection is set to faster, less inplacesare allowed, but it makes the compilation faster.The interaction of which one give the lower peak memory usage iscomplicated and not predictable, so if you are close to the peakmemory usage, triyng both could give you a small gain.
    Value:  regular

check_stack_trace ({'warn', 'log', 'off', 'raise'})
    Doc:  A flag for checking the stack trace during the optimization process. default (off): does not check the stack trace of any optimization log: inserts a dummy stack trace that identifies the optimizationthat inserted the variable that had an empty stack trace.warn: prints a warning if a stack trace is missing and also a dummystack trace is inserted that indicates which optimization insertedthe variable that had an empty stack trace.raise: raises an exception if a stack trace is missing
    Value:  off

metaopt__verbose (<class 'int'>)
    Doc:  0 for silent, 1 for only warnings, 2 for full output withtimings and selected implementation
    Value:  0

unittests__rseed (<class 'str'>)
    Doc:  Seed to use for randomized unit tests. Special value 'random' means using a seed of None.
    Value:  666

warn__round (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF66190>>)
    Doc:  Warn when using `tensor.round` with the default mode. Round changed its default from `half_away_from_zero` to `half_to_even` to have the same default as NumPy.
    Value:  False

profile (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E574335F50>>)
    Doc:  If VM should collect profile information
    Value:  False

profile_optimizer (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57EF4D510>>)
    Doc:  If VM should collect optimizer profile information
    Value:  False

profile_memory (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF66250>>)
    Doc:  If VM should collect memory profile information and print it
    Value:  False

<pytensor.configparser.ConfigParam object at 0x000001E57EF23E10>
    Doc:  Useful only for the VM Linkers. When lazy is None, auto detect if lazy evaluation is needed and use the appropriate version. If the C loop isn't being used and lazy is True, use the Stack VM; otherwise, use the Loop VM.
    Value:  None

numba__vectorize_target ({'parallel', 'cpu', 'cuda'})
    Doc:  Default target for numba.vectorize.
    Value:  cpu

numba__fastmath (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57FF66350>>)
    Doc:  If True, use Numba's fastmath mode.
    Value:  True

numba__cache (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E57EA08350>>)
    Doc:  If True, use Numba's file based caching.
    Value:  True

compiledir_format (<class 'str'>)
    Doc:  Format string for platform-dependent compiled module subdirectory
(relative to base_compiledir). Available keys: device, gxx_version,
hostname, numpy_version, platform, processor, pytensor_version,
python_bitwidth, python_int_bitwidth, python_version, short_platform.
Defaults to compiledir_%(short_platform)s-%(processor)s-
%(python_version)s-%(python_bitwidth)s.
    Value:  compiledir_%(short_platform)s-%(processor)s-%(python_version)s-%(python_bitwidth)s

<pytensor.configparser.ConfigParam object at 0x000001E57E892B10>
    Doc:  platform-independent root directory for compiled modules
    Value:  C:\Users\jmahe\AppData\Local\PyTensor

<pytensor.configparser.ConfigParam object at 0x000001E57F9C0CD0>
    Doc:  platform-dependent cache directory for compiled modules
    Value:  C:\Users\jmahe\AppData\Local\PyTensor\compiledir_Windows-10-10.0.22631-SP0-Intel64_Family_6_Model_191_Stepping_2_GenuineIntel-3.11.8-64

blas__ldflags (<class 'str'>)
    Doc:  lib[s] to include for [Fortran] level-3 blas implementation
    Value:  -L"C:\Users\jmahe\miniconda3\envs\pytensor-dev\Library\bin" -L"C:\Users\jmahe\miniconda3\envs\pytensor-dev" -llapack -lblas -lcblas -lm -Wl,-rpath,C:\Users\jmahe\miniconda3\envs\pytensor-dev\libs

blas__check_openmp (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E52663E850>>)
    Doc:  Check for openmp library conflict.
WARNING: Setting this to False leaves you open to wrong results in blas-related operations.
    Value:  True

scan__allow_gc (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E556AECBD0>>)
    Doc:  Allow/disallow gc inside of Scan (default: False)
    Value:  False

scan__allow_output_prealloc (<bound method BoolParam._apply of <pytensor.configparser.BoolParam object at 0x000001E556B992D0>>)
    Doc:  Allow/disallow memory preallocation for outputs inside of scan (default: True)
    Value:  True

What should I do to make the tests pass?

[Dhruvanshu-Joshi]

Hi @ricardoV94
Is here something left here?

[ricardoV94]

That was all thanks for the reminder @Dhruvanshu-Joshi :)