Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update debug_to_file API to remove type_code #8183

Merged
merged 32 commits into from
Apr 29, 2024
Merged

Conversation

steven-johnson
Copy link
Contributor

No description provided.

The .npy format is NumPy's native format for storing multidimensional arrays (aka tensors/buffers). Being able to load/save in this format makes it (potentially) a lot easier to interchange data with the Python ecosystem, as well as providing a file format that support floating-point data more robustly than any of the others that we current support.

This adds load/save support for a useful subset:
- We support the int/uint/float types common in Halide (except for f16/bf16 for now)
- We don't support reading or writing files that are in `fortran_order`
- We don't support any object/struct/etc files, only numeric primitives
- We only support loading files that are in the host's endianness (typically little-endian)

Note that at present this doesn't support f16 / bf16 formats, but that could likely be added with minimal difficulty.

The tricky bit of this is that the reading code has to parse a (limited) Python dict in text form. Please review that part carefully.

TODO: we could probably add this as an option for `debug_to_file()` without too much pain in a followup PR.
Built on top of #8175, this adds .npy as an option. This is actually pretty great because it's easy to do something like

```
ss = numpy.load("my_file.npy")
print(ss)
```

in Python and get nicely-formatted output, which can sometimes be a lot easier for debugging that inserting lots of print() statements (see #8176)

Did a drive-by change to the correctness test to use this format instead of .mat.
@steven-johnson steven-johnson requested a review from abadams April 8, 2024 22:54
Base automatically changed from srj/npy-debug-to-file to main April 11, 2024 18:04
@steven-johnson
Copy link
Contributor Author

PTAL

@steven-johnson
Copy link
Contributor Author

Finally clean, PTAL

@steven-johnson
Copy link
Contributor Author

Monday Morning Review Ping

@steven-johnson steven-johnson merged commit d55d82b into main Apr 29, 2024
19 checks passed
@steven-johnson steven-johnson deleted the srj/debug-to-file-api branch April 29, 2024 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants