-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change :float
to {:f, 64}
and add {:f, 32}
dtype
#739
Conversation
I feel that we may want to have a "shortcut" version for declaring the dtypes, like in Nx. Also, the inspection of the series and dataframes could be different, and follow Nx as well - just |
I think we should definitely still support |
Suggestion: rename |
Yep I agree -- let's keep the 'simple' names like |
The idea is to support more dtypes in the way they are represented in Arrow/Polars. In the near future, we won't need to "normalise" numeric columns anymore, and by consequence, Explorer is going to use less memory if needed. This is related to #471
4014fdb
to
2e1ed88
Compare
This is going to make easier to adopt "aliases" for dtypes.
Use the `"f64"` string instead of `float[64]`.
@josevalim @cigrainger I made changes applying your suggestions. Please take a look. The only thing I didn't change was to add the aliases |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dropped one Q but otherwise this looks fantastic. I'm excited to support more explicit datatypes, esp w/ zero-copy conversion to tensors.
README.md
Outdated
@@ -13,7 +13,7 @@ data exploration to Elixir. | |||
|
|||
Explorer high-level features are: | |||
|
|||
- Simply typed series: `:binary`, `:boolean`, `:category`, `:date`, `:datetime`, `:duration`, `:float`, `:integer`, `:string`, and `:time`. | |||
- Simply typed series: `:binary`, `:boolean`, `:category`, `:date`, `:datetime`, `:duration`, `{:f, 32}`, `{:f, 64}`, `:integer`, `:string`, and `:time`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we shouldn't do something like:
- Simply typed series: `:binary`, `:boolean`, `:category`, `:date`, `:datetime`, `:duration`, `{:f, 32}`, `{:f, 64}`, `:integer`, `:string`, and `:time`. | |
- Simply typed series: `:binary`, `:boolean`, `:category`, `:date`, `:datetime`, `:duration`, `:float` (`{:f, 32}` or `{:f, 64}`), `:integer`, `:string`, and `:time`. |
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed! I added more mentions of it in other places as well: 3605cfc
It adds `:f32` and `:f64` as shortcuts/aliases for the float dtypes. This is related to #739
* Add shortcut atoms for float dtypes It adds `:f32` and `:f64` as shortcuts/aliases for the float dtypes. This is related to #739 * Update series.ex --------- Co-authored-by: José Valim <jose.valim@dashbit.co>
The idea is to support more dtypes in the way they are represented in Arrow/Polars.
In the near future, we won't need to "normalise" numeric columns anymore, and by consequence, Explorer is going to use less memory if needed.
This is related to #471