You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Though our current "Deserialise" API is very helpful to work with parquet files, It still lacks APIs that gives ability to read parquet file as RowGroups. My current project team is in need of such API as we work with huge data like 10million record files, deserializing such a huge file and load entire business objects collection in memory is not working for us due to memory constraints.
So, It would be very helpful to dev teams if we can have these two APIs.
int GetRowGroupCount();
IEnumerable< T > ReadRowGroup(int i);
Actual behavior
No such API exists
Steps to reproduce the behavior
Deserialize a parquet file with 10 million records.
Code snippet reproducing the behavior
The text was updated successfully, but these errors were encountered:
srigumm
changed the title
Read and Deserializing parquet file as RowGroups
Read and deserialize parquet file as RowGroups
Jan 19, 2020
Version: Parquet.Net
Runtime Version: .Net Core
OS: Windows/Linux/MacOSX
Expected behavior
Though our current "Deserialise" API is very helpful to work with parquet files, It still lacks APIs that gives ability to read parquet file as RowGroups. My current project team is in need of such API as we work with huge data like 10million record files, deserializing such a huge file and load entire business objects collection in memory is not working for us due to memory constraints.
So, It would be very helpful to dev teams if we can have these two APIs.
int GetRowGroupCount();
IEnumerable< T > ReadRowGroup(int i);
Actual behavior
No such API exists
Steps to reproduce the behavior
Code snippet reproducing the behavior
The text was updated successfully, but these errors were encountered: