Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accessing grdecl grid cells without allocating memory for the whole file? #720

Open
bartoli opened this issue Aug 3, 2020 · 1 comment

Comments

@bartoli
Copy link

bartoli commented Aug 3, 2020

I tried libecl to access grdecl files from C/C++.

From grdecl files, i need to extract the cell corners coordinates, to find wether a cell is present or not, and the values for some properties for each cell. But i am having issues of excessive memory consumption. Trying to open a 4GB grdecl file is crashing my computer (16GB RAM) with too big allocs (grid ix about 500x500x500).

What i do is

  • Load the various properties using ecl_kw_fscanf_alloc_grdecl_dynamic__(). This stores values in binary format, but uncompresses the '5*-9999' elements, so the buffer for the COORD, ACTNUM, & ZCORN is still 4GB in memory
  • call ecl_grid_alloc_GRDECL_kw() to create an ecl_grid_type. This crashes the computer for the big file.

As i understand, this will always allocate a buffer of nxnynz cells. And the structure for a cell is about 250 bytes each. For a 500x500x500 grid, this only already needs 31GB or RAM.

I don't even need to have all cells in memory at a given time. I could just go with "cell iterator", that from raw ecl_kw_type buffers, gives me access to the corner coordinates for each cell sucesssive cell, so i can store them in an other format.
Is there a way to access the cells geometry information and properties values without the need to have this much data in memory?

@lars-petter-hauge
Copy link
Contributor

Hi @bartoli

Unfortunately I don't believe there is an option to do anything other than to populate the entire grid model on instantiation.

I agree that it would be useful to be able to lazily load parts of the grid though, but unfortunately it isn't anything that is on our priority list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants