Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haven::read_sas() unable to allocate memory for <16MB SAS page sizes #705

Closed
DanChaltiel opened this issue Jan 3, 2023 · 3 comments · Fixed by #713
Closed

haven::read_sas() unable to allocate memory for <16MB SAS page sizes #705

DanChaltiel opened this issue Jan 3, 2023 · 3 comments · Fixed by #713
Labels
bug an unexpected problem or unintended behavior readstat

Comments

@DanChaltiel
Copy link

The same problem as in #697 can also happen with <16MB catalogs if one of the formats is considered invalid.
It is a kind of a corner case but I guess it can provide some information.

For instance, I have to add the following format to my catalog so that my .sas7bdat file is readable with SAS, but this unfortunately fails with haven:

libname catout 'your/path';
Proc format lib=catout;
Value $ate ;
run;

This SAS code creates a formats.sasb7cat catalog file (link) that weighs only 17ko.

Using read_sas(input, catalog) with this catalog file will output the following error, regardless of the input .sas7bdat file:

Failed to parse xxx.sas7bdat: Unable to allocate memory.

I'm not sure whether this relates to ReadStats so I posted it here, sorry if this was not the thing to do.

@gorcha gorcha added readstat bug an unexpected problem or unintended behavior labels Jan 10, 2023
@gorcha
Copy link
Member

gorcha commented Jan 10, 2023

Hi @DanChaltiel, are you able to provide a corresponding sas7bdat file I can use to test?

@DanChaltiel
Copy link
Author

Hi @gorcha, unfortunately, all my sas7bdat files are proprietary and cannot be shared online.
However, I tested with a few and this happened each so you might want to give it a try with the first dummy file you find.
If you cannot reproduce the error, I will try to find a file that does not contain any sensitive information but I'm not sure it even exists.

@gorcha
Copy link
Member

gorcha commented Jan 10, 2023

Hey @DanChaltiel, no worries at all - it look like I'm able to reproduce the error using any sas7bdat file anyway so all good. Thanks!

gorcha added a commit that referenced this issue Feb 21, 2023
Maintains iconv hack from c1f9f19 and solaris hack from 4a878a1.

* Fix various SAS catalog file reading bugs (fix #529, fix #653, fix #680, fix #696, fix #705).
* Increase maximum SAS page file size to 16MB (fix #697).
* Ignore invalid SAV timestamp strings (fix #683).
* Fix compiler warnings (fix #707).
@gorcha gorcha closed this as completed in 196e8eb Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug an unexpected problem or unintended behavior readstat
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants