-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read EAV -oneshot #138
Comments
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
Proof-of-concept -not generalized. See #138
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
wibeasley
added a commit
that referenced
this issue
Feb 26, 2017
more robustly from during the creation of ds_eav_2. Also, move the now-old snippets towards the bottom. See #138
The playground for the oneshot is done. I'm moving on to experimenting w/ batching & EAV (#145). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Read EAV (instead of wide/flat) from the REDCap API, and convert it to a flat (non-EAV) dataset. This will be more work for R on the client than before. But it should be faster overall, because the REDCap webserver doesn't have to concatenate each cell in the EAV to as long of a string.
The advantages will be even greater when the REDCap project is 'sparse', in the sense that many variables aren't assigned to every event. The string concatenation in PHP still has to add a comma (and thus the memory-performance hit) in the wide format. But in the EAV format, those columns/rows/cells are simply passed over.
Even if it took the client the same amount of time, it's still better, because the work is distributed to the clients, instead of clogging up the webserver on long 10k+ reads.
(Thanks to discussion and guidance from @taylorr4, @123andy, & @aarenson for a 1+ years)
The text was updated successfully, but these errors were encountered: