-
Notifications
You must be signed in to change notification settings - Fork 898
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ultimate batch saving speedup #15761
Commits on Aug 9, 2017
-
Check if IC has serializable_keys? and transform them
Check if IC has serializable_keys? and transform them, this saves some computing since we don not have to use :use_ar_object just because the model has: serialize <col_name>
Configuration menu - View commit details
-
Copy full SHA for 23c6e47 - Browse repository at this point
Copy the full SHA 23c6e47View commit details -
Configuration menu - View commit details
-
Copy full SHA for 374f1fd - Browse repository at this point
Copy the full SHA 374f1fdView commit details -
Use to_sym for individual keys instead of symbolize_keys!
Use to_sym for individual keys instead of symbolize_keys!
Configuration menu - View commit details
-
Copy full SHA for 8c8a941 - Browse repository at this point
Copy the full SHA 8c8a941View commit details -
Performance tweaks for the base saver
Performance tweaks for the base saver, getting repeated logic computed in initializer. Getting record_key abrtratcion for fetching attributes of non AR records.
Configuration menu - View commit details
-
Copy full SHA for e6d0c48 - Browse repository at this point
Copy the full SHA e6d0c48View commit details -
Perf tweaks of the batch saver
Perf tweaks of the batch saver: 1. Having iterator that fetches raw SQL without creationg AR objects. 2. Select on the fetched data to reduce the mem size needed. 3. Bumping batch_size to 10k, since the objects are 10x smaller than AR objects. And few small tweaks in the core saving
Configuration menu - View commit details
-
Copy full SHA for 2894ecf - Browse repository at this point
Copy the full SHA 2894ecfView commit details -
Perf tweaks in SQL mixin: 1. Store connection for the whole batch 2. Precalculate pg_types for faster type_casting
Configuration menu - View commit details
-
Copy full SHA for b32d820 - Browse repository at this point
Copy the full SHA b32d820View commit details -
Configuration menu - View commit details
-
Copy full SHA for 84556e4 - Browse repository at this point
Copy the full SHA 84556e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7e34e96 - Browse repository at this point
Copy the full SHA 7e34e96View commit details -
Configuration menu - View commit details
-
Copy full SHA for 59b7895 - Browse repository at this point
Copy the full SHA 59b7895View commit details -
Move batch size to attributes with setter
Move batch size to attributes with setter, so we avoid computing it multiple times.
Configuration menu - View commit details
-
Copy full SHA for c80763d - Browse repository at this point
Copy the full SHA c80763dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f09ddd - Browse repository at this point
Copy the full SHA 7f09dddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 08ee63f - Browse repository at this point
Copy the full SHA 08ee63fView commit details -
Store primary_key_value to avoid multiple method calls
Store primary_key_value to avoid multiple method calls
Configuration menu - View commit details
-
Copy full SHA for 577e10f - Browse repository at this point
Copy the full SHA 577e10fView commit details
Commits on Aug 10, 2017
-
Use reorder for find_in_batches working properly
Use reorder for find_in_batches working properly
Configuration menu - View commit details
-
Copy full SHA for 60e14d1 - Browse repository at this point
Copy the full SHA 60e14d1View commit details