[Mitigation] configurable chunk size for inserts #92
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello @calebporzio !
I think it's my first contribution here, so first of all thanks for this amazing package. It's simplicity does not equal show the value it provides.
That said, I am working on a project where we use
Sushi
to work with files stored in the filesystem.This files represent the rows in the model, for each of them we get some properties to save along. The list of files is not static and can grow/shrink over time, and with that grows the number of attributes.
In the other day I tried to access my
sushi
model after creating one more file and got the error:SQLSTATE[HY000]: General error: 1 too many SQL variables
.After sometime debugging I found out that reducing the chunk size to
20
instead of the hardcoded100
solves the issue. In other Sushi models we have 55 is the limit (it depends on the number of attributes that the model have).I think this error is a little bit difficult to debug with a naked eye, and you could assume that your Sqlite instalation is broken somehow and start upgrading versions and trying other crazy configs. (I am looking in the mirror right now!!!) that's why I also added a troubleshoot section in the readme, because everything could be working great and then break avoc without much warning. (and also because readme is indexed by search engines and anyone searching for that error with sushi keyword might find the answer quickly).
I hope you agree with the proposed changes, let me know if there is something more I can do or if I didn't explain myself well.
Wish you the best,
Pedro