Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mitigation] configurable chunk size for inserts #92

Merged
merged 1 commit into from
Aug 11, 2021

Conversation

pxpm
Copy link
Contributor

@pxpm pxpm commented Jul 22, 2021

Hello @calebporzio !

I think it's my first contribution here, so first of all thanks for this amazing package. It's simplicity does not equal show the value it provides.

That said, I am working on a project where we use Sushi to work with files stored in the filesystem.

This files represent the rows in the model, for each of them we get some properties to save along. The list of files is not static and can grow/shrink over time, and with that grows the number of attributes.

In the other day I tried to access my sushi model after creating one more file and got the error: SQLSTATE[HY000]: General error: 1 too many SQL variables.

After sometime debugging I found out that reducing the chunk size to 20 instead of the hardcoded 100 solves the issue. In other Sushi models we have 55 is the limit (it depends on the number of attributes that the model have).

I think this error is a little bit difficult to debug with a naked eye, and you could assume that your Sqlite instalation is broken somehow and start upgrading versions and trying other crazy configs. (I am looking in the mirror right now!!!) that's why I also added a troubleshoot section in the readme, because everything could be working great and then break avoc without much warning. (and also because readme is indexed by search engines and anyone searching for that error with sushi keyword might find the answer quickly).

I hope you agree with the proposed changes, let me know if there is something more I can do or if I didn't explain myself well.

Wish you the best,
Pedro

@calebporzio
Copy link
Owner

This is perfect, thanks!

@calebporzio calebporzio merged commit f82805f into calebporzio:master Aug 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants