Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrieve statement metadata after statement execution #41

Merged
merged 5 commits into from
Jul 3, 2024

Conversation

apeschar
Copy link
Contributor

@apeschar apeschar commented Apr 24, 2021

Currently, we get the result metadata immediately after preparing the statement. But this metadata is not guaranteed to match the actual result. And in case of INSERT ... RETURNING, it does not.

MySQL documentation mentions (emphasis mine):

If you call mysql_stmt_result_metadata() after mysql_stmt_prepare() but before mysql_stmt_execute(), the column types in the metadata are as determined by the optimizer. If you call mysql_stmt_result_metadata() after mysql_stmt_execute(), the column types in the metadata are as actually present in the result set. In most cases, these should be the same.

And most ≠ all.

So this PR reloads the result metadata after each statement execution.

This fixes #33, and also ensures correctness of the row structure if table or view definitions change after statement preparation. Also see that issue for reproduction steps.

@apeschar apeschar marked this pull request as ready for review April 24, 2021 17:51
@apeschar
Copy link
Contributor Author

I've rebased this on master, and taken into account the comments on #39.

Specifically mysql_free_result is replaced by mysql_free_result_start and mysql_free_result_cont in the non-blocking implementation.

@apeschar
Copy link
Contributor Author

Specifically mysql_free_result is replaced by mysql_free_result_start and mysql_free_result_cont in the non-blocking implementation.

I realised this is not needed since freeing the metadata should never block. I've restored the original changes rebased on master.

For the "non-blocking" version (which isn't needed), see: master...apeschar:insert-returning-new

This PR should also fix #29 (at least if leaking the metadata is all) and supplant #39 without causing segfaults.

@paurkedal
Copy link
Collaborator

I did some testing with the tweaks to the tests from #29. This fixes the memory leak for blocking_stress_test and it seems to reduce the issue for nonblocking_lwt_stress_test and nonblocking_async_stress_test. There is probably an additional memory leak somewhere, but I think that shouldn't prevent a partial fix. Also, the Caqti test suite runs without issues using this branch for the mariadb driver.

rr0gi added a commit to ahrefs/ocaml-mariadb that referenced this pull request Jun 13, 2024
ocaml-community#41

* apeschar/insert-returning:
  Free statement metadata in blocking implementation
  Restore mysql_free_result binding
  Use free_meta in nonblocking
  Reload meta after each execute
  Delay mysql_stmt_result_metadata until after execute
@ygrek ygrek merged commit 796d240 into ocaml-community:master Jul 3, 2024
@ygrek
Copy link
Collaborator

ygrek commented Jul 3, 2024

verified it fixes memory leak
thanks a lot for figuring it out
sorry took so long :]

@apeschar
Copy link
Contributor Author

apeschar commented Jul 3, 2024

Awesome, thanks!

@paurkedal paurkedal mentioned this pull request Jul 17, 2024
paurkedal added a commit to paurkedal/opam-repository that referenced this pull request Nov 28, 2024
CHANGES:

  - Added `Stmt.start_txn` (ocaml-community/ocaml-mariadb#59 by Corentin Leruth).
  - Added `Res.insert_id` as binding for `mysql_stmt_insert_id` (ocaml-community/ocaml-mariadb#58 by
    Corentin Leruth).
  - Updated to support recent OCaml versions (ocaml-community/ocaml-mariadb#45 by @kit-ty-kate).
  - Fixed too-early retrieval of statement metadata (ocaml-community/ocaml-mariadb#41 by Albert Peschar).
  - Fixed decoding bug for the integer type (ocaml-community/ocaml-mariadb#54 by Raman Varabets, tested
    by ocaml-community/ocaml-mariadb#61 by Corentin Leruth).
  - Fixed a memory leaks related to result metadata (ocaml-community/ocaml-mariadb#39 by Albert Peschar).
  - The build system is now dune and dune-configurator (ocaml-community/ocaml-mariadb#52 by Petter A.
    Urkedal) and some of the examples have been converted to a test suite
    (ocaml-community/ocaml-mariadb#60 by Petter A. Urkedal).
  - The project has been transferred to ocaml-community with Petter A.
    Urkedal as the new maintainer.
paurkedal added a commit to paurkedal/opam-repository that referenced this pull request Nov 28, 2024
CHANGES:

  - Added `Stmt.start_txn` (ocaml-community/ocaml-mariadb#59 by Corentin Leruth).
  - Added `Res.insert_id` as binding for `mysql_stmt_insert_id` (ocaml-community/ocaml-mariadb#58 by
    Corentin Leruth).
  - Updated to support recent OCaml versions (ocaml-community/ocaml-mariadb#45 by @kit-ty-kate).
  - Fixed too-early retrieval of statement metadata (ocaml-community/ocaml-mariadb#41 by Albert Peschar).
  - Fixed decoding bug for the integer type (ocaml-community/ocaml-mariadb#54 by Raman Varabets, tested
    by ocaml-community/ocaml-mariadb#61 by Corentin Leruth).
  - Fixed a memory leaks related to result metadata (ocaml-community/ocaml-mariadb#39 by Albert Peschar).
  - The build system is now dune and dune-configurator (ocaml-community/ocaml-mariadb#52 by Petter A.
    Urkedal) and some of the examples have been converted to a test suite
    (ocaml-community/ocaml-mariadb#60 by Petter A. Urkedal).
  - The project has been transferred to ocaml-community with Petter A.
    Urkedal as the new maintainer.
paurkedal added a commit to paurkedal/opam-repository that referenced this pull request Nov 29, 2024
CHANGES:

  - Added `Stmt.start_txn` (ocaml-community/ocaml-mariadb#59 by Corentin Leruth).
  - Added `Res.insert_id` as binding for `mysql_stmt_insert_id` (ocaml-community/ocaml-mariadb#58 by
    Corentin Leruth).
  - Updated to support recent OCaml versions (ocaml-community/ocaml-mariadb#45 by @kit-ty-kate).
  - Fixed too-early retrieval of statement metadata (ocaml-community/ocaml-mariadb#41 by Albert Peschar).
  - Fixed decoding bug for the integer type (ocaml-community/ocaml-mariadb#54 by Raman Varabets, tested
    by ocaml-community/ocaml-mariadb#61 by Corentin Leruth).
  - Fixed a memory leaks related to result metadata (ocaml-community/ocaml-mariadb#39 by Albert Peschar).
  - The build system is now dune and dune-configurator (ocaml-community/ocaml-mariadb#52 by Petter A.
    Urkedal) and some of the examples have been converted to a test suite
    (ocaml-community/ocaml-mariadb#60 by Petter A. Urkedal).
  - The project has been transferred to ocaml-community with Petter A.
    Urkedal as the new maintainer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mariadb 10.5 Insert ... Returning
3 participants