Skip to content

InvalidCachedStatementError during a database migration #458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jcugat opened this issue Jun 21, 2019 · 3 comments
Closed

InvalidCachedStatementError during a database migration #458

jcugat opened this issue Jun 21, 2019 · 3 comments

Comments

@jcugat
Copy link

jcugat commented Jun 21, 2019

  • asyncpg version: 0.18.3
  • PostgreSQL version: 9.6.8
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : AWS RDS Aurora PostgreSQL, can reproduce it locally
  • Python version: 3.6.8
  • Platform: Linux
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes
  • If you built asyncpg locally, which version of Cython did you use?:
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : uvloop

I recently got bit by what's described at the end of this issue: #103 (review)

I did a schema migration of a table while the service was running, and a few errors like this appeared (once for each process):

    results = await connection.fetch(query)
  File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 421, in fetch
    return await self._execute(query, args, 0, timeout)
  File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 1414, in _execute
    query, args, limit, timeout, return_status=return_status)
  File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 1422, in __execute
    return await self._do_execute(query, executor, timeout)
  File "/usr/local/lib/python3.6/site-packages/asyncpg/connection.py", line 1444, in _do_execute
    result = await executor(stmt, None)
  File "asyncpg/protocol/protocol.pyx", line 196, in bind_execute
asyncpg.exceptions.InvalidCachedStatementError: cached statement plan is invalid due to a database schema or configuration change

The problem is that the query executed was inside a transaction, so there was no way for asyncpg to recover from that state (and why the exception was raised).

Is changing the parameter statement_cache_size=0 the only way to avoid this? Do you know if there is a lot of performance penalty when disabling this cache?

@elprans
Copy link
Member

elprans commented Jun 21, 2019

Disabling statement cache for migrations is the way to avoid this. The performance difference in this context is negligible.

@jcugat
Copy link
Author

jcugat commented Jul 1, 2019

We've applied this change and it's working fine 👍

@jcugat jcugat closed this as completed Jul 1, 2019
@andrew222651
Copy link
Contributor

Disabling statement cache for migrations is the way to avoid this. The performance difference in this context is negligible.

Is there a way to do this using SQLAlchemy?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants