Skip to content

to_sql is very slow for sql server with sqlalchemy #23274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
anjsudh opened this issue Oct 22, 2018 · 3 comments
Closed

to_sql is very slow for sql server with sqlalchemy #23274

anjsudh opened this issue Oct 22, 2018 · 3 comments

Comments

@anjsudh
Copy link
Contributor

anjsudh commented Oct 22, 2018

df.to_sql is very slow, and takes hours, when connecting to sql server using sql alchemy connection
https://github.com/mkleehammer/pyodbc/wiki/Features-beyond-the-DB-API#fast_executemany

Performance is much faster after using fast_executemany

@event.listens_for(engine_local, 'before_cursor_execute')
def receive_before_cursor_execute(conn, cursor, statement, params, context, executemany):
    if executemany:
        cursor.fast_executemany = True

Can enable fast_executemany in to_sql implementation for SQL Server

@TomAugspurger
Copy link
Contributor

Duplicate of #8953 I believe. #21401 will fix this.

@TomAugspurger TomAugspurger added this to the No action milestone Oct 22, 2018
@tauqeerhasan
Copy link

please share the full code to export dataframe to database..
i have 10300000 rows and df.to_sql is working very very slow.. i need a fast performance code..

@sepjc96
Copy link

sepjc96 commented Mar 24, 2021

@tauqeerhasan have you figured out how to enhance the speed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants