Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: to_sql with ADBC driver fails when schema exists without tables #59841

Open
3 tasks done
WillAyd opened this issue Sep 19, 2024 · 1 comment
Open
3 tasks done

BUG: to_sql with ADBC driver fails when schema exists without tables #59841

WillAyd opened this issue Sep 19, 2024 · 1 comment
Assignees
Labels
Bug IO SQL to_sql, read_sql, read_sql_query

Comments

@WillAyd
Copy link
Member

WillAyd commented Sep 19, 2024

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

This is hard to create a reproducer for given sqlite does not have schemas and is not affected by the issue. However, with the proper database setup running code like:

with adbc_driver_postgresql.dbapi.connect() as conn:
    df.to_sql("table", schema="schema_with_no_tables")

Will produce the following traceback:

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/util/_decorators.py:333, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    327 if len(args) > num_allow_args:
    328     warnings.warn(
    329         msg.format(arguments=_format_argument_list(allow_args)),
    330         FutureWarning,
    331         stacklevel=find_stack_level(),
    332     )
--> 333 return func(*args, **kwargs)

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/core/generic.py:3087, in NDFrame.to_sql(self, name, con, schema, if_exists, index, index_label, chunksize, dtype, method)
   2889 """
   2890 Write records stored in a DataFrame to a SQL database.
   2891 
   (...)
   3083 [(1,), (None,), (2,)]
   3084 """  # noqa: E501
   3085 from pandas.io import sql
-> 3087 return sql.to_sql(
   3088     self,
   3089     name,
   3090     con,
   3091     schema=schema,
   3092     if_exists=if_exists,
   3093     index=index,
   3094     index_label=index_label,
   3095     chunksize=chunksize,
   3096     dtype=dtype,
   3097     method=method,
   3098 )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:842, in to_sql(frame, name, con, schema, if_exists, index, index_label, chunksize, dtype, method, engine, **engine_kwargs)
    837     raise NotImplementedError(
    838         "'frame' argument should be either a Series or a DataFrame"
    839     )
    841 with pandasSQL_builder(con, schema=schema, need_transaction=True) as pandas_sql:
--> 842     return pandas_sql.to_sql(
    843         frame,
    844         name,
    845         if_exists=if_exists,
    846         index=index,
    847         index_label=index_label,
    848         schema=schema,
    849         chunksize=chunksize,
    850         dtype=dtype,
    851         method=method,
    852         engine=engine,
    853         **engine_kwargs,
    854     )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2386, in ADBCDatabase.to_sql(self, frame, name, if_exists, index, index_label, schema, chunksize, dtype, method, engine, **engine_kwargs)
   2381 # pandas if_exists="append" will still create the
   2382 # table if it does not exist; ADBC is more explicit with append/create
   2383 # as applicable modes, so the semantics get blurred across
   2384 # the libraries
   2385 mode = "create"
-> 2386 if self.has_table(name, schema):
   2387     if if_exists == "fail":
   2388         raise ValueError(f"Table '{table_name}' already exists.")

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2422, in ADBCDatabase.has_table(self, name, schema)
   2419 if not schema_record:
   2420     continue
-> 2422 for table_record in schema_record["db_schema_tables"]:
   2423     if table_record["table_name"] == name:
   2424         return True

TypeError: 'NoneType' object is not iterable

Issue Description

The pandas code does not handle well the case where schema_record["db_schema_tables"] may be None

Expected Behavior

No error

Installed Versions

2.2.2

@WillAyd WillAyd added Bug IO SQL to_sql, read_sql, read_sql_query labels Sep 19, 2024
@aram-cinnamon
Copy link
Contributor

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
Development

No branches or pull requests

2 participants