Skip to content

Bug: ListingTableFactory fails to read data when the final path element contains a . #17212

@BlakeOrth

Description

@BlakeOrth

Describe the bug

If a path to an external table contains a . in the final path element (i.e. folders are named with . delimited versioning) tables created via the ListingTableFactory (leveraged by datafusion-cli) fail to read the table data.

To Reproduce

DataFusion CLI v49.0.1
> CREATE EXTERNAL TABLE
versioned_example(a string)
STORED AS parquet
LOCATION '/tmp/tables/table.v1/';
0 row(s) fetched.
Elapsed 0.004 seconds.

> INSERT INTO versioned_example VALUES ('data');
+-------+
| count |
+-------+
| 1     |
+-------+
1 row(s) fetched.
Elapsed 0.007 seconds.

> describe versioned_example;
+-------------+-----------+-------------+
| column_name | data_type | is_nullable |
+-------------+-----------+-------------+
| a           | Utf8View  | YES         |
+-------------+-----------+-------------+
1 row(s) fetched.
Elapsed 0.002 seconds.

> select * from versioned_example;
+---+
| a |
+---+
+---+
0 row(s) fetched.
Elapsed 0.002 seconds.

> CREATE EXTERNAL TABLE
versioned_example_readback
STORED AS parquet
LOCATION '/tmp/tables/table.v1/';
0 row(s) fetched.
Elapsed 0.002 seconds.

> describe versioned_example_readback;
+-------------+-----------+-------------+
| column_name | data_type | is_nullable |
+-------------+-----------+-------------+
+-------------+-----------+-------------+
0 row(s) fetched.
Elapsed 0.001 seconds.

> select * from versioned_example_readback;
0 row(s) fetched.
Elapsed 0.002 seconds.

Expected behavior

Tables with . in their final path element should act like any other table and successfully read the data under the directory.

Additional context

/pull/17050 fixes this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions