Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[objectstore] Problem with special characters in file path #4454

Closed
timvw opened this issue Jun 28, 2023 · 1 comment
Closed

[objectstore] Problem with special characters in file path #4454

timvw opened this issue Jun 28, 2023 · 1 comment
Labels

Comments

@timvw
Copy link

timvw commented Jun 28, 2023

Describe the bug
Files with special characters (@, %) are not supported by ObjectStore.

They are happily accepted on local filesystem and/or S3 though..

To Reproduce
touch "test@dir%2523with_special%2520characters.parquet"

eg: Error: ObjectStore(NotFound { path: "/Users/tim/src/github/qv/test@dir%2523with_special%2520characters.parquet", source: Os { code: 2, kind: NotFound, message: "No such file or directory" } })

Expected behavior
Would like these paths also to be supported

Additional context
Would solve timvw/qv#70

@timvw timvw added the bug label Jun 28, 2023
@tustvold
Copy link
Contributor

tustvold commented Jun 28, 2023

This is by design see https://docs.rs/object_store/latest/object_store/path/struct.Path.html#path-safety. You need to use Path::parse if you are providing an already escaped path.

For example the following works

#[tokio::test]
async fn test_escaped() {
    let root = TempDir::new().unwrap();
    let integration = LocalFileSystem::new_with_prefix(root.path()).unwrap();
    let path =
        Path::parse("test@dir%2523with_special%2520characters.parquet").unwrap();
    integration.put(&path, "foo".into()).await.unwrap();

    integration.head(&path).await.unwrap();
}

@timvw timvw closed this as completed Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants