GH-48721: [C++] Add test for UTF-8 filenames on Windows

[HyukjinKwon]

Rationale for this change

4937d9f (ARROW-5102) added the TODO comment requesting a test with valid UTF-8 filenames. Later, the UTF-8 to UTF-16 conversion logic on Windows was introduced in commit eb23ea9 (ARROW-5648) which should fix the issue.

Essentially we should add a test:

1. Creates file with UTF-8 filename: "test_file_한국어_😀.txt"
2. FileOutputStream::Open() converts UTF-8 to UTF-16 via the following call chain:
   

   arrow/cpp/src/arrow/io/file.cc

   Lines 359 to 364 in 727106f

   	Result<std::shared_ptr<FileOutputStream>> FileOutputStream::Open(const std::string& path,
   	bool append) {
   	auto stream = std::shared_ptr<FileOutputStream>(new FileOutputStream());
   	RETURN_NOT_OK(stream->impl_->Open(path, append));
   	return stream;
   	}

   (FileOutputStream::Open(path))
   

   arrow/cpp/src/arrow/io/file.cc

   Lines 186 to 188 in 727106f

   	Status SetFileName(const std::string& file_name) {
   	return ::arrow::internal::PlatformFilename::FromString(file_name).Value(&file_name_);
   	}

   (OSFile::SetFileName(path))
   

   arrow/cpp/src/arrow/util/io_util.cc

   Lines 604 to 608 in 727106f

   	Result<PlatformFilename> PlatformFilename::FromString(std::string_view file_name) {
   	RETURN_NOT_OK(ValidatePath(file_name));
   	ARROW_ASSIGN_OR_RAISE(auto ns, StringToNative(file_name));
   	return PlatformFilename(std::move(ns));
   	}

   (PlatformFilename::FromString())
   

   arrow/cpp/src/arrow/util/io_util.cc

   Lines 143 to 149 in 727106f

   	Result<NativePathString> StringToNative(std::string_view s) {
   	#if _WIN32
   	return ::arrow::util::UTF8ToWideString(s);
   	#else
   	return std::string(s);
   	#endif
   	}

   (StringToNative())
   

   arrow/cpp/src/arrow/util/utf8.h

   Line 34 in 727106f

   ARROW_EXPORT Result<std::wstring> UTF8ToWideString(std::string_view source);

   (UTF8ToWideString())
3. Performs write and read operations
4. Check data exists

This test complements existing FileNameWideCharConversionRangeException test (invalid UTF-8):

arrow/cpp/src/arrow/io/file_test.cc

Lines 111 to 116 in 727106f

	TEST_F(TestFileOutputStream, FileNameWideCharConversionRangeException) {
	// Invalid utf-8 filename
	std::string file_name = "\x80";
	ASSERT_RAISES(Invalid, FileOutputStream::Open(file_name));
	ASSERT_RAISES(Invalid, ReadableFile::Open(file_name));
	}

Why Windows-only?

• Wrapped in #if defined(_MSC_VER) because Windows uses NativePathString = std::wstring (UTF-16):

  arrow/cpp/src/arrow/util/io_util.h

  Line 48 in 727106f

  using NativePathString = std::wstring;

• Unix uses byte strings:

  arrow/cpp/src/arrow/util/io_util.h

  Lines 44 to 51 in 727106f

  	// NOTE: 8-bit path strings on Windows are encoded using UTF-8.
  	// Using MBCS would fail encoding some paths.
  	#if defined(_WIN32)
  	using NativePathString = std::wstring;
  	#else
  	using NativePathString = std::string;
  	#endif

What changes are included in this PR?

This PR adds the test described above.

Are these changes tested?

Unittest was added.

Are there any user-facing changes?

No, test-only.

• GitHub Issue: [C++] Add test for UTF-8 filenames on Windows #48721

[github-actions]

⚠️ GitHub issue #48721 has been automatically assigned in GitHub to PR creator.

[HyukjinKwon]

한국어 is "Korean" in Korean FYI .. :-)..

[pitrou]

Thanks contributing this @HyukjinKwon . Looks good in general, just one comment.

Also, can you please trim down the PR description?

[pitrou]

Why test this only on Windows? Let's test this on all platforms.