-
-
Notifications
You must be signed in to change notification settings - Fork 6.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make std::filesystem::path conversion to/from UTF-8 encoded string explicit #4631
base: develop
Are you sure you want to change the base?
Conversation
61a26dd
to
a395539
Compare
…plicit. Signed-off-by: Richard Musil <risa2000x@gmail.com>
a395539
to
008c50b
Compare
I am bit confused about how I could not run it on Windows (seems like |
asytle and the amalgamation is deterministic. There should not be different results after subsequent calls.
It's based on Python. The calls the Makefile makes are: python3 -mvenv tools/astyle/venv
tools/astyle/venv/bin/pip3 install --quiet --upgrade pip
tools/astyle/venv/bin/pip3 install --quiet -r tools/astyle/requirements.txt
tools/amalgamate/amalgamate.py -c tools/amalgamate/config_json_fwd.json -s . --verbose=yes
tools/astyle/venv/bin/astyle --project=tools/astyle/.astylerc include/nlohmann/adl_serializer.hpp include/nlohmann/byte_container_with_subtype.hpp include/nlohmann/detail/abi_macros.hpp include/nlohmann/detail/conversions/from_json.hpp include/nlohmann/detail/conversions/to_chars.hpp include/nlohmann/detail/conversions/to_json.hpp include/nlohmann/detail/exceptions.hpp include/nlohmann/detail/hash.hpp include/nlohmann/detail/input/binary_reader.hpp include/nlohmann/detail/input/input_adapters.hpp include/nlohmann/detail/input/json_sax.hpp include/nlohmann/detail/input/lexer.hpp include/nlohmann/detail/input/parser.hpp include/nlohmann/detail/input/position_t.hpp include/nlohmann/detail/iterators/internal_iterator.hpp include/nlohmann/detail/iterators/iter_impl.hpp include/nlohmann/detail/iterators/iteration_proxy.hpp include/nlohmann/detail/iterators/iterator_traits.hpp include/nlohmann/detail/iterators/json_reverse_iterator.hpp include/nlohmann/detail/iterators/primitive_iterator.hpp include/nlohmann/detail/json_custom_base_class.hpp include/nlohmann/detail/json_pointer.hpp include/nlohmann/detail/json_ref.hpp include/nlohmann/detail/macro_scope.hpp include/nlohmann/detail/macro_unscope.hpp include/nlohmann/detail/meta/call_std/begin.hpp include/nlohmann/detail/meta/call_std/end.hpp include/nlohmann/detail/meta/cpp_future.hpp include/nlohmann/detail/meta/detected.hpp include/nlohmann/detail/meta/identity_tag.hpp include/nlohmann/detail/meta/is_sax.hpp include/nlohmann/detail/meta/std_fs.hpp include/nlohmann/detail/meta/type_traits.hpp include/nlohmann/detail/meta/void_t.hpp include/nlohmann/detail/output/binary_writer.hpp include/nlohmann/detail/output/output_adapters.hpp include/nlohmann/detail/output/serializer.hpp include/nlohmann/detail/string_concat.hpp include/nlohmann/detail/string_escape.hpp include/nlohmann/detail/string_utils.hpp include/nlohmann/detail/value_t.hpp include/nlohmann/json.hpp include/nlohmann/json_fwd.hpp include/nlohmann/ordered_map.hpp include/nlohmann/thirdparty/hedley/hedley.hpp include/nlohmann/thirdparty/hedley/hedley_undef.hpp tests/abi/config/config.hpp tests/abi/config/custom.cpp tests/abi/config/default.cpp tests/abi/config/noversion.cpp tests/abi/diag/diag.cpp tests/abi/diag/diag.hpp tests/abi/diag/diag_off.cpp tests/abi/diag/diag_on.cpp tests/abi/inline_ns/use_current.cpp tests/abi/inline_ns/use_v3_10_5.cpp tests/abi/main.cpp tests/benchmarks/src/benchmarks.cpp tests/cmake_add_subdirectory/project/main.cpp tests/cmake_fetch_content/project/main.cpp tests/cmake_fetch_content2/project/main.cpp tests/cmake_import/project/main.cpp tests/cmake_import_minver/project/main.cpp tests/cmake_target_include_directories/project/Bar.cpp tests/cmake_target_include_directories/project/Bar.hpp tests/cmake_target_include_directories/project/Foo.cpp tests/cmake_target_include_directories/project/Foo.hpp tests/cmake_target_include_directories/project/main.cpp tests/cuda_example/json_cuda.cu tests/src/fuzzer-driver_afl.cpp tests/src/fuzzer-parse_bjdata.cpp tests/src/fuzzer-parse_bson.cpp tests/src/fuzzer-parse_cbor.cpp tests/src/fuzzer-parse_json.cpp tests/src/fuzzer-parse_msgpack.cpp tests/src/fuzzer-parse_ubjson.cpp tests/src/make_test_data_available.hpp tests/src/test_utils.hpp tests/src/unit-32bit.cpp tests/src/unit-algorithms.cpp tests/src/unit-allocator.cpp tests/src/unit-alt-string.cpp tests/src/unit-assert_macro.cpp tests/src/unit-binary_formats.cpp tests/src/unit-bjdata.cpp tests/src/unit-bson.cpp tests/src/unit-byte_container_with_subtype.cpp tests/src/unit-capacity.cpp tests/src/unit-cbor.cpp tests/src/unit-class_const_iterator.cpp tests/src/unit-class_iterator.cpp tests/src/unit-class_lexer.cpp tests/src/unit-class_parser.cpp tests/src/unit-class_parser_diagnostic_positions.cpp tests/src/unit-comparison.cpp tests/src/unit-concepts.cpp tests/src/unit-constructor1.cpp tests/src/unit-constructor2.cpp tests/src/unit-convenience.cpp tests/src/unit-conversions.cpp tests/src/unit-custom-base-class.cpp tests/src/unit-deserialization.cpp tests/src/unit-diagnostic-positions-only.cpp tests/src/unit-diagnostic-positions.cpp tests/src/unit-diagnostics.cpp tests/src/unit-disabled_exceptions.cpp tests/src/unit-element_access1.cpp tests/src/unit-element_access2.cpp tests/src/unit-hash.cpp tests/src/unit-inspection.cpp tests/src/unit-items.cpp tests/src/unit-iterators1.cpp tests/src/unit-iterators2.cpp tests/src/unit-iterators3.cpp tests/src/unit-json_patch.cpp tests/src/unit-json_pointer.cpp tests/src/unit-large_json.cpp tests/src/unit-locale-cpp.cpp tests/src/unit-merge_patch.cpp tests/src/unit-meta.cpp tests/src/unit-modifiers.cpp tests/src/unit-msgpack.cpp tests/src/unit-no-mem-leak-on-adl-serialize.cpp tests/src/unit-noexcept.cpp tests/src/unit-ordered_json.cpp tests/src/unit-ordered_map.cpp tests/src/unit-pointer_access.cpp tests/src/unit-readme.cpp tests/src/unit-reference_access.cpp tests/src/unit-regression1.cpp tests/src/unit-regression2.cpp tests/src/unit-serialization.cpp tests/src/unit-testsuites.cpp tests/src/unit-to_chars.cpp tests/src/unit-type_traits.cpp tests/src/unit-ubjson.cpp tests/src/unit-udl.cpp tests/src/unit-udt.cpp tests/src/unit-udt_macro.cpp tests/src/unit-unicode1.cpp tests/src/unit-unicode2.cpp tests/src/unit-unicode3.cpp tests/src/unit-unicode4.cpp tests/src/unit-unicode5.cpp tests/src/unit-user_defined_input.cpp tests/src/unit-windows_h.cpp tests/src/unit-wstring.cpp tests/src/unit.cpp single_include/nlohmann/json.hpp single_include/nlohmann/json_fwd.hpp docs/mkdocs/docs/examples/*.cpp |
After I pushed my PR, I noticed here when checking the diff that in my PR the amalgamation changed |
This PR makes the default conversions of
std::filesystem::path
from/to JSONUTF-8
encoded string explicit.This is a follow up on #4271.
Original implementation relied on the platform locale to be
UTF-8
and usedstd::string
asstd::filesystem::path
conversion source/target. This did not work on Windows (where the implicit locale is using "multibyte" encoding) and may potentially trigger a similar issue on platforms which do not have the locale set toUTF-8
.Proposed change uses explicit conversion functions from
std
library toUTF-8
encoded string. Unfortunately, theUTF-8
handling ofstd
library changed fromC++17
toC++20
so the code uses different codepaths for corresponding C++ standards.This code has been inspired by:
https://github.com/alf-p-steinbach/C---how-to---make-non-English-text-work-in-Windows/blob/main/microlibs/cppm/stdlib_workarounds/fs_path.hpp#L19