Incorrect escaping in Printf on master (@sprintf("GAP[%%]"))

[petvana]

On master, the Printf library is redisegned but the escape sequences seems not to be hanled correctly (or like in C as written in docs). Now, it is not possible to print single '%' in the middle of string.

julia> using Printf
julia> @sprintf("GAP[%%]")
ERROR: LoadError: ArgumentError: invalid format string: 'GAP[%%]', invalid type specifier: ']'
Stacktrace:
 [1] Printf.Format(f::String)
   @ Printf ~/software/julia/julia-dev/usr/share/julia/stdlib/v1.6/Printf/src/Printf.jl:157
 [2] @sprintf(__source__::LineNumberNode, __module__::Module, fmt::Any, args::Vararg{Any, N} where N)
   @ Printf ~/software/julia/julia-dev/usr/share/julia/stdlib/v1.6/Printf/src/Printf.jl:783
in expression starting at REPL[3]:1

C style

julia> @ccall printf("GAP[%%]\n"::Cstring)::Cvoid
GAP[%]

The problem of the new design comes from the commit #32859 where the structure in stdlib/Printf/src/Printf.jl

struct Format{S, T}
    str::S # original full format string as CodeUnits
    # keep track of non-format specifier strings to print
    # length(substringranges) == length(formats) + 1
    # so when printing, we start with printing
      # str[substringranges[1]], then formats[1] + args[1]
      # then str[substringranges[2]], then formats[2] + args[2]
      # and so on, then at the end, str[substringranges[end]]
    substringranges::Vector{UnitRange{Int}}
    formats::T # Tuple of Specs
end

expects that length(substringranges) == length(formats) + 1 but it is not true in general if "%%" occurs in the middle of the string format. To provide simple example, there is no format but several substrings in the following:

@assert @sprintf("a%%f%%d%%x") == "a%f%d%x"

Utilized version of master branch

julia> versioninfo()
Julia Version 1.6.0-DEV.unknown
Commit b89c4a3 (2020-09-28 08:12 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i5-1035G1 CPU @ 1.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-10.0.1 (ORCJIT, icelake-client)

Btw, all it works well on 1.5.2.

[petvana]

There is also a newly added unit test that fails in Julia 1.5.2, see (pull/37470).

@test @sprintf("%s%%%s", "a", "b") == "a%%b"

I propose to update it to

@test @sprintf("%s%%%s", "a", "b") == "a%b"

[quinnj]

Thanks for reporting @petvana! Fix is up: #37785.

[petvana]

Wow! Thanks for the rapid response. You solved the existing test case, but the original issue with "[%%]" is still unsolved.

The printing in function format(buf::Vector{UInt8}, pos::Integer, f::Format, args...) is fixed, but the substringranges seem not be correctly detected in Format(f::AbstractString).

Thus, I propose to add to more tests like:

@test Printf.@sprintf("1%%") == "1%"
@test Printf.@sprintf("%%1") == "%1"
@test Printf.@sprintf("1%%2") == "1%2"
@test Printf.@sprintf("1%%%d", 2) == "1%2"
@test Printf.@sprintf("1%%2%%3") == "1%2%3"

[petvana]

I have tried to fix that in my fork petvana/julia/tree/jq/37784. I have moved the format detection into a separate inlined function for better readability. If you like it, I can submit a pull request to the branch. It succeeds in all tests, but I have concerns about its performance. @quinnj Which instances you use for performance testing?

[KristofferC]

Make sure to run any performance testing on top of #37802. At least for floats.

[quinnj]

@petvana, yes, feel free to do a PR against my branch and I can review/try it out.