Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build errors for JLD, HDF5, Cairo, SQLite #134

Closed
kafisatz opened this issue Dec 6, 2017 · 13 comments · Fixed by #141
Closed

build errors for JLD, HDF5, Cairo, SQLite #134

kafisatz opened this issue Dec 6, 2017 · 13 comments · Fixed by #141

Comments

@kafisatz
Copy link

kafisatz commented Dec 6, 2017

I am unable to build these packages on v0.7 and Windows10.
Some file is always missing.

Pkg.add fails, Pkg.build fails too.
Notably Pkg.build("WinRPM") works fine though.

Does anyone know why said file is missing?

I note that I originally opened this in Cairo.jl
It was suggested that this may be a winrpm issue.

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-DEV.2690 (2017-11-30 01:27 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 7825911026* (6 days old master)
|__/                   |  x86_64-w64-mingw32

julia> versioninfo()
Julia Version 0.7.0-DEV.2690
Commit 7825911026* (2017-11-30 01:27 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, skylake)
Environment:
  JULIA_HOME = C:\Julia-0.7.X\bin\

julia> Pkg.build("Cairo")
Executing .juliarc.jl... done
INFO: Building WinRPM
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_Leap_42.2/repodata/repomd.xml
WARNING: findfirst(A, v) is deprecated, use findfirst(equalto(v), A) instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] findfirst(::Array{UInt16,1}, ::Int64) at .\deprecated.jl:56
 [3] #download#5 at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:62 [inlined]
 [4] download at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:56 [inlined]
 [5] (::getfield(WinRPM, Symbol("#cacheget#9")){Bool,Bool})(::String, ::Bool) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:151
 [6] update(::Bool, ::Bool) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:162
 [7] update() at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:129
 [8] top-level scope
 [9] include_relative(::Module, ::String) at .\loading.jl:509
 [10] include at .\sysimg.jl:15 [inlined]
 [11] include(::String) at .\loading.jl:543
 [12] top-level scope
 [13] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [14] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [15] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [16] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [17] top-level scope
 [18] eval(::Module, ::Expr) at .\sysimg.jl:60
 [19] process_options(::Base.JLOptions) at .\client.jl:322
 [20] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\deps\build.jl:2
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_Leap_42.2/repodata/repomd.xml
Executing .juliarc.jl... done
INFO: Building Cairo
WARNING: is_windows is deprecated, use Sys.iswindows instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] is_windows() at .\deprecated.jl:31
 [3] top-level scope
 [4] include_relative(::Module, ::String) at .\loading.jl:509
 [5] include at .\sysimg.jl:15 [inlined]
 [6] include(::String) at .\loading.jl:543
 [7] top-level scope
 [8] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [9] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [10] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [11] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [12] top-level scope
 [13] eval(::Module, ::Expr) at .\sysimg.jl:60
 [14] process_options(::Base.JLOptions) at .\client.jl:322
 [15] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:31
WARNING: is_apple is deprecated, use Sys.isapple instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] is_apple() at .\deprecated.jl:31
 [3] top-level scope
 [4] include_relative(::Module, ::String) at .\loading.jl:509
 [5] include at .\sysimg.jl:15 [inlined]
 [6] include(::String) at .\loading.jl:543
 [7] top-level scope
 [8] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [9] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [10] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [11] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [12] top-level scope
 [13] eval(::Module, ::Expr) at .\sysimg.jl:60
 [14] process_options(::Base.JLOptions) at .\client.jl:322
 [15] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:39
WARNING: is_windows is deprecated, use Sys.iswindows instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] is_windows() at .\deprecated.jl:31
 [3] xx(::String, ::Vararg{String,N} where N) at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:110
 [4] top-level scope
 [5] include_relative(::Module, ::String) at .\loading.jl:509
 [6] include at .\sysimg.jl:15 [inlined]
 [7] include(::String) at .\loading.jl:543
 [8] top-level scope
 [9] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [10] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [11] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [12] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [13] top-level scope
 [14] eval(::Module, ::Expr) at .\sysimg.jl:60
 [15] process_options(::Base.JLOptions) at .\client.jl:322
 [16] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:112
WARNING: is_linux is deprecated, use Sys.islinux instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] is_linux() at .\deprecated.jl:31
 [3] top-level scope
 [4] include_relative(::Module, ::String) at .\loading.jl:509
 [5] include at .\sysimg.jl:15 [inlined]
 [6] include(::String) at .\loading.jl:543
 [7] top-level scope
 [8] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [9] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [10] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [11] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [12] top-level scope
 [13] eval(::Module, ::Expr) at .\sysimg.jl:60
 [14] process_options(::Base.JLOptions) at .\client.jl:322
 [15] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:112
INFO: Updating WinRPM package list
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_Leap_42.2/repodata/repomd.xml
WARNING: findfirst(A, v) is deprecated, use findfirst(equalto(v), A) instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] findfirst(::Array{UInt16,1}, ::Int64) at .\deprecated.jl:56
 [3] #download#5 at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:62 [inlined]
 [4] download at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:56 [inlined]
 [5] (::getfield(WinRPM, Symbol("#cacheget#9")){Bool,Bool})(::String, ::Bool) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:151
 [6] update(::Bool, ::Bool) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:162
 [7] package_available(::WinRPM.RPM) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\winrpm_bindeps.jl:23
 [8] can_provide(::WinRPM.RPM, ::Dict{Symbol,Any}, ::BinDeps.LibraryDependency) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:785
 [9] #_find_library#46(::Type, ::Function, ::BinDeps.LibraryDependency) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:642
 [10] collect_to! at .\array.jl:604 [inlined]
 [11] collect_to_with_first!(::Array{Tuple{BinDeps.LibraryDependency,Array{Any,1}},1}, ::Tuple{BinDeps.LibraryDependency,Array{Any,1}}, ::Base.Generator{Array{BinDeps.LibraryDependency,1},getfield(BinDeps, Symbol("##49#50"))}, ::Int64) at .\array.jl:591
 [12] collect(::Base.Generator{Array{BinDeps.LibraryDependency,1},getfield(BinDeps, Symbol("##49#50"))}) at .\array.jl:572
 [13] allf at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:799 [inlined]
 [14] satisfied_providers at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:801 [inlined]
 [15] satisfy!(::BinDeps.LibraryGroup, ::Array{DataType,1}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:867 (repeats 2 times)
 [16] top-level scope at .\<missing>:970
 [17] include_relative(::Module, ::String) at .\loading.jl:509
 [18] include at .\sysimg.jl:15 [inlined]
 [19] include(::String) at .\loading.jl:543
 [20] top-level scope
 [21] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [22] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [23] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [24] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [25] top-level scope
 [26] eval(::Module, ::Expr) at .\sysimg.jl:60
 [27] process_options(::Base.JLOptions) at .\client.jl:322
 [28] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:162
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_Leap_42.2/repodata/repomd.xml
INFO: Packages to install: libintl8, glib2-lang, libffi6, libglib-2_0-0, libgobject-2_0-0, glib2
INFO: Downloading: libintl8
WARNING: findfirst(A, v) is deprecated, use findfirst(equalto(v), A) instead.
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] findfirst(::Array{UInt16,1}, ::Int64) at .\deprecated.jl:56
 [3] #download#5 at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:62 [inlined]
 [4] download at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:56 [inlined]
 [5] do_install(::WinRPM.Package) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:455
 [6] do_install at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:445 [inlined]
 [7] #install#21(::Bool, ::Function, ::WinRPM.Packages{Array{LibExpat.ETree,1}}) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:392
 [8] (::getfield(WinRPM, Symbol("#kw##install")))(::Array{Any,1}, ::typeof(WinRPM.install), ::WinRPM.Packages{Array{LibExpat.ETree,1}}) at .\<missing>:0
 [9] #install#20(::Bool, ::Function, ::Array{String,1}, ::String) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:368
 [10] (::getfield(WinRPM, Symbol("#kw##install")))(::Array{Any,1}, ::typeof(WinRPM.install), ::Array{String,1}, ::String) at .\<missing>:0 (repeats 2 times)
 [11] (::getfield(WinRPM, Symbol("##36#37")){WinRPM.RPM})() at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\winrpm_bindeps.jl:42
 [12] run(::getfield(WinRPM, Symbol("##36#37")){WinRPM.RPM}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\BinDeps.jl:474
 [13] run(::BinDeps.SynchronousStepCollection) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\BinDeps.jl:517
 [14] satisfy!(::BinDeps.LibraryDependency, ::Array{DataType,1}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:937
 [15] satisfy!(::BinDeps.LibraryGroup, ::Array{DataType,1}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:903
 [16] satisfy!(::BinDeps.LibraryGroup) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:867
 [17] top-level scope at .\<missing>:970
 [18] include_relative(::Module, ::String) at .\loading.jl:509
 [19] include at .\sysimg.jl:15 [inlined]
 [20] include(::String) at .\loading.jl:543
 [21] top-level scope
 [22] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [23] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [24] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [25] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [26] top-level scope
 [27] eval(::Module, ::Expr) at .\sysimg.jl:60
 [28] process_options(::Base.JLOptions) at .\client.jl:322
 [29] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:162
INFO: Extracting: libintl8
WARNING: open(cmd) now returns only a Process<:IO object
Stacktrace:
 [1] depwarn at .\deprecated.jl:68 [inlined]
 [2] getindex(::Base.Process, ::Int64) at .\deprecated.jl:1123
 [3] indexed_next(::Base.Process, ::Int64, ::Int64) at .\tuple.jl:60
 [4] do_install(::WinRPM.Package) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:469
 [5] do_install at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:445 [inlined]
 [6] #install#21(::Bool, ::Function, ::WinRPM.Packages{Array{LibExpat.ETree,1}}) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:392
 [7] (::getfield(WinRPM, Symbol("#kw##install")))(::Array{Any,1}, ::typeof(WinRPM.install), ::WinRPM.Packages{Array{LibExpat.ETree,1}}) at .\<missing>:0
 [8] #install#20(::Bool, ::Function, ::Array{String,1}, ::String) at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\WinRPM.jl:368
 [9] (::getfield(WinRPM, Symbol("#kw##install")))(::Array{Any,1}, ::typeof(WinRPM.install), ::Array{String,1}, ::String) at .\<missing>:0 (repeats 2 times)
 [10] (::getfield(WinRPM, Symbol("##36#37")){WinRPM.RPM})() at C:\Users\bernhard.konig\.julia\v0.7\WinRPM\src\winrpm_bindeps.jl:42
 [11] run(::getfield(WinRPM, Symbol("##36#37")){WinRPM.RPM}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\BinDeps.jl:474
 [12] run(::BinDeps.SynchronousStepCollection) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\BinDeps.jl:517
 [13] satisfy!(::BinDeps.LibraryDependency, ::Array{DataType,1}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:937
 [14] satisfy!(::BinDeps.LibraryGroup, ::Array{DataType,1}) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:903
 [15] satisfy!(::BinDeps.LibraryGroup) at C:\Users\bernhard.konig\.julia\v0.7\BinDeps\src\dependencies.jl:867
 [16] top-level scope at .\<missing>:970
 [17] include_relative(::Module, ::String) at .\loading.jl:509
 [18] include at .\sysimg.jl:15 [inlined]
 [19] include(::String) at .\loading.jl:543
 [20] top-level scope
 [21] evalfile(::String, ::Array{String,1}) at .\loading.jl:538
 [22] cd(::getfield(, Symbol("##2#4")){String}, ::String) at .\file.jl:59
 [23] (::getfield(, Symbol("##1#3")))(::IOStream) at .\none:11
 [24] open(::getfield(, Symbol("##1#3")), ::String, ::String) at .\iostream.jl:194
 [25] top-level scope
 [26] eval(::Module, ::Expr) at .\sysimg.jl:60
 [27] process_options(::Base.JLOptions) at .\client.jl:322
 [28] _start() at .\client.jl:406
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:162

ERROR: The system cannot find the file specified.
C:\Users\bernhard.konig\.julia\v0.7\WinRPM\cache\2\noarch%2Fmingw64-libintl8-0.18.1.1-18.13.noarch.cpio



System ERROR:
The system cannot find the file specified.

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Scanning the drive for archives:

===========================================================================================================[ ERROR: Cairo ]============================================================================================================
LoadError: failed process: Process(`'C:\Julia-0.7.X\bin\7z.exe' x -y 'C:\Users\bernhard.konig\.julia\v0.7\WinRPM\cache\2\noarch%2Fmingw64-libintl8-0.18.1.1-18.13.noarch.cpio' '-oC:\Users\bernhard.konig\.julia\v0.7\WinRPM\deps'`, ProcessExited(2)) [2]
in expression starting at C:\Users\bernhard.konig\.julia\v0.7\Cairo\deps\build.jl:162

=======================================================================================================================================================================================================================================
===========================================================================================================[ BUILD ERRORS ]============================================================================================================
WARNING: Cairo had build errors.

 - packages with build errors remain installed in C:\Users\bernhard.konig\.julia\v0.7
 - build the package(s) and all dependencies with `Pkg.build("Cairo")`
 - build a single package by running its `deps/build.jl` script

===========================================================================================================
@amellnik
Copy link

amellnik commented Dec 9, 2017

I incorrectly opened an issue about this against BinDeps here and it's also being discussed for here. WinRPM downloads the file as a .rpm and extracts the .cpio it contains, but the .cpio has a different name -- it's the same but without noarch%2F at the beginning.

@amellnik
Copy link

amellnik commented Dec 9, 2017

So the real mystery here is not why it's not working on 0.7, but how it ever worked on 0.6. Both versions are shelling out as

`'C:\Users\Alex\AppData\Local\Julia-0.7.0-DEV\bin\7z.exe' x -y 
'C:\Users\Alex\.julia\v0.7\WinRPM\cache\2\noarch%2Fmingw64-libwinpthread1-5.0.3-1.2.noarch.cpio' 
'-oC:\Users\Alex\.julia\v0.7\WinRPM\deps'`

and

`'C:\Users\Alex\AppData\Local\Julia-0.6.0\bin\7z.exe' x -y 
'C:\Users\Alex\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libwinpthread1-5.0.3-1.2.noarch.cpio' 
'-oC:\Users\Alex\.julia\v0.6\WinRPM\deps'`

where the name of the cpio file is not correct, but somehow this works on 0.6?

Edit: also calling that exact command from 0.6 fails:

 run(`'C:\Users\Alex\AppData\Local\Julia-0.6.0\bin\7z.exe' x -y 'C:\Users\Alex\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libwinpthread1-5.0.3-1.2.noarch.cpio' '-oC:\Users\Alex\.julia\v0.6\WinRPM\deps'`)

@amellnik
Copy link

amellnik commented Dec 9, 2017

There's also a broader issue related to the downloaded rpm file names having url-escaped characters, while their contents don't. For example, correcting the issue above (with cpio = replace(cpio, "noarch%2F", "")) the ZMQ build gets stuck a few deps later when WinRPM downloads noarch%2Fmingw64-libstdc%2B%2B6-7.2.0-1.2.noarch.rpm and the extracts mingw64-libstdc++6-7.2.0-1.2.noarch.cpio from it.

@RandomString123
Copy link
Contributor

I ran across this in some testing I was doing. I am half way to verifying my theory but this is not an issue with the filename of the extracted file. The code has a reference to that so it does not need the name. The issue is how .7 returns open(cmd) I believe in .6 it would be a tuple of (stream, success_code) in .7 it is just (stream). I hope to finish the testing tomorrow and maybe get a pull request in to fix the issue.

@RandomString123
Copy link
Contributor

RandomString123 commented Feb 16, 2018

Ok, I was wrong with this one. So it looks like 7z.exe behaves differently in .7 than it did in .6. In .6 7z seems to extract files with the same name provided as the source. In .7 it seems as though it is processing it based on what is in the file. The later is most likely correct but breaks WinRPM becuase it assumes the file out is the file in just with a different extension.

I am going to do some research and see if there is a different command line that will get the old and new 7z to behave the same way. If not ... i'm not sure.

> PS C:\Users\ME\AppData\local\Julia-0.7.0-DEV\bin> C:\Users\ME\AppData\Local\Julia-0.6.2\bin\7z.exe x -y 'C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm' '-oC:\Users\ME\.julia\v0.6\WinRPM\cache\2'
> 
> 7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
> 
> Processing archive: C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm
> 
**> Extracting  noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio**
> 
> Everything is Ok
> 
> Size:       82444
> Compressed: 33416
> PS C:\Users\ME\AppData\local\Julia-0.7.0-DEV\bin> C:\Users\ME\AppData\Local\Julia-0.7.0-DEV\bin\7z.exe x -y 'C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm' '-oC:\Users\ME\.julia\v0.6\WinRPM\cache\2'
> 
> 7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
> 
> Scanning the drive for archives:
> 1 file, 33416 bytes (33 KiB)
> 
> Extracting archive: C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm
> 
> Path = C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm
> Type = Rpm
> Physical Size = 33416
> Headers Size = 2620
> CPU = noarch
> Host OS = linux
> Created = 2018-01-04 16:05:48
> 
**> Path = mingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio.lzma**
> Size = 30796
> Created = 2018-01-04 16:05:48
> 
**> Path = mingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio.lzma**
> Type = lzma
> 
> Everything is Ok
> 
> Size:       82444
> Compressed: 33416

@RandomString123
Copy link
Contributor

Hate to double comment, but something even stranger. Just listing the same archive with the different versions results in different outputs.

PS C:\Users\ME\AppData\local\Julia-0.7.0-DEV\bin> C:\Users\ME\AppData\Local\Julia-0.7.0-DEV\bin\7z.exe l -y 'C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm' '-oC:\Users\ME\.julia\v0.6\WinRPM\cache\2'

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Scanning the drive for archives:
1 file, 33416 bytes (33 KiB)

Listing archive: C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm

--
Path = C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm
Type = Rpm
Physical Size = 33416
Headers Size = 2620
CPU = noarch
Host OS = linux
Created = 2018-01-04 16:05:48
----
Path = mingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio.lzma
Size = 30796
Created = 2018-01-04 16:05:48
--
Path = mingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio.lzma
Type = lzma

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
                    .....                            mingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio
------------------- ----- ------------ ------------  ------------------------
                                              33416  1 files
PS C:\Users\ME\AppData\local\Julia-0.7.0-DEV\bin> C:\Users\ME\AppData\Local\Julia-0.6.2\bin\7z.exe l -y 'C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm' '-oC:\Users\ME\.julia\v0.6\WinRPM\cache\2'

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18

Listing archive: C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm

--
Path = C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.rpm
Type = Rpm
----
Size = 30796
--
Path = noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio.lzma
Type = lzma

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
                    .....                     30796  noarch%2Fmingw64-libgcc_s_seh1-7.2.0-3.1.noarch.cpio
------------------- ----- ------------ ------------  ------------------------
                                              30796  1 files, 0 folders

@nalimilan
Copy link
Member

Looks like the handling of %2F has changed in the new 7zip version 0.7 is using? Or maybe 7zip is confused by it, treating it as if it was a path separator? What happens if you remove %2F from the filename (inside the archive and/or in the archive name)?

@RandomString123
Copy link
Contributor

I think it is more than %2F I am guessing that .6 did something weird and used the filename and that .7 uses the name in the archive. See the output below, even when removed and just renamed to noarchming64... it strips off the noarch.

Overall this seems bad for WinRPM as the code makes the assumption that filename(in) -> filename(out). And even for a rewrite of this section of the code for .7 I don't think that assuming that every archive contains a .cpio with the same name as the .rpm is a safe route to to take.

Path = C:\Users\ME\.julia\v0.6\WinRPM\cache\2\noarchmingw64-cpp-7.2.0-3.1.noarch.rpm
Type = Rpm
Physical Size = 6246085
Headers Size = 3046
CPU = noarch
Host OS = linux
Created = 2018-01-04 16:05:48
----
Path = mingw64-cpp-7.2.0-3.1.noarch.cpio.lzma
Size = 6243039
Created = 2018-01-04 16:05:48
--
Path = mingw64-cpp-7.2.0-3.1.noarch.cpio.lzma
Type = lzma

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
                    .....                            mingw64-cpp-7.2.0-3.1.noarch.cpio
------------------- ----- ------------ ------------  ------------------------
                                            6246085  1 files                                  6246085  1 files

@nalimilan
Copy link
Member

I think it is more than %2F I am guessing that .6 did something weird and used the filename and that .7 uses the name in the archive. See the output below, even when removed and just renamed to noarchming64... it strips off the noarch.

OK, but what happens when you rename the file inside the archive to remove %2F?

And even for a rewrite of this section of the code for .7 I don't think that assuming that every archive contains a .cpio with the same name as the .rpm is a safe route to to take.

I think that's an essential and safe assumption for RPM files. Else, how would you find out what's the name of the file you need?

@RandomString123
Copy link
Contributor

There is no %2F inside the archive. the .6 version of 7zip is adding it to the filename upon extraction. Here is the archive viewed using the desktop version of 7zip.
image

We can make the assumption about the filename and see if things break. I think the correct way to do it would be to use a library to extract the file list and use it to get the filename. However I don't think there is any such library for Julia&RPMs.

@nalimilan
Copy link
Member

Ah, OK. Then I guess the bug is actually on the 0.6 7zip: it erroneously adds noarch2%F because it's treated as a directory prefix somewhere. We should just adapt the code on 0.7 to use the file without that bogus prefix, which AFAICT doesn't add anything.

@RandomString123
Copy link
Contributor

RandomString123 commented Feb 16, 2018

Correct. I'll make that change and fence it off for .7 and submit a PR for it. ... under the assumption it works.

@nalimilan
Copy link
Member

See also #137.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants