Pkg.add("ParallelAccelerator") results in dirty state

[tkelman]

Due to one of the files having executable permissions in git, but something wrong happening with the libgit2 clone or checkout. The executable permissions are removed in the working copy, and strangely that change is staged? The package being dirty is very bad, as it means Pkg won't upgrade or remove it unless you go in and manually reset the change.

~/julianightly> rm -rf pkgtmp; JULIA_PKGDIR=$PWD/pkgtemp bin/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-pre+5654 (2016-07-24 09:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit aa2e898 (1 day old master)
|__/                   |  x86_64-unknown-linux-gnu

julia> Pkg.init()
INFO: Initializing package repository /global/homes/t/tkelman/julianightly/pkgtemp/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl

julia> Pkg.add("Images", v"0.5.5", v"0.5.5+")
INFO: Cloning cache of ColorTypes from https://github.com/JuliaGraphics/ColorTypes.jl.git
INFO: Cloning cache of ColorVectorSpace from https://github.com/JuliaGraphics/ColorVectorSpace.jl.git
INFO: Cloning cache of Colors from https://github.com/JuliaGraphics/Colors.jl.git
INFO: Cloning cache of Compat from https://github.com/JuliaLang/Compat.jl.git
INFO: Cloning cache of FileIO from https://github.com/JuliaIO/FileIO.jl.git
INFO: Cloning cache of FixedPointNumbers from https://github.com/JeffBezanson/FixedPointNumbers.jl.git
INFO: Cloning cache of Graphics from https://github.com/JuliaGraphics/Graphics.jl.git
INFO: Cloning cache of Images from https://github.com/timholy/Images.jl.git
INFO: Cloning cache of Reexport from https://github.com/simonster/Reexport.jl.git
INFO: Cloning cache of SIUnits from https://github.com/Keno/SIUnits.jl.git
INFO: Cloning cache of TexExtensions from https://github.com/Keno/TexExtensions.jl.git
INFO: Cloning cache of Zlib from https://github.com/dcjones/Zlib.jl.git
INFO: Installing ColorTypes v0.2.5
INFO: Installing ColorVectorSpace v0.1.5
INFO: Installing Colors v0.6.6
INFO: Installing Compat v0.8.6
INFO: Installing FileIO v0.1.0
INFO: Installing FixedPointNumbers v0.1.4
INFO: Installing Graphics v0.1.3
INFO: Installing Images v0.5.5
INFO: Installing Reexport v0.0.3
INFO: Installing SIUnits v0.0.6
INFO: Installing TexExtensions v0.0.3
INFO: Installing Zlib v0.1.12
INFO: Package database updated

julia> Pkg.status()
1 required packages:
 - Images                        0.5.5              8840a952 (dirty)
11 additional packages:
 - ColorTypes                    0.2.5
 - ColorVectorSpace              0.1.5
 - Colors                        0.6.6
 - Compat                        0.8.6
 - FileIO                        0.1.0
 - FixedPointNumbers             0.1.4
 - Graphics                      0.1.3
 - Reexport                      0.0.3
 - SIUnits                       0.0.6
 - TexExtensions                 0.0.3
 - Zlib                          0.1.12

julia> exit()
~/julianightly> cd pkgtemp/v0.5/Images/
~/julianightly/pkgtemp/v0.5/Images> git status
Not currently on any branch.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   test/map.jl

~/julianightly/pkgtemp/v0.5/Images> ls -al test
total 8
drwxr-x--- 2 tkelman tkelman  4096 Jul 25 08:54 .
drwxr-x--- 6 tkelman tkelman   512 Jul 25 08:54 ..
-rw-r----- 1 tkelman tkelman    85 Jul 25 08:54 REQUIRE
-rwxr-x--- 1 tkelman tkelman 25276 Jul 25 08:54 algorithms.jl
-rw-r----- 1 tkelman tkelman 18441 Jul 25 08:54 core.jl
-rw-r----- 1 tkelman tkelman  1337 Jul 25 08:54 corner.jl
-rw-r----- 1 tkelman tkelman   463 Jul 25 08:54 distances.jl
-rw-r----- 1 tkelman tkelman 18633 Jul 25 08:54 edge.jl
-rw-r----- 1 tkelman tkelman 13339 Jul 25 08:54 map.jl
-rw-r----- 1 tkelman tkelman  3362 Jul 25 08:54 overlays.jl
-rw-r----- 1 tkelman tkelman   729 Jul 25 08:54 parallel.jl
-rw-r----- 1 tkelman tkelman   631 Jul 25 08:54 restrict.jl
-rw-r----- 1 tkelman tkelman   433 Jul 25 08:54 runtests.jl
-rw-r----- 1 tkelman tkelman  1657 Jul 25 08:54 writemime.jl

[timholy]

I've noticed the executable bit keeps getting set on some files. Does this show up in GitHub PRs and I'm just not noticing? (Very likely.) Or is it from someone pushing straight to master?

Of relevance for using this as a test case, I just unset the executable flag in all Images.jl files (sorry). I'm tempted to use git config core.fileMode false, but http://stackoverflow.com/questions/1580596/how-do-i-make-git-ignore-file-mode-chmod-changes warns that it's not great practice. In this case I can't really see any problem, though.

[tkelman]

It does show up in PR's, ref https://github.com/timholy/Images.jl/pull/516/files#diff-af28e1ea7bb1f554f483a8671aab187b - so if you make a new tag this might not be the best test case any more, but we could do Pkg.add("Images", v"0.5.5", v"0.5.5+") for testing. Or commit something in a smaller package like Example.jl for testing purposes.

[timholy]

I am on the verge of tagging a new Images, but as you say the previous version should work as a test case.

[tkelman]

@wildart I suspect this is a libgit2 bug with executable permissions.

[wildart]

I'm not sure, but I use default settings for checked out files:

julia/base/libgit2/types.jl

Lines 109 to 110 in c389aa3

	dir_mode::Cuint = Cuint(0), # Cuint(0o755),
	file_mode::Cuint = Cuint(0), #Cuint(0o644),

[tkelman]

How do we track down where in either libgit2 at the C level or in our bindings this is going wrong? Having packages not be updatable or removable (without manual fixes first) if they have any executable files in them would be pretty bad.

[wildart]

test\map.jl has executable bit set in v0.5.5, but unset on master.

I wonder if package caching of introduces this error. Beause repo first cached and then checked out from this cache. Maybe during this transition executable bit is lost.

[tkelman]

yeah we can put in place fixes for packages where we know this is happening if they don't actually need the executable bits, but for miscellaneous scripts etc it may help to have the ability to do so without things going wrong. not a bad theory that it might be cache (bare clone) related.

[wildart]

Cache is not a problem. Apparently, something funny is going when checking out particular commit. Because another file test\algorithms.jl has execution bit set as well, and it's correctly checked out, but not test\map.jl.

[timholy]

As I've made additional commits I've unset executable bits in Images, so it might be me doing the unsetting. I am sure I am not doing the setting, but I am not the only one contributing to that repo.