v2-sdist includes a file twice in the tarball

[rudymatela]

Describe the bug
When running v2-sdist in the latest version of Speculate, one of the benchmark (example) files is included twice in the generated tarball.

To Reproduce
Steps to reproduce the behavior:

$ wget https://hackage.haskell.org/package/speculate-0.4.1/speculate-0.4.1.tar.gz
...
2020-03-23 19:55:10 (200 KB/s) - ‘speculate-0.4.1.tar.gz’ saved [101631/101631]

$ tar -xzf speculate-0.4.1.tar.gz 

$ cd speculate-0.4.1/

$ cabal v2-sdist 
Wrote tarball sdist to
./dist-newstyle/sdist/speculate-0.4.1.tar.gz

$ tar -tf dist-newstyle/sdist/speculate-0.4.1.tar.gz | grep eg/plus-abs.hs
speculate-0.4.1/eg/plus-abs.hs
speculate-0.4.1/eg/plus-abs.hs

The plus-abs example is included twice in the tarball! o.O

Expected behavior
I would expected for the generated tarball to not have repeated files.

System information

• Arch Linux (2020-03-09)
• cabal version 3.0.0.0 (compiled using version 3.0.1.0 of the cabal library)
• ghc version 8.8.3

Proof:

$ cat /var/log/pacman.log | grep "pacman.*u" | tail -1
[2020-03-09T15:41:47-0300] [PACMAN] Running 'pacman -Su'

$ cabal --version
cabal-install version 3.0.0.0
compiled using version 3.0.1.0 of the Cabal library 

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.3

Additional context
I have a test on Speculate's Makefile that assures that cabal included all files tracked by git and only the files tracked by git (in case I forget to reference them on the cabal file). This ended up breaking my test.

[rudymatela]

Some further info:

This bug is not present in v1-sdist. See:

$ tar -xf speculate-0.4.1.tar.gz 

$ cd speculate-0.4.1/

$ cabal v1-sdist 
Warning: Cannot run preprocessors. Run 'configure' command first.
Building source dist for speculate-0.4.1...
Source tarball created: dist/speculate-0.4.1.tar.gz

$ tar -tf dist/speculate-0.4.1.tar.gz | grep "eg/plus-abs.hs"
speculate-0.4.1/eg/plus-abs.hs

(Incidentally, I prefer that here the location of the source tarball is given relative to my current working directory. But I digress.)

[rudymatela]

Alternative steps to reproduce the bug, i.e. how I found it:

$ git clone https://github.com/rudymatela/speculate
...

$ cd speculate

$ git checkout 8a8f662
...

$ make test-sdist
./test/sdist
+ cabal sdist
Wrote tarball sdist to
dist-newstyle/sdist/speculate-0.4.1.tar.gz
++ find dist-newstyle -name speculate-0.4.1.tar.gz
+ pkg=dist-newstyle/sdist/speculate-0.4.1.tar.gz
++ mktemp -d /tmp/test-sdist-XXXXXXXXXX
+ tmp=/tmp/test-sdist-XHaufrs8IQ
+ tar -tf dist-newstyle/sdist/speculate-0.4.1.tar.gz
+ sort --ignore-case
+ tar -tf dist-newstyle/sdist/speculate-0.4.1.tar.gz
+ sort --ignore-case --unique
+ diff -rud /tmp/test-sdist-XHaufrs8IQ/ls-cabal-i /tmp/test-sdist-XHaufrs8IQ/ls-cabal-iu
--- /tmp/test-sdist-XHaufrs8IQ/ls-cabal-i	2020-03-23 20:18:30.694629795 -0300
+++ /tmp/test-sdist-XHaufrs8IQ/ls-cabal-iu	2020-03-23 20:18:30.701296478 -0300
@@ -86,7 +86,6 @@
 speculate-0.4.1/eg/monad.hs
 speculate-0.4.1/eg/oddeven.hs
 speculate-0.4.1/eg/plus-abs.hs
-speculate-0.4.1/eg/plus-abs.hs
 speculate-0.4.1/eg/pretty-compact.hs
 speculate-0.4.1/eg/pretty.hs
 speculate-0.4.1/eg/ratio.hs
+ rm -r /tmp/test-sdist-XHaufrs8IQ

[phadej]

This is because you have

benchmark plus-abs
  main-is:           plus-abs.hs

and

extra-source-files:
                  ...            
                  , eg/*.hs

and even we have nub it doesn't work because the files are included with different filepermission

-rw-r--r-- 0/0             409 2001-09-09 04:46 speculate-0.4.1/eg/plus-abs.hs
-rwxr-xr-x 0/0             409 2001-09-09 04:46 speculate-0.4.1/eg/plus-abs.hs

---

I think this will be fixed as I get to do something about
#6454

At least I'll verify that nub actually does what it's expected to do.

(v1-sdist does less magic with permissions, and I think so will v2-sdist "soon").

[phadej]

And as a separate comment. Please avoid posting too many comments at once. I get a mail for every each of them. Makes me want to mark some issues as spam.

EDIT: or alternatively just ignore any issues I don't run myself into.

[phadej]

Fixed by #6666