libkhmer-dev debian package? #885

kdm9 · 2015-03-30T02:17:36Z

(Was going to write this as an email, but figured github is more collaborative 😄)

Now that #788 is merged, what's you opinion on adding libkhmer-dev as a package output of the khmer source debian package. I know you've been working on packaging for debian, and I'm happy to help out.

I know this somewhat flies in the face of it being the "unsupported" cpp library, so there may be issues from the debian folks not wanting to include it for that reason.

Thoughts/opinions?

Cheers,
Kevin

TODO:

The text was updated successfully, but these errors were encountered:

mr-c · 2015-03-30T09:55:52Z

Sure!

Debian packaging is managed at http://anonscm.debian.org/cgit/debian-med/khmer.git via the Debian Med team http://debian-med.alioth.debian.org/docs/policy.html

The Debian package for khmer v1.3 is in the 'NEW' queue https://ftp-master.debian.org/new.html.

If it isn't there already we can add a note to the README.Debian about the semantic versioning agreement (and what isn't covered).

kdm9 · 2015-05-09T15:46:59Z

About this @mr-c, I'm starting on Vince Buffalo's scythe, which (working with Andreas Tille and others) should give me all I need to do this up to Debian Med's standards. I'll work on the libkhmer-dev package once I've polished out scythe. So it's a work in progress still 😄

mr-c · 2015-05-09T18:01:55Z

Sweet, welcome to the Debian Med team!

kdm9 · 2015-05-10T09:13:50Z

I've got access to git.debian.org, and made a branch, so my work will appear in the daube/libkhmer-dev branch.

I won't push to master.

kdm9 · 2015-07-05T09:38:11Z

@mr-c, I've started work on this again. It would be great if at some point when you had time we could discuss this on gitter (preferably live). I have a few questions around the patch queue and debian-ising lib/Makefile.

mr-c · 2015-07-05T10:19:03Z

Sweet. I've re-opened Gitter on my laptop & installed the client on my phone. I'm in the UCT+01:00 timezone until July 12th.

As for the packaging, I'm going to cut a release candidate of 2.0 today; lets use that for now.

kdm9 · 2015-07-06T07:05:35Z

Brainfart checklist, which will probably be added to:

we should set SONAME in the shared object
makefile flags USE_SYSTEM_{ZLIB,LIBBZ2,SEQAN} would be helpful in lib/makefile, and possibly also the root makefile
We should ensure that lib/Makefile accepts the dpkg-buildflags environment variables.
should we prefix the installed headers somehow, to avoid file name clashes under /usr/include?

mr-c · 2015-07-06T08:18:54Z

Yes, lets put the headers in oxli/something.hh.
When compiling the CPython module you can use system zlib & bz2 by tweaking setup.cfg
Stock seqan does not yet work with us (we had to patch 1.4.x and 2.x has issues)

kdm9 · 2015-07-06T08:46:23Z

#1148 contains the upstreamed changes

kdm9 · 2015-07-07T03:53:22Z

Today's questions:

Should make install copy/append lib/magic somewhere? Or shall we leave that to dh_override_auto_install in d/rules?
Debug this jenkins failure (on Debian) link. appears to be related to the header install path.
Should we jump straight to calling this package liboxli (or liboxli++ even)?

kdm9 · 2015-07-07T09:13:43Z

Known issues w/ current debian package (see here)

Need to package pkgconfig file
Change shared lib package to liboxli2, as that's what it will be when we release it.
Need to package lib/magic

There'll be more, I suggest we add to this list as we find them

mr-c · 2015-07-07T09:17:00Z

Re: lib/magic Let's just solve this for Debian via debian/rules. /etc/magic is from the file package; /usr/share/doc/file/README.Debian tells us how to install our own magic as package maintainers.

mr-c · 2015-07-07T09:18:31Z

I've install zlib1g-dev on ci.oxli.org

mr-c · 2015-07-07T09:21:13Z

+1 for liboxli & python3-khmer

mr-c · 2015-07-07T10:05:27Z

You can make a README.Debian for the lib package for now, should be upstreamed to khmer later.

kdm9 · 2015-07-07T10:53:16Z

Todo:

Readme.debian for liboxli* packages
Check the magic installation and removal works
Is it worth having a base package,khmer-common, for the magics? And having all packages depend on that? (from commit message of debian repo)

mr-c · 2015-07-07T11:02:08Z

magic file can go in oxli-common, yep

On Tue, Jul 7, 2015 at 11:53 AM Kevin Murray notifications@github.com
wrote:

Todo:

Readme.debian for liboxli* packages

Check the magic installation and removal works

Is it worth having a base package,khmer-common, for the magics? And
having all packages depend on that? (from commit message of debian repo)

—
Reply to this email directly or view it on GitHub
#885 (comment).

Michael R. Crusoe: Programmer & Bioinformatician crusoe@ucdavis.edu
The lab for Data Intensive Biology; University of California, Davis
https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

kdm9 · 2015-07-31T06:34:27Z

Another checklist!

@mr-c to force-push to master from big-rebase. on git.debian.org
import 2.0rc2 tarball (and push --all / push --tags 😄)
pop all the lintian warnings

mr-c · 2015-08-13T21:43:06Z

~~- [ ] install oxli into /usr/bin/~~

kdm9 · 2015-08-27T01:35:18Z

@mr-c do we still need oxli in /usr/bin? I though you'd nuked it for 2.0?

kdm9 · 2015-08-27T01:59:39Z

Manually ran gbp buildpackage in a fresh debian:unstable docker container. Got a few strange test failures, but this could be do to with the docker container running as root:

PYBUILD_SYSTEM=custom \
    PYBUILD_TEST_ARGS="cd {dir}; export PATH={destdir}/usr/lib/khmer/bin:$PATH ; \
    cd {build_dir}; {interpreter} -m nose --attr \
    '!known_failing,!jenkins,!huge'" dh_auto_test
    pybuild --test --test-nose -i python{version} -p 3.4 --dir .
I: pybuild base:170: cd /khmer; export PATH=/khmer/debian/khmer/usr/lib/khmer/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 ;     cd /khmer/.pybuild/pythonX.Y_3.4/build; python3.4 -m nose --attr     '!known_failing,!jenkins,!huge'
.....................................................................................................................................................................................................F........................................................................................................................................F...........F.F...SSSSSSSSSS........................F........................................................................................................................................................................................................
======================================================================
FAIL: khmer.tests.test_labelhash.test_save_fail_readonly
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_labelhash.py", line 157, in test_save_fail_readonly
    assert 0, "this should fail: read-only file"
AssertionError: this should fail: read-only file

======================================================================
FAIL: khmer.tests.test_read_parsers.test_num_reads_truncated
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 88, in test_num_reads_truncated
    for read in rparser:
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 91, in test_num_reads_truncated
    assert "Sequence is empty" in str(err), str(err)
AssertionError

======================================================================
FAIL: khmer.tests.test_read_parsers.test_read_truncated
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 275, in test_read_truncated
    for read in rparser:
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 279, in test_read_truncated
    assert "Sequence is empty" in str(err), str(err)
AssertionError

======================================================================
FAIL: khmer.tests.test_read_parsers.test_read_pair_iterator_in_error_mode_xfail
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 334, in test_read_pair_iterator_in_error_mode_xfail
    for rpair in rparser.iter_read_pairs():
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 338, in test_read_pair_iterator_in_error_mode_xfail
    assert "Invalid read pair" in str(exc), str(exc)
AssertionError

======================================================================
FAIL: khmer.tests.test_scripts.test_load_into_counting_nonwritable
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_scripts.py", line 192, in test_load_into_counting_nonwritable
    assert 'does not have write permission; exiting' in err, err
nose.proxy.AssertionError: 
|| This is the script load-into-counting.py in khmer.
|| You are running khmer version 2.0-rc3
|| You are also using screed version 0.9
||
|| If you use this script in a publication, please cite EACH of the following:
||
||   * MR Crusoe et al., 2014. http://dx.doi.org/10.6084/m9.figshare.979190
||   * Q Zhang et al., http://dx.doi.org/10.1371/journal.pone.0101271
||   * A. Döring et al. http://dx.doi.org:80/10.1186/1471-2105-9-11
||
|| Please see http://khmer.readthedocs.org/en/latest/citations.html for details.


PARAMETERS:
 - kmer size =    20        (-k)
 - n tables =     2         (-N)
 - max tablesize = 1e+03    (-x)

Estimated memory usage is 2e+03 bytes (n_tables x max_tablesize)
--------
Saving k-mer countgraph to /tmp/khmertest_z0n4rlak/test-nonwritable
Loading kmers from sequences in ['/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test-data/test-abund-read-2.fa']
making countgraph
consuming input /khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test-data/test-abund-read-2.fa
Total number of unique k-mers: 94
saving /tmp/khmertest_z0n4rlak/test-nonwritable
fp rate estimated to be 0.008
DONE.
wrote to: /tmp/khmertest_z0n4rlak/test-nonwritable.info

-------------------- >> begin captured stdout << ---------------------
running: load-into-counting.py in: /khmer/.pybuild/pythonX.Y_3.4/build
arguments ['load-into-counting.py', '-x', '1e3', '-N', '2', '-k', '20', '/tmp/khmertest_z0n4rlak/test-nonwritable', '/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test-data/test-abund-read-2.fa']

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 587 tests in 45.964s

FAILED (SKIP=10, failures=5)
E: pybuild pybuild:262: test: plugin custom failed with: exit code=1: cd /khmer; export PATH=/khmer/debian/khmer/usr/lib/khmer/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 ;     cd /khmer/.pybuild/pythonX.Y_3.4/build; python3.4 -m nose --attr     '!known_failing,!jenkins,!huge'
dh_auto_test: pybuild --test --test-nose -i python{version} -p 3.4 --dir . returned exit code 13

The build itself worked fine.

mr-c · 2015-08-27T05:42:31Z

Indeed, we don't need oxli in /usr/bin at this time.

mr-c · 2015-08-27T05:43:40Z

Yeah, don't build as root :-)

kdm9 · 2015-08-27T06:37:04Z

Should we get rid of the ruby-ronn builddep by hand-crafting (or just git add-ing) the manpage?

kdm9 · 2015-08-27T08:17:20Z

Another docker build, this time with an unprivileged user. Looks like a similar bug to the OSX lurgy in #1190, #1046 et al.

PYBUILD_SYSTEM=custom \
    PYBUILD_TEST_ARGS="cd {dir}; export PATH={destdir}/usr/lib/khmer/bin:$PATH ; \
    cd {build_dir}; {interpreter} -m nose --attr \
    '!known_failing,!jenkins,!huge'" dh_auto_test
    pybuild --test --test-nose -i python{version} -p 3.4 --dir .
I: pybuild base:170: cd /home/khmer/khmer; export PATH=/home/khmer/khmer/debian/khmer/usr/lib/khmer/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 ;     cd /home/khmer/khmer/.pybuild/pythonX.Y_3.4/build; python3.4 -m nose --attr     '!known_failing,!jenkins,!huge'
..............................................................................................................................................................................................................................................................................................................................................F...........F.F...SSSSSSSSSS.................................................................................................................................................................................................................................
======================================================================
FAIL: khmer.tests.test_read_parsers.test_num_reads_truncated
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 88, in test_num_reads_truncated
    for read in rparser:
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 91, in test_num_reads_truncated
    assert "Sequence is empty" in str(err), str(err)
AssertionError

======================================================================
FAIL: khmer.tests.test_read_parsers.test_read_truncated
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 275, in test_read_truncated
    for read in rparser:
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 279, in test_read_truncated
    assert "Sequence is empty" in str(err), str(err)
AssertionError

======================================================================
FAIL: khmer.tests.test_read_parsers.test_read_pair_iterator_in_error_mode_xfail
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 334, in test_read_pair_iterator_in_error_mode_xfail
    for rpair in rparser.iter_read_pairs():
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/khmer/khmer/.pybuild/pythonX.Y_3.4/build/khmer/tests/test_read_parsers.py", line 338, in test_read_pair_iterator_in_error_mode_xfail
    assert "Invalid read pair" in str(exc), str(exc)
AssertionError

----------------------------------------------------------------------
Ran 587 tests in 42.286s

FAILED (SKIP=10, failures=3)
E: pybuild pybuild:262: test: plugin custom failed with: exit code=1: cd /home/khmer/khmer; export PATH=/home/khmer/khmer/debian/khmer/usr/lib/khmer/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 ;     cd /home/khmer/khmer/.pybuild/pythonX.Y_3.4/build; python3.4 -m nose --attr     '!known_failing,!jenkins,!huge'
dh_auto_test: pybuild --test --test-nose -i python{version} -p 3.4 --dir . returned exit code 13
debian/rules:25: recipe for target 'override_dh_auto_test' failed

mr-c · 2015-08-27T08:27:01Z

Yeah, I've been getting that in Debian testing and unstable for a while now.

mr-c · 2015-08-27T17:06:47Z

I'm okay with the ruby-ronn build-dep.

kdm9 · 2016-12-03T23:46:22Z

Closing this, as we now have liboxli-dev (w00t)

kdm9 added the question label Mar 30, 2015

mr-c added enhancement and removed question labels Mar 30, 2015

mr-c mentioned this issue May 8, 2015

produce and distribute khmer via a RPM and a DEB package #170

Closed

kdm9 mentioned this issue Jul 6, 2015

-DVERSION should be renamed -DOXLI_VERSION or similar #1149

Open

mr-c added this to the 2.0 milestone Jul 30, 2015

kdm9 mentioned this issue Aug 27, 2015

Document OS X Python bug and submit a minimal test case #1190

Closed

2 tasks

standage modified the milestones: unscheduled, 2.0 Nov 15, 2016

kdm9 closed this as completed Dec 3, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libkhmer-dev debian package? #885

libkhmer-dev debian package? #885

kdm9 commented Mar 30, 2015

mr-c commented Mar 30, 2015

kdm9 commented May 9, 2015

mr-c commented May 9, 2015

kdm9 commented May 10, 2015

kdm9 commented Jul 5, 2015

mr-c commented Jul 5, 2015

kdm9 commented Jul 6, 2015

mr-c commented Jul 6, 2015

kdm9 commented Jul 6, 2015

kdm9 commented Jul 7, 2015

kdm9 commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

kdm9 commented Jul 7, 2015

mr-c commented Jul 7, 2015

kdm9 commented Jul 31, 2015

mr-c commented Aug 13, 2015

kdm9 commented Aug 27, 2015

kdm9 commented Aug 27, 2015

mr-c commented Aug 27, 2015

mr-c commented Aug 27, 2015

kdm9 commented Aug 27, 2015

kdm9 commented Aug 27, 2015

mr-c commented Aug 27, 2015

mr-c commented Aug 27, 2015

kdm9 commented Dec 3, 2016

libkhmer-dev debian package? #885

libkhmer-dev debian package? #885

Comments

kdm9 commented Mar 30, 2015

TODO:

mr-c commented Mar 30, 2015

kdm9 commented May 9, 2015

mr-c commented May 9, 2015

kdm9 commented May 10, 2015

kdm9 commented Jul 5, 2015

mr-c commented Jul 5, 2015

kdm9 commented Jul 6, 2015

mr-c commented Jul 6, 2015

kdm9 commented Jul 6, 2015

kdm9 commented Jul 7, 2015

kdm9 commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

mr-c commented Jul 7, 2015

kdm9 commented Jul 7, 2015

mr-c commented Jul 7, 2015

kdm9 commented Jul 31, 2015

mr-c commented Aug 13, 2015

kdm9 commented Aug 27, 2015

kdm9 commented Aug 27, 2015

mr-c commented Aug 27, 2015

mr-c commented Aug 27, 2015

kdm9 commented Aug 27, 2015

kdm9 commented Aug 27, 2015

mr-c commented Aug 27, 2015

mr-c commented Aug 27, 2015

kdm9 commented Dec 3, 2016