[WIP] added a silhouette calculation #876

lldelisle · 2019-09-13T10:13:59Z

Welcome to deepTools GitHub repository! Please check the following regarding
your pull request :

Does the PR contain new feature?
Does the PR contain bugfix?
Does the PR contain documentation changes?
Does the PR contain changes to the galaxy wrapper?

Hi,
This is a beginning of a PR that I would like to discuss.
What I did:

add the --silhouette arguments in heatmapperOptionalArgs.
when this option is put, when hmcluster is launched, the silhouette is evaluated with silhouette_samples from sklearn. It is stored in a new attribute of the _matrix class.
In the std err the mean of the silhouette is output.
in the save_BED method, it will always print an additional column with the silhouette values (initialized to nan).

What is missing:

tests

What I would like to discuss:

This will change the bed output in all cases (even if --silhouette is not put). I think it would be better to change it.

For the moment silhouette is a new attribute of _matrix and I agree it would make more sense in regions but then I am wondering if regions should not be a class to be sure of what is what in this array. For me it looks like, the region is defined from

deepTools/deeptools/mapReduce.py

Lines 123 to 126 in df8536a

    
           if includeLabels: 
        
               bed_regions_list = [[chrom, x[4], x[2], x[3], x[5], x[6]] for x in bed_interval_tree.findOverlaps(chrom, reg[0], reg[1], trimOverlap=True, numericGroups=True, includeStrand=True)] 
        
           else: 
        
               bed_regions_list = [[chrom, x[4], x[5], x[6]] for x in bed_interval_tree.findOverlaps(chrom, reg[0], reg[1], trimOverlap=True, includeStrand=True)]

But this is only when a bed is provided, I do not know which are the other cases.

Should we plot the values? A summary per cluster of the values?

Thanks

* Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826)

File is removed upon clean.

* copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number

…into addSilhouette

lldelisle · 2019-12-13T09:41:36Z

Hi,
I think the code is not working... but I can propose a solution, do you want me to commit on this branch?

dpryan79 · 2019-12-13T09:43:20Z

I'm fixing it locally and will update the PR.

lldelisle · 2019-12-13T09:52:47Z

ok perfect

dpryan79 · 2019-12-19T07:10:42Z

@lldelisle I think this should be producing more reasonable results again. Can you double check that with your own data?

dpryan79 · 2020-01-14T12:29:52Z

@lldelisle Have you had a chance to test the current PR by chance?

* copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * Release 3.3.1 (#873) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Actionable active actions acting actively (#874) * give actions another try * wrong docs? * ok * hmm * WTF * ah, we CAN give a path * hmm * actions everywhere * foo * artifacts * fix #889 (#891) * Fix888 (#892) * fix x-axis profile tick positions * set minimum matplotlib version to 3.1.0 * fix hexbin and overlapped_lines too * fix #887 (#893) * update change log * Seaborn colormaps (#894) * add seaborn colormaps * bump version and finally change license * indenting * update colormaps in galaxy wrapper * update version in galaxy wrapper * changelog * wrong issue number * pep8 * pep8 * pep8 * pep8 * added clusterUsingSamples to heatmap * Added a couple of assertions to cehck the range of samples' indices * Using --xRange and --yRange fails in galaxy due to the single quote. … (#901) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Using --xRange and --yRange fails in galaxy due to the single quote. Removed them. * Try just changing the wrapper * fix wrapper linting * plotCorrelation wrapper works properly now * Add separate linting step to catch some of this in the future Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * Documentation fixes, closes #886 (#905) * Fix #902 (#906) * [WIP] added a silhouette calculation (#876) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * added a silhouette calculation * remove sklearn, implement with scipy and numpy * Update requirements.txt * Update setup.py * Update heatmapper.py * update galaxy wrapper * Fix run time issues * refactor, the order matters here. * removing debugging stuff * Update heatmapper.py * Update plotHeatmap.py Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * mention silhouette score * update help location * see if this fixes things (#909) Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Ann Loraine <aloraine@uncc.edu> Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com> Co-authored-by: Ömer An <bounlu@gmail.com> Co-authored-by: LeilyR <leila.rabbani@gmail.com> Co-authored-by: cgirardot <girardot@embl.de> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> Co-authored-by: Lucille Delisle <lucille.delisle@epfl.ch>

* copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * Release 3.3.1 (#873) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Actionable active actions acting actively (#874) * give actions another try * wrong docs? * ok * hmm * WTF * ah, we CAN give a path * hmm * actions everywhere * foo * artifacts * fix #889 (#891) * Fix888 (#892) * fix x-axis profile tick positions * set minimum matplotlib version to 3.1.0 * fix hexbin and overlapped_lines too * fix #887 (#893) * update change log * Seaborn colormaps (#894) * add seaborn colormaps * bump version and finally change license * indenting * update colormaps in galaxy wrapper * update version in galaxy wrapper * changelog * wrong issue number * pep8 * pep8 * pep8 * pep8 * added clusterUsingSamples to heatmap * Added a couple of assertions to cehck the range of samples' indices * Using --xRange and --yRange fails in galaxy due to the single quote. … (#901) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Using --xRange and --yRange fails in galaxy due to the single quote. Removed them. * Try just changing the wrapper * fix wrapper linting * plotCorrelation wrapper works properly now * Add separate linting step to catch some of this in the future Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * Documentation fixes, closes #886 (#905) * Fix #902 (#906) * [WIP] added a silhouette calculation (#876) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * added a silhouette calculation * remove sklearn, implement with scipy and numpy * Update requirements.txt * Update setup.py * Update heatmapper.py * update galaxy wrapper * Fix run time issues * refactor, the order matters here. * removing debugging stuff * Update heatmapper.py * Update plotHeatmap.py Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * mention silhouette score * update help location * see if this fixes things (#909) * update Azure OSX client version * Fix typos in documentation (#916) Fixes two typos in example code. * copy wrapper fixes from Bjoern's PR * Fix dotted line (#921) * Fix the dashed line in plotHeatmap with reference-point TES and sorting by region_length * fix test * Implement #924 (#925) * Basic implementation of --linesAtTickMarks and galaxy wrapper * bump version to 3.4.0 * typo * stupid eigenvalues Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: A. Loraine <aloraine@uncc.edu> Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com> Co-authored-by: Ömer An <bounlu@gmail.com> Co-authored-by: LeilyR <leila.rabbani@gmail.com> Co-authored-by: cgirardot <girardot@embl.de> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> Co-authored-by: Lucille Delisle <lucille.delisle@epfl.ch> Co-authored-by: Sichong <scpeng@ucdavis.edu>

* copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * Release 3.3.1 (#873) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Actionable active actions acting actively (#874) * give actions another try * wrong docs? * ok * hmm * WTF * ah, we CAN give a path * hmm * actions everywhere * foo * artifacts * fix #889 (#891) * Fix888 (#892) * fix x-axis profile tick positions * set minimum matplotlib version to 3.1.0 * fix hexbin and overlapped_lines too * fix #887 (#893) * update change log * Seaborn colormaps (#894) * add seaborn colormaps * bump version and finally change license * indenting * update colormaps in galaxy wrapper * update version in galaxy wrapper * changelog * wrong issue number * pep8 * pep8 * pep8 * pep8 * added clusterUsingSamples to heatmap * Added a couple of assertions to cehck the range of samples' indices * Using --xRange and --yRange fails in galaxy due to the single quote. … (#901) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * Using --xRange and --yRange fails in galaxy due to the single quote. Removed them. * Try just changing the wrapper * fix wrapper linting * plotCorrelation wrapper works properly now * Add separate linting step to catch some of this in the future Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * Documentation fixes, closes #886 (#905) * Fix #902 (#906) * [WIP] added a silhouette calculation (#876) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Release 3.3.1 (#872) * copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number * added a silhouette calculation * remove sklearn, implement with scipy and numpy * Update requirements.txt * Update setup.py * Update heatmapper.py * update galaxy wrapper * Fix run time issues * refactor, the order matters here. * removing debugging stuff * Update heatmapper.py * Update plotHeatmap.py Co-authored-by: Devon Ryan <dpryan79@users.noreply.github.com> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> * mention silhouette score * update help location * see if this fixes things (#909) * update Azure OSX client version * Fix typos in documentation (#916) Fixes two typos in example code. * copy wrapper fixes from Bjoern's PR * Fix dotted line (#921) * Fix the dashed line in plotHeatmap with reference-point TES and sorting by region_length * fix test * Implement #924 (#925) * Basic implementation of --linesAtTickMarks and galaxy wrapper * bump version to 3.4.0 * typo * stupid eigenvalues * Fix #928 (#929) * Don't force shared memory. Co-authored-by: Björn Grüning <bjoern.gruening@gmail.com> Co-authored-by: Ann Loraine <aloraine@uncc.edu> Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com> Co-authored-by: bounlu <bounlu@gmail.com> Co-authored-by: Leily Rabbani <rabbani@pc390.ie-freiburg.mpg.de> Co-authored-by: LeilyR <leila.rabbani@gmail.com> Co-authored-by: cgirardot <girardot@embl.de> Co-authored-by: Björn Grüning <bjoern@gruenings.eu> Co-authored-by: Steffen Möller <steffen_moeller@gmx.de> Co-authored-by: Lucille Delisle <lucille.delisle@epfl.ch> Co-authored-by: Sichong <scpeng@ucdavis.edu>

dpryan79 and others added 7 commits April 1, 2019 22:56

Merge branch 'release/3.3.0'

507702f

fixes linting issues (#837)

9092b83

Delete #test.bg# (#859)

76dbf59

File is removed upon clean.

added a silhouette calculation

11b064f

remove sklearn, implement with scipy and numpy

10544eb

dpryan79 changed the base branch from master to develop December 12, 2019 14:11

dpryan79 and others added 6 commits December 12, 2019 15:19

Merge branch 'develop' into addSilhouette

4ceffec

Update requirements.txt

4f95604

Update setup.py

bc3f169

Update heatmapper.py

bf853ea

update galaxy wrapper

4bd1150

Merge branch 'addSilhouette' of https://github.com/lldelisle/deepTools …

8fb8cee

…into addSilhouette

Fix run time issues

c169873

dpryan79 added 2 commits December 16, 2019 15:11

refactor, the order matters here.

ab3c06c

removing debugging stuff

aed4440

dpryan79 added 3 commits January 14, 2020 13:31

Merge branch 'develop' into addSilhouette

c5ceb18

Update heatmapper.py

ebc1bc7

Update plotHeatmap.py

93b3aeb

dpryan79 merged commit e8856ce into deeptools:develop Jan 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] added a silhouette calculation #876

[WIP] added a silhouette calculation #876

lldelisle commented Sep 13, 2019

lldelisle commented Dec 13, 2019

dpryan79 commented Dec 13, 2019

lldelisle commented Dec 13, 2019

dpryan79 commented Dec 19, 2019

dpryan79 commented Jan 14, 2020

	if includeLabels:
	bed_regions_list = [[chrom, x[4], x[2], x[3], x[5], x[6]] for x in bed_interval_tree.findOverlaps(chrom, reg[0], reg[1], trimOverlap=True, numericGroups=True, includeStrand=True)]
	else:
	bed_regions_list = [[chrom, x[4], x[5], x[6]] for x in bed_interval_tree.findOverlaps(chrom, reg[0], reg[1], trimOverlap=True, includeStrand=True)]

[WIP] added a silhouette calculation #876

[WIP] added a silhouette calculation #876

Conversation

lldelisle commented Sep 13, 2019

lldelisle commented Dec 13, 2019

dpryan79 commented Dec 13, 2019

lldelisle commented Dec 13, 2019

dpryan79 commented Dec 19, 2019

dpryan79 commented Jan 14, 2020