Code

Large data mode#182

Code

Closed

evanroyrees wants to merge 76 commits intoKwanLab:devKwanLab/Autometa:devfrom evanroyrees:large-data-modeevanroyrees/Autometa:large-data-modeCopy head branch name to clipboard

+2,472-532

Commits

Commits on Mar 12, 2021

• 📝 Update bug report template

  evanroyrees

  committed
  1660b3c

  Copy full SHA for 1660b3c

Commits on Mar 16, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into dev

  evanroyrees

  committed
  7909150

  Copy full SHA for 7909150

Commits on Mar 17, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into dev

  evanroyrees

  committed
  a0c472f

  Copy full SHA for a0c472f

Commits on Apr 20, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into dev

  evanroyrees

  committed
  d41a775

  Copy full SHA for d41a775

Commits on Apr 21, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into dev

  evanroyrees

  committed
  3bcf59d

  Copy full SHA for 3bcf59d

Commits on Apr 22, 2021

• 🔥 Remove length table provided to bedtools genomecov

  Show description for a6a1f22
  evanroyrees

  committed
  a6a1f22

  Copy full SHA for a6a1f22

Commits on Apr 27, 2021

• --wip-- [skip ci]

  evanroyrees

  committed
  aa74276

  Copy full SHA for aa74276

Commits on May 19, 2021

• 🎨 Add large-data-mode feature to recursive_dbscan binning function

  Show description for 00b895c
  evanroyrees

  committed
  00b895c

  Copy full SHA for 00b895c

• 🎨 Add large-data-mode feature of recursive rank kmer embedding

  evanroyrees

  committed
  057c2a5

  Copy full SHA for 057c2a5

• 🎨 Add --cache argument for caching kmer embeddings.

  evanroyrees

  committed
  13c8381

  Copy full SHA for 13c8381

• Merge branch 'dev' of https://github.com/WiscEvan/Autometa into dev

  evanroyrees

  committed
  307b3bd

  Copy full SHA for 307b3bd

• Merge branch 'dev' of https://github.com/WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  2787312

  Copy full SHA for 2787312

Commits on May 25, 2021

• 🐛 Fix argparse parameters in recursive_dbscan to convert inputs to specified type

  Show description for b9f1613
  evanroyrees

  committed
  b9f1613

  Copy full SHA for b9f1613

• 📝 Add logging statements between rank counts subsetting logic

  evanroyrees

  committed
  e34fe7f

  Copy full SHA for e34fe7f

Commits on Jun 15, 2021

• 🐛 Fix incorrect args called in parse(...)

  Show description for df3ea72
  evanroyrees

  committed
  df3ea72

  Copy full SHA for df3ea72

Commits on Jul 7, 2021

• --WIP--

  evanroyrees

  committed
  5282a60

  Copy full SHA for 5282a60

• Merge branch 'large-data-mode' of https://github.com/WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  f2aef76

  Copy full SHA for f2aef76

• 🐛 import NCBI to be used in writing canonical ranks in main binning output table

  evanroyrees

  committed
  791b43f

  Copy full SHA for 791b43f

Commits on Jul 12, 2021

• 🎨 clean-up WIP for recursive_dbscan

  Show description for ec3750e
  evanroyrees

  committed
  ec3750e

  Copy full SHA for ec3750e

• Merge branch 'dev' of github.com:KwanLab/Autometa into large-data-mode

  evanroyrees

  committed
  e0276c0

  Copy full SHA for e0276c0

Commits on Jul 19, 2021

• Add more logging debug information to determine marker annotation times

  evanroyrees

  committed
  7e18355

  Copy full SHA for 7e18355

• 🎨 Add more information to logger message for marker gene cutoff filter

  evanroyrees

  committed
  11c02f6

  Copy full SHA for 11c02f6

Commits on Jul 29, 2021

• 🎨 Add binning caching

  evanroyrees

  committed
  e6dd75c

  Copy full SHA for e6dd75c

• 🎨 Add binning checkpointing logic

  evanroyrees

  committed
  ab1f416

  Copy full SHA for ab1f416

Commits on Aug 2, 2021

• 🔥 Replace print with logger.debug

  evanroyrees

  committed
  9d77c04

  Copy full SHA for 9d77c04

Commits on Aug 3, 2021

• 🐛 Fix checkpoint restart logic when retrieving binned contigs

  Show description for 0259f2d
  evanroyrees

  committed
  0259f2d

  Copy full SHA for 0259f2d

• 🐛 Fix compressed read functionality in get_checkpoint_info(...) for gzipped checkpoint files

  evanroyrees

  committed
  1de779a

  Copy full SHA for 1de779a

• 🎨 Add logger emit message when reading annotations for binning utilities

  Show description for 7cfc496
  evanroyrees

  committed
  7cfc496

  Copy full SHA for 7cfc496

• 🎨📝 Add binning-checkpoints parameter and update type hint for get_checkpoint_info(...)

  evanroyrees

  committed
  a5e65d8

  Copy full SHA for a5e65d8

• 🐛 Fix gzipped writing of binning checkpoints file

  evanroyrees

  committed
  c185633

  Copy full SHA for c185633

• 🐛 Reindex bins for binning checkpoints

  Show description for 7f8e216
  evanroyrees

  committed
  7f8e216

  Copy full SHA for 7f8e216

• 🎨📝 Add header lines 'Parameters' and 'Runtime Variables' to binning checkpoints

  Show description for d6f09a6
  evanroyrees

  committed
  d6f09a6

  Copy full SHA for d6f09a6

• 🎨 Add timestamp info to runtime variables in binning checkpoints

  evanroyrees

  committed
  006b627

  Copy full SHA for 006b627

Commits on Aug 4, 2021

• 📝 Add lines between parameter docstrings

  evanroyrees

  committed
  a0fe1da

  Copy full SHA for a0fe1da

• 🎨 Move large-data-mode code to large_data_mode.py.

  Show description for 92517c0
  evanroyrees

  committed
  92517c0

  Copy full SHA for 92517c0

• 🐛 Fix function call for checkpoint(...)

  evanroyrees

  committed
  6a8ea66

  Copy full SHA for 6a8ea66

Commits on Aug 9, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into large-data-mode

  evanroyrees

  committed
  3743469

  Copy full SHA for 3743469

Commits on Aug 11, 2021

• 🐛🎨🔥 Add logger and remove print statements

  evanroyrees

  committed
  98ab81d

  Copy full SHA for 98ab81d

• 📝 Add description to reindex_bin_names(...) func return docstring

  evanroyrees

  committed
  5e702b6

  Copy full SHA for 5e702b6

• 🎨 Add copyright, logging and encoding to header of script l.d.m. loginfo script

  evanroyrees

  committed
  88398c3

  Copy full SHA for 88398c3

• 🎨📝 Change choices statement for --rank-filter in recursive_dbscan.py for docs build

  evanroyrees

  committed
  de1b9a2

  Copy full SHA for de1b9a2

• 🎨📝🐛 Change choices statement for --rank-filter in large_data_mode.py for docs build

  evanroyrees

  committed
  9ec5744

  Copy full SHA for 9ec5744

• 🎨📝 Add line in contributing with link to good first issues

  evanroyrees

  committed
  8c13c19

  Copy full SHA for 8c13c19

• 🎨📝 Add line in contributing with link to good first issues

  evanroyrees

  committed
  7ced8db

  Copy full SHA for 7ced8db

Commits on Aug 24, 2021

• 🎨 Add reference to dead_prot.accession2taxid in NCBI class

  Show description for 378960c
  evanroyrees

  committed
  378960c

  Copy full SHA for 378960c

• Merge branch 'large-data-mode' of github.com:WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  9b4ad1b

  Copy full SHA for 9b4ad1b

Commits on Aug 25, 2021

• 🎨 Change file extension of unclustered seqs written to unclustered.fasta while binned seqs are written to <cluster>.fna

  evanroyrees

  committed
  91e252b

  Copy full SHA for 91e252b

Commits on Sep 30, 2021

• 🎨🐍 merge upstream dev

  Show description for e0507f9
  evanroyrees

  committed
  e0507f9

  Copy full SHA for e0507f9

• 🎨🐍🍏 Re-change autometa-parse-bed entrypoint to autometa-bedtools-genomecov

  evanroyrees

  committed
  da12095

  Copy full SHA for da12095

Commits on Oct 1, 2021

• 🎨🐛 change list-comprehension for taxids search with RMQ in lca.py to set comprehension

  Show description for 2f08998
  evanroyrees

  committed
  2f08998

  Copy full SHA for 2f08998

• 🎨📝 add docstring to search_prot_accessions(...) in lca.py

  evanroyrees

  committed
  c54261d

  Copy full SHA for c54261d

Commits on Oct 3, 2021

• 🐛 incorrect type-hint for reduce_taxids_to_lcas(...)

  evanroyrees

  committed
  cf9802c

  Copy full SHA for cf9802c

Commits on Oct 4, 2021

• 🎨🐛📝 Incorrect dict iteration in convert_sseqids_to_taxids(...)

  evanroyrees

  committed
  cba2d8a

  Copy full SHA for cba2d8a

Commits on Oct 6, 2021

• 🎨🐛🔥 clean-up of lca.py

  Show description for 41c0c57
  evanroyrees

  committed
  41c0c57

  Copy full SHA for 41c0c57

• 🎨 change logger.warning(...) to logger.warn(...)

  evanroyrees

  committed
  9fe4149

  Copy full SHA for 9fe4149

• 🎨 Add functionality to parse prot.accession2taxid.FULL.gz

  Show description for 26daab4
  evanroyrees

  committed
  26daab4

  Copy full SHA for 26daab4

Commits on Oct 11, 2021

• 🎨🐍 Add method to read sseqid_to_taxid output table

  Show description for 1d4840a
  evanroyrees

  committed
  1d4840a

  Copy full SHA for 1d4840a

Commits on Oct 14, 2021

• 🎨🐛 Add exception handling for autometa-length-filter

  Show description for 05a4c9f
  evanroyrees

  committed
  05a4c9f

  Copy full SHA for 05a4c9f

• 🎨 Add core_dist_n_jobs param to run_hdbscan(...)

  Show description for 23e9c76
  evanroyrees

  committed
  23e9c76

  Copy full SHA for 23e9c76

• 🎨 Add logic to handle when the user already has provided a length-filtered metagenome and is trying to retrieve gc_content and other assembly stats.

  evanroyrees

  committed
  08983c2

  Copy full SHA for 08983c2

Commits on Oct 15, 2021

• 🐛 Fix incorrect import in automeat.config.databases from hmmer to hmmscan

  evanroyrees

  committed
  cc6bef0

  Copy full SHA for cc6bef0

• Merge branch 'large-data-mode' of https://github.com/WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  8528aa2

  Copy full SHA for 8528aa2

Commits on Oct 19, 2021

• 🎨🐛🔥 Remove domain kwarg in get_clusters(...)

  Show description for d1e2198
  evanroyrees

  committed
  d1e2198

  Copy full SHA for d1e2198

Commits on Oct 21, 2021

• 🎨🐛 Fix cluster metric addition/filter

  Show description for 9f951c7
  evanroyrees

  committed
  9f951c7

  Copy full SHA for 9f951c7

Commits on Oct 26, 2021

• 🎨🐛 Fix bug at canonical rank kmer embedding stage

  Show description for 366b2cf
  evanroyrees

  committed
  366b2cf

  Copy full SHA for 366b2cf

Commits on Nov 11, 2021

• 🎨🐍 Add numba thread-safe configuration in recursive_dbscan.py

  evanroyrees

  committed
  461363d

  Copy full SHA for 461363d

Commits on Nov 16, 2021

• 🐛 change sseqid_to_taxid_output check in lca.py

  Show description for 46e5944
  evanroyrees

  committed
  46e5944

  Copy full SHA for 46e5944

• 🎨 Add logger emit to large-data-mode corresponding to markers annotation

  evanroyrees

  committed
  0f5205f

  Copy full SHA for 0f5205f

Commits on Nov 18, 2021

• 🎨🐛🐍 Add --cpus param to autometa-binning and autometa-large-data-mode-binning entrypoints

  Show description for b1a903e
  evanroyrees

  committed
  b1a903e

  Copy full SHA for b1a903e

Commits on Nov 24, 2021

• 🎨 Add trimap to choices of kmer embed methods (large-data-mode)

  Show description for 08a3e36
  evanroyrees

  committed
  08a3e36

  Copy full SHA for 08a3e36

Commits on Nov 30, 2021

• ⬆️ pin scikit-learn to 0.24 to prevent errors arising from hdbscan internals using joblib.

  Show description for a0cfee4
  evanroyrees

  committed
  a0cfee4

  Copy full SHA for a0cfee4

Commits on Dec 2, 2021

• Merge branch 'dev' of https://github.com/KwanLab/Autometa into large-data-mode

  evanroyrees

  committed
  263d220

  Copy full SHA for 263d220

• Merge branch 'large-data-mode' of github.com:WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  62c7042

  Copy full SHA for 62c7042

Commits on Dec 9, 2021

• 🎨🐍 Add pd.DataFrame type input handling for get_metabin_stats(markers=Union[str,pd.DataFrame])

  evanroyrees

  committed
  f61a92f

  Copy full SHA for f61a92f

• Merge branch 'large-data-mode' of https://github.com/WiscEvan/Autometa into large-data-mode

  evanroyrees

  committed
  e39e2cd

  Copy full SHA for e39e2cd

Commits on Dec 21, 2021

• 🎨 refactor get_metabin_stats(...) using pandas groupby object

  evanroyrees

  committed
  3f28826

  Copy full SHA for 3f28826