Code

Update keyterms module#257

Code

Merged

bdewilde merged 41 commits intodevelopchartbeat-labs/textacy:developfrom update-keyterms-modulechartbeat-labs/textacy:update-keyterms-moduleCopy head branch name to clipboardJul 14, 2019

+2,103-1,478

Commits

Commits on Jul 5, 2019

• Move keyterm extraction functionality into subpkg

  bdewilde

  committed
  7b3aedd

  Copy full SHA for 7b3aedd

• Move SGRank func into its own module

  bdewilde

  committed
  8ed00f2

  Copy full SHA for 8ed00f2

• Move YAKE into its own module, and refactor

  Show description for 669c5c7
  bdewilde

  committed
  669c5c7

  Copy full SHA for 669c5c7

• Further refactor yake, for clarity

  bdewilde

  committed
  73e801d

  Copy full SHA for 73e801d

• Add new and better tests for sgrank

  bdewilde

  committed
  7f40d24

  Copy full SHA for 7f40d24

• Add tests for yake, and check for empty docs

  bdewilde

  committed
  61bc1c5

  Copy full SHA for 61bc1c5

• Test the new sgrank, not the old one

  bdewilde

  committed
  f5737db

  Copy full SHA for f5737db

• Add math.log2 to compat, use in yake

  bdewilde

  committed
  eb0687f

  Copy full SHA for eb0687f

Commits on Jul 6, 2019

• Add general-purpose keyterm-related funcs to utils mod

  bdewilde

  committed
  af3f47a

  Copy full SHA for af3f47a

• Add module of general funcs for graph-based kt extraction

  bdewilde

  committed
  12d29f3

  Copy full SHA for 12d29f3

• Add module for updated textrank kt extraction

  bdewilde

  committed
  e58c1c2

  Copy full SHA for e58c1c2

• Add module encodings for PY2

  bdewilde

  committed
  dfc5eb3

  Copy full SHA for dfc5eb3

• Add textrank tests, fix PY2 bug in func

  bdewilde

  committed
  02577ee

  Copy full SHA for 02577ee

• Move get_topn_terms func into utils, and generalize it

  bdewilde

  committed
  3a74d37

  Copy full SHA for 3a74d37

• Fix bug in textrank get candidates, and filter topn

  bdewilde

  committed
  dbce75e

  Copy full SHA for dbce75e

• Add generalized ngram candidate generation to utils

  bdewilde

  committed
  ddae7c8

  Copy full SHA for ddae7c8

Commits on Jul 7, 2019

• Rename sgrank args, and make alg 50% faster

  bdewilde

  committed
  66fff58

  Copy full SHA for 66fff58

• Fix empty doc case for sgrank, change topn float behavior

  bdewilde

  committed
  e1068be

  Copy full SHA for e1068be

• Refactor sgrank code, for readability

  bdewilde

  committed
  b4531a7

  Copy full SHA for b4531a7

• Filter similar keyterms out of sgrank results

  bdewilde

  committed
  05cd8ba

  Copy full SHA for 05cd8ba

• Filter similar keyterms out of textrank results

  bdewilde

  committed
  6551c88

  Copy full SHA for 6551c88

• Always filter similar keyterms from yake results

  Show description for cbceb98
  bdewilde

  committed
  cbceb98

  Copy full SHA for cbceb98

• Generalize TextRank alg to similar variations

  Show description for 2bb0877
  bdewilde

  committed
  2bb0877

  Copy full SHA for 2bb0877

Commits on Jul 9, 2019

• Extend textrank to include positionrank algo

  bdewilde

  committed
  72a7a86

  Copy full SHA for 72a7a86

Commits on Jul 11, 2019

• Add scake keyterm extraction algorithm

  bdewilde

  committed
  85cd8cf

  Copy full SHA for 85cd8cf

Commits on Jul 14, 2019

• Implement efficient k-truss algorithm, plus bugfixes

  bdewilde

  committed
  2ca17d7

  Copy full SHA for 2ca17d7

• Add tests for scake algorithm

  bdewilde

  committed
  f94e9ce

  Copy full SHA for f94e9ce

• Bail out of scake for empty docs

  bdewilde

  committed
  546c318

  Copy full SHA for 546c318

• Move remaining keyterms functionality into subpkg

  Show description for 6e94ae9
  bdewilde

  committed
  6e94ae9

  Copy full SHA for 6e94ae9

• Add corresponding tests for keyterm func

  bdewilde

  committed
  73ebedc

  Copy full SHA for 73ebedc

• Rename keyterm tests to reflect renamed arg

  bdewilde

  committed
  b4ab522

  Copy full SHA for b4ab522

• Rename ke utils test file so it's unique

  Show description for b05d46c
  bdewilde

  committed
  b05d46c

  Copy full SHA for b05d46c

• Remove top-level keyterms module and tests

  bdewilde

  committed
  91c63f0

  Copy full SHA for 91c63f0

• Update quickstart docs and tests for new ke subpkg

  bdewilde

  committed
  ddcaf87

  Copy full SHA for ddcaf87

• Update ke subpkg sphinx docs

  bdewilde

  committed
  0e10c09

  Copy full SHA for 0e10c09

• Add and standardize kt candidate selection funcs

  bdewilde

  committed
  eb1156d

  Copy full SHA for eb1156d

• Make ngram candidate func more flexible on inputs

  bdewilde

  committed
  865a7b6

  Copy full SHA for 865a7b6

• Add POS-filtering to all keyterm extraction funcs

  bdewilde

  committed
  c38a621

  Copy full SHA for c38a621

• Exclude stopwords from unigram candidates in yake

  bdewilde

  committed
  394093d

  Copy full SHA for 394093d

• Allow different term normalizations in yake

  Show description for a58d663
  bdewilde

  committed
  a58d663

  Copy full SHA for a58d663

• Rename ke subpkg tests dir

  bdewilde

  committed
  d1cd21a

  Copy full SHA for d1cd21a