v1.10.2

This patch release mostly addresses another issue with the DeID model(s).

The underlying RoBERTa models have a token limit (512) and because of that later parts of larger documents would fail to de-identify.
This Release (or more specifically, PR #405) fixes that issue by allowing the user to specify the overlapping tokens (and defaults to 5).

What's Changed

• CU-8693v3tt6 SOMED opcs refset selection by @mart-r in #402
• CU-8693v6epd: Move typing imports away from pydantic by @mart-r in #403
• CU-8693qx9yp Deid chunking - hugging face pipeline approach by @shubham-s-agarwal in #405

New Contributors

• @shubham-s-agarwal made their first contribution in #405

Full Changelog: v1.10.1...v1.10.2

v1.10.1

This patch release mostly addresses two things regarding DeID models:

1. DeID models created with older medcat versions (prior to v1.9.3) would silently fail to deidentify anything [#397].
2. DeID model multiprocessing doesn't work in all cases. This release adds a more descriptive exception when this happens as well as a description of two workarounds (either save the model on disk and load back up; or use the cpu-only torch)[#393 and #400].

The patch release also fixes some of the issues with dependencies [#392].

There are a few other smaller changes which you can see below.

What's Changed

• CU-8693cv3w0 Fix fallback spacy model existance on pip installs by @mart-r in #386
• CU-8693b0a61 Add method to get spacy model version by @mart-r in #381
• CU-8693kp0gw: Pin more recent versions for major dependencies; Avoid major bumps where applicable by @mart-r in #392
• add: metacat can predict on spans in arbitrary spangroups by @jkgenser in #391
• CU-8693ruk7p: Bump mypy version in dev-requirements by @mart-r in #396
• Bump django from 3.2.23 to 3.2.24 in /webapp/webapp by @dependabot in #395
• CU-8693t24ed: Add workaround for older DeID models in newer MedCAT by @mart-r in #397
• CU-2hz5ump deid mulitprocessing by @mart-r in #393
• Cu 8693u6b4u tests continue on fail by @mart-r in #400

New Contributors

• @jkgenser made their first contribution in #391

Full Changelog: v1.10.0...v1.10.1

v1.10.0

Main changes to mention:

• Better model-compatibility regarding known config.linking.filters.cuis issue
  • The default value (an empty dict {}) will be converted to an empty set in memory
• When saving a model, the config is now saved separately
  • We retain backwards compatibility for when the config is saved alongside the CDB (as was the case this far)
• Add a CDB merge method (in medcat.utils.cdb_utils)
• We can now safely fall back to en_core_web_md spacy model
  • If the spacy model isn't provided by the CAT for some reason
  • This includes the possibility of automatically downloading the dependency if it doesn't exist
• Many other changes as per below

What's Changed

• Bump urllib3 from 1.26.5 to 1.26.17 in /webapp/webapp by @dependabot in #351
• Cu 8692wbcq5 docs builds by @mart-r in #359
• CU-8692uznvd: Allow empty-dict config.linking.filters.cuis and convert to set in memory by @mart-r in #352
• CU-8692t3fdf separate config on save by @mart-r in #350
• CU-2cdpd4t: Unify default addl_info in different methods. by @mart-r in #363
• CU-86931prq4: GHA runs on node12 by @mart-r in #368
• Cu 1yn0v9e duplicate multiprocessing methods by @mart-r in #364
• CU-2exy49p: Make sure the cdb.add_concept really adds a concept or somehow make it clear by @adam-sutton-1992 in #370
• 869377m3u: Add comment regarding demo link load times to README by @mart-r in #376
• Added README.md documentation for CPU only installations by @adam-sutton-1992 in #365
• Cu 8692zguyq no preferred name by @mart-r in #367
• Add trainer callbacks for Transformer NER by @baixiac in #377
• Bump django from 3.2.20 to 3.2.23 in /webapp/webapp by @dependabot in #369
• Bump urllib3 from 1.26.17 to 1.26.18 in /webapp/webapp by @dependabot in #360
• CU-2e77a31 improve print stats by @mart-r in #366
• CU2e77a5x - Add a CDB merge function by @adam-sutton-1992 in #373
• CU-8693az82g Remove cdb tests side effects by @mart-r in #380
• CU-8693bpq82 fallback spacy model by @mart-r in #384
• Fix stopwords loading bug by @jenniferajiang in #383
• CU-8693cv3w0 Fix fallback spacy model existance on pip installs by @mart-r in #386
• CU-8693b0a61 Add method to get spacy model version by @mart-r in #381

New Contributors

• @adam-sutton-1992 made their first contribution in #370
• @jenniferajiang made their first contribution in #383

Full Changelog: v1.9.3...v1.10.0

v1.8.2

This is a patch release that addresses:

• Remove py2neo dependency and code (#356)
  • This prevented anyone from installing medcat since it was no longer available on pypi
  • PS! It may still work for some folk if the py2neo package is cached by pip, but at some point the cache will be invalidated

Full Changelog: v1.8.1...v1.8.2

v1.7.4

This is a patch release that addresses:

• Remove py2neo dependency and code (#356)
  • This prevented anyone from installing medcat since it was no longer available on pypi
  • PS! It may still work for some folk if the py2neo package is cached by pip, but at some point the cache will be invalidated

Full Changelog: v1.7.3...v1.7.4

v1.6.1

This is a patch release that addresses:

• Fix fakedict/config issue (#341)
• Remove py2neo dependency and code (#356)
  • This prevented anyone from installing medcat since it was no longer available on pypi
  • PS! It may still work for some folk if the py2neo package is cached by pip, but at some point the cache will be invalidated
• Some minor changes to make GHA pass
  • Fix for Rosalind links (#342)
  • Allow production workflow on .post branches
  • Make production workflow use correct branch
  • Bump ubuntu version in publish workflow (from 18.04 to 20.04)
  • Bump mypy version to 0.991 for compatibility
  • Pin pydantic version to less than 1.8 for spacy compatibility

Full Changelog: v1.6.0...v1.6.1

v1.5.3

This is a patch release that addresses:

• Fix fakedict/config issue (#341)
• Remove py2neo dependency and code (#356)
  • This prevented anyone from installing medcat since it was no longer available on pypi
  • PS! It may still work for some folk if the py2neo package is cached by pip, but at some point the cache will be invalidated
• Some minor changes to make GHA pass
  • Fix for Rosalind links (#287 and #342)
  • Allow main workflow on .post branches
  • Allow prudiction workflow on .post branches
  • Make production workflow use correct branch
  • Bump ubuntu version in publish workflow (from 18.04 to 20.04)
  • Bump mypy version to 0.991 for compatibility
  • Pin pydantic version to less than 1.8 for spacy compatibility

Full Changelog: v1.5.0...v1.5.3

v1.9.3 - patch fix for dependency issues

What's Changed

• CU-8692wgmkm: Remove py2neo dependency and the code that used it by @mart-r in #356
• CU-8692wcmp7: update transformers to the latest version by @tomolopolis in #355
• v1.19.3 by @tomolopolis in #357

Full Changelog: v1.9.2...v1.9.3

v1.9.2

What's Changed

• CU-8692kpchc Fix for Rosalind link not working by @mart-r in #342
• Fix issue 340 (snomed preprocessing) by @mart-r in #343
• CU-8692kn0yv Fix issue with fake dict in identifier based config by @mart-r in #341
• CU-8692mevx8 Fix issue with filters not taking effect in train_supervised method by @mart-r in #345
• CU-8692wb8gf: 'tokenizers>=0.12.0', # 0.13.1 doesn't seem to build by @tomolopolis in #353
• v1.19.2 Patch Fix by @tomolopolis in #354

Full Changelog: v1.9.1...v1.9.2

v1.8.1

This attempts to fix issues related to the following PRs:
#341
#345

Full Changelog: 1.8.0...v1.8.1