Code

SemMedDB V43 update#8

Code

Merged

erikyao merged 15 commits intomainbiothings/semmeddb:mainfrom tidy_CUIbiothings/semmeddb:tidy_CUICopy head branch name to clipboardJan 11, 2023

+774-121

Commits

Commits on Nov 29, 2022

• format the code

  erikyao

  committed
  27d8134

  Copy full SHA for 27d8134

• tiny refactor to eliminate pandas dependency

  erikyao

  committed
  468ea76

  Copy full SHA for 468ea76

Commits on Dec 5, 2022

• major refactor: rename variables and functions; simply object CUI format checking; specify csv reading options

  erikyao

  committed
  bbb64f6

  Copy full SHA for bbb64f6

Commits on Dec 6, 2022

• resort to RE filter again with a simplified pattern; use pd.Series-wise matching operation to accelarate

  erikyao

  committed
  2531bb2

  Copy full SHA for 2531bb2

• tiny refactor: rename variables and simplify if conditions

  erikyao

  committed
  377ddaf

  Copy full SHA for 377ddaf

• restructure the parser; add functions about MRCUI data

  erikyao

  committed
  ec678ee

  Copy full SHA for ec678ee

• add filter on novelty scores for SemMed data

  erikyao

  committed
  20669dd

  Copy full SHA for 20669dd

Commits on Dec 30, 2022

• major refactor: preprocess all predications in Pandas

  erikyao

  committed
  f86216b

  Copy full SHA for f86216b

Commits on Dec 31, 2022

• add two marker columns, IS_SUBJECT_PIPED and IS_OBJECT_PIPED; change retired predication dtype to Pyarrow string before concatenation; add the client to query Node Normalizer

  erikyao

  committed
  856268c

  Copy full SHA for 856268c

Commits on Jan 4, 2023

• add functions to locate and delete equivalent NCBIGene IDs for piped CUIs; add a function to determine the document ID for each predication; change node normalizer parameter conflate to True

  erikyao

  committed
  37f78ee

  Copy full SHA for 37f78ee

Commits on Jan 10, 2023

• add cache reading/writing for Node Normalizer queries; revise equivalent NCBIGene ID row location algorithm (reducing running time from 24min to 40sec

  erikyao

  committed
  7dd09da

  Copy full SHA for 7dd09da

• add comments

  erikyao

  committed
  facb3b8

  Copy full SHA for facb3b8

• revise function delete_equivalent_ncbigene_ids to reduce peak memory usage from 3x dataframe size to 2x

  erikyao

  committed
  b21e641

  Copy full SHA for b21e641

• change the version number

  erikyao

  committed
  1d04d49

  Copy full SHA for 1d04d49

Commits on Jan 11, 2023

• change dumper file links in the manifest; update README

  erikyao

  committed
  279888e

  Copy full SHA for 279888e