ARElight 0.25.0

📘 Complete list of updates

Main Changes

Support of the new UI
It is expected to be a separated UI for the ARElight, which is sourced there:
https://github.com/nicolay-r/ARElight-server

• Support bulk-translate #162
• Inference argument JSON exporter function #142
• ARElight operations -- logger #143
• ARElight inference -- support dynamic log writing #141
• 🔥 Support SQlite filtering operation for relation pairs. #144
• 🔥 🐛 MLTextTranslatorPipelineItem -- causes issues with the indices in tranlation (batching mode) #146
• 🔴 Translate: optional source. #153

Refactored features

• Drop support for the SQLiteService from AREkit #161
• Improve DeepPavlov NER performance by adopting batching #140
• Support new pipelines concept from AREkit with optional batching #136 🔥
• Add SQLite support for sampling #129
• Support prediction writing in SQLite #130
• Refactoring SQLite impemenation for OpenNRE #134
• OpenNRE implement SentenceREDataset based on SQLite storage #132
• • SentenceRELoader create a custom implementation (torch 🔥 dependency)
• • SentenceREDataset implement, fix the original issue of storing the whole data in memory! (torch 🔥 dependency )
• AREkit version of 0.25.0 (c8fb0bb)

Minor

• Writers relocation from AREkit #165
• CustomTermsSplitterPipelineItem -- remove that class since it only complicates NER pipeline #135
• Appropriate formatting of unit tests (https://github.com/nicolay-r/ARElight/blob/main/test/test_translation.py)
• 🔧 Predict header formatter -- sync #137
• 🔧 Sync samples storing path between different pipeline elements #138
• Fix httpx package 📦 related issues
• 🐛 SQLite3Writer -- extension is missed for the results of BERT application #145
• torch -- take control over num_workers parameter #149
• 🔥 🐛 MLTextTranslatorPipelineItem -- causes issues with the indices in tranlation (batching mode) #146
• ❌ save_demo_page method is no longer applicable. #151
• Clarifying cuda/ cpu parameter choices #156
• 🐛 MystemStemmer -- bring back wrapper for Mystem #163
• 🐛 BaseSerializerPipelineItem -- legacy, move from AREkit earlier version. #164

Full Changelog: 0.24.0...0.25.0

ARElight 0.24.0

📘 The complete list of changes

Simplified usability / Flexibility

• 🔧 SampleRowProvider -- Enable Crop Mode #125
• input_data for the pipeline -- design mechanism for supporting an output independing of the sequence of elements #90
• Refactoring installation process and usage as library #61
• D3JS templates and the related pipeline provider #77
• 🔧 Fix launch of the tranformer-based models #78
• ❌ Drop training support (43cac8c)
• Make home folder for states downloading #95
• tqdm -- for inference we may use defined progress bar since we know the total amount of rows #96
• Stemmer -- provide paramer options that allow to enable/disable stemming #97
• googletrans -- Integrate Text Translator (transfer learning) ✨ #108
• Torch and Huggingface
• infer_bert -- switch to LM teminology, support pre-trained states as parameters #70
• ❓ Support of OpenNRE with torch and transformers [NIVTS project findinds] #63
• Remove DeepPavlov and switch to the transformers #69
• Library
• Make it even simpler -- a single cmd script for everything #71
• 🔧 Switch to AREkit==0.24.0
• 🔧 Support new DeepPavlov==1.2.0
• 📦 Make it as a package
• DATA_DIR dependencies -- refactoring #84
• Remove Downloads #85

Minor fixes

• Remove data and predefined text dir #101
• Entities Formatters -- simplify structure, place them in the same infer.py script ✨ #103
• DataFrame processing -- text column customization via cmd parameter ✨ #107
• BERT -- pretrained state might be None if checkpoint is cached. ✨ #106
• Entities Formatters -- simplify structure, place them in the same infer.py script ✨ #103
• Remove pandas dependency #117
• graphs_operations_weighted results in empty graph (no nodes) #112
• Generalization -- settings for label provider/mapping [OpenNRE inference] #119
• from-files -- use DirectoryFilesDocOperation #121
• Setup the correct dependency from the OpenNRE 0.1.1 (ca76a3e)

Change Log

Full Changelog: 0.23.1...0.24.0

ARElight 0.23.1

Increased performance x2 due to support streaming from AREkit-0.23.1 🔥
Simplified API
Stop docking support
Updated project description

Updates:
#62

ARElight 0.23.0-p0

Improved stabillty and added new functionality

• Fixed problem with long-sequences of NER: #57
• Provided support of the dataframes in input :#58
• Generalization of NER, support of English language: #53
• Fixed output formatting, sync prediction and sample file names: #54
• Support sentence splitting selection algo: #55

ARElight 0.23.0

ARElight 0.23.0

• Support of the AREkit 0.23.0
• Removed support of Neural Networks
• Simplified API

ARElight-0.22.1

Changes

Sync with AREkit-0.22.1 API

• Removed support of collections serlalization
• Simplified implementations
• Proved unit testing scripts with html logs

Embeddings:

• news_mystem_skipgram_1000_20_2015.bin.gz now is a part of the AREkit-0.22.1

ARElight-0.22.0

#40 clarified in 0.22.0 branch