Skip to content

johtani/wiki-json-loader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

40 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wikipedia json data Loader

License: MIT

Load JSON files that made by wiki-extractor-rs to search engines. Now, the tool support only JSON files that are created wiki-extractor-rs default settings.

Requirements

  • Rust nightly : elasticsearch-rs support only nightly rustc

Support search engines

  • Elasticsearch >=7.x
  • Azure Cognitive Search

Usage

Sample schema and setting yaml in sample directory.

Prepare

For Elasticsearch, sample index settings/mappings in ./sample/elasticsearch directory. For Azure Cognitive Search, sample index settings/mappings in ./sample/azure_cognitive_search directory.

The command will create an index with with schema json if the index doesn't exist.

Load command

Show help with the following command::

$ ./bin/wiki-json-loader -h

Options

$ ./wiki-json-loader -c <SEARCH_ENGINE_CONFIG> -s <SEARCH_ENGINE_TYPE> <INPUT_DIR>

About

Load wiki json to search engines. Implemented Rust

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages