This is based on the MeCab documentation.
MeCab is an open source morphological analysis engine developed as a joint research project between Kyoto University Information Research department and the Nippon Telegraph and Telecommunications Communication Science Laboratories. It is built with the goal of general purpose analysis and does not depend on any particular language corpus/dictionary.
MeCab uses Conditional Random Fields (CRF) parameter estimation, improving upon the Hidden Markov Models as used by ChaSen. MeCab also typically performs faster than ChaSen, Juman, and KAKASI. Incidentally, mekabu is the author's favourite dish.
(Translator's note: MeCab in Japanese is pronounced 'mekabu' - which is the thick part of wakame seaweed just above the root)
- Generic design that does not depend on a dictionary corpus
- Based on Conditional Random Fields (CRF) for high precision analysis
- Faster than ChaSen and KAKASI libraries
- Dictionary search algorithm uses high speed TRIE structure double-array
- Library is re-entrant
- Bindings for various scripting languages (Perl, Ruby, Python, Java, C#)
(TODO)
(TODO)
See project page for up to date downloads
- C++ compiler (compiles with g++ 3.4.3 and VC7)
- iconv (libiconv): used for dictionary format conversion
Installation is the same as typical free software.
% tar zxfv mecab-X.X.tar.gz
% cd mecab-X.X
% ./configure
% make
% make check
% su
# make install
% tar zxfv mecab-ipadic-2.7.0-XXXX.tar.gz
% mecab-ipadic-2.7.0-XXXX
% ./configure
% make
% su
# make install
For binary installation, please use the self-extracting installer. Do the same for dictionary installation.
See below MeCab accepting input from stdin upon startup. MeCab assumes one line per sentence.
% mecab
すもももももももものうち
すもも 名詞,一般,*,*,*,*,すもも,スモモ,スモモ
も 助詞,係助詞,*,*,*,*,も,モ,モ
もも 名詞,一般,*,*,*,*,もも,モモ,モモ
も 助詞,係助詞,*,*,*,*,も,モ,モ
もも 名詞,一般,*,*,*,*,もも,モモ,モモ
の 助詞,連体化,*,*,*,*,の,ノ,ノ
うち 名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ
EOS
The output format significantly differs from ChaSen, becoming:
表層形\t品詞,品詞細分類1,品詞細分類2,品詞細分類3,活用形,活用型,原形,読み,発音
or in English:
Original Form\t
Part of Speech,
Part of Speech section 1,
Part of Speech section 2,
Part of Speech section 3,
Conjugated form,
Inflection,
Reading,
Pronounciation
If a file is passed in as the argument, it becomes the analysis target. It is also possible to direct output to a file using the -o option.
% mecab INPUT -o OUTPUT
Use the -O option as below
% mecab -O wakati
太郎はこの本を二郎を見た女性に渡した。
太郎 は この 本 を 二郎 を 見 た 女性 に 渡し た 。
Use the -O option as below
% mecab -Oyomi (Assign readings)
% mecab -Ochasen (ChaSen compatible)
% mecab -Odump (Full information dump)
These output formats are stored in /usr/local/lib/mecab/ipadic/dicrc.
The user can also create custom definitions. Please take a look
at the Output Formats documentation (TODO).
(TODO)
% tar zxfv mecab-ipadic-2.7.0-xxxx
% cd mecab-ipadic-2.7.0-xxxx
% ./configure --with-charset=sjis
% make
% tar zxfv mecab-ipadic-2.7.0-xxxx
% ./configure --with-charset=utf8
% make
(TODO)
% cd mecab-ipadic-2.7.0-xxxx
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -t utf-8
# make install
(TODO)
(TODO)
ホリエモン市
ホリエモン 名詞,固有名詞,地域,一般,*,*,*
市 名詞,接尾,地域,*,*,*,市,シ,シ
EOS
ホリエモンさん
ホリエモン 名詞,固有名詞,人名,一般,*,*,*
さん 名詞,接尾,人名,*,*,*,さん,サン,サン
(TODO)
%mecab --unk-feature "未知語"
ホリエモンさん
ホリエモン 未知語
さん 名詞,接尾,人名,*,*,*,さん,サン,サン
(TODO)
% mecab -N2
今日もしないとね。
今日 名詞,副詞可能,*,*,*,*,今日,キョウ,キョー
も 助詞,係助詞,*,*,*,*,も,モ,モ
し 動詞,自立,*,*,サ変・スル,未然形,する,シ,シ
ない 助動詞,*,*,*,特殊・ナイ,基本形,ない,ナイ,ナイ
と 助詞,接続助詞,*,*,*,*,と,ト,ト
ね 助詞,終助詞,*,*,*,*,ね,ネ,ネ
。 記号,句点,*,*,*,*,。,。,。
EOS
今日 名詞,副詞可能,*,*,*,*,今日,キョウ,キョー
もし 副詞,一般,*,*,*,*,もし,モシ,モシ
ない 形容詞,自立,*,*,形容詞・アウオ段,基本形,ない,ナイ,ナイ
と 助詞,接続助詞,*,*,*,*,と,ト,ト
ね 助詞,終助詞,*,*,*,*,ね,ネ,ネ
。 記号,句点,*,*,*,*,。,。,。
EOS
Jorge Nocedal for making the FORTRAN implementation of L-BFGS open to the public.
http://www.ece.northwestern.edu/~nocedal/lbfgs.html
J. Nocedal. Updating Quasi-Newton Matrices with Limited Storage (1980), Mathematics of Computation 35, pp. 773-782. D.C. Liu and J. Nocedal. On the Limited Memory Method for Large Scale Optimization (1989), Mathematical Programming B, 45, 3, pp. 503-528.
めかぶ