Skip to content

Adding a new lexicon

César Del Solar edited this page Jan 25, 2020 · 13 revisions

Note: This is very out of date

These are the steps I used to add the CSW12 lexicon:

wget http://svn.pietdepsi.com/repos/projects/zyzzyva/trunk/data/words/British/CSW12.txt (thanks Michael!)
python makedawg.py ../words/CSW12.txt -o CSW12.trie -r CSW12_r.trie   (in my dbCreator directory)
mv CSW12.trie ../words/
mv CSW12_r.trie ../words/

added code in this commit: https://github.com/domino14/Webolith/commit/cb3d6a4d4050499249d2d9f47542b915ade217af

./dbCreator OWL2 CSW07 CSW12   (in my dbCreator-build-desktop directory)

mysql -u root -p mypass
for each of base_lexicon, base_word, base_alphagram:
    delete from base_tablename_
    load data infile '/Users/cesar/coding/webolith/dbCreator-build-desktop/{alphs,words,lex}.txt' into table base_tablename_ fields terminated by ',';

then: python manage.py dumpdata wordwalls.namedlist > wordwalls/fixtures/namedLists.json

then copy the namedLists.json and the alphs, words, lex .txt file to the remote server and load em in!

on aws server, it seems i have to do:

load data local infile '/home/ubuntu/words.txt' into table base_word fields terminated by ',';  (not just load data infile)

finally, loadWordsIntoRedis