Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Speaker ID script improvements #1538

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
213 commits
Select commit Hold shift + click to select a range
ff7f51b
sid-fix-2017-02-11: [egs,scripts]: Moving SRE10 NNET2 scripts from sr…
Apr 10, 2017
3616ff8
[egs] egs/fisher_swbd/s5/local/online/run_nnet2_ms.sh, change default…
weegreenblobbie Feb 10, 2017
e1083d9
[egs] Add example scripts for Frisian-Dutch language (FAME! corpus)
Feb 11, 2017
80295c1
Early parts of 'shortcut' compilation
danpovey Dec 8, 2016
f16da00
Cosmetic changes in nnet3 code
danpovey Oct 5, 2016
06bd75a
Some code refactoring that will make it easier to implement online re…
danpovey Oct 8, 2016
871b39a
Some bug fixes to previous commit (RE refactoring code in nnet3).
danpovey Oct 8, 2016
7efa27f
Further refactoring to nnet3 compilation to make it easier to impleme…
danpovey Oct 10, 2016
c5f441b
Fix a few bugs shown up by valgrind testing
danpovey Oct 11, 2016
cab8420
Refactoring generation of the computation 'steps' for more clarity an…
danpovey Oct 15, 2016
6d0d9ff
Some minor refactoring to make online computation easier (remove unus…
danpovey Oct 15, 2016
7a53e66
Further progress [note, this is partial work, backing up. Search for…
danpovey Oct 26, 2016
b08940a
Going some way towards optimization for online decoding (identified p…
danpovey Nov 7, 2016
57a4af9
Get the online optimization code working to the point where the tests…
danpovey Nov 11, 2016
abdd595
Add a couple of previously omitted files
danpovey Nov 11, 2016
10d6a1a
Change name from online to looped (less confusable)
danpovey Nov 11, 2016
87f695b
Finishing the decodable objects (not yet for online computatoin), and…
danpovey Nov 14, 2016
ee9b963
Add decoding program nnet3-latgen-faster-looped
danpovey Nov 14, 2016
7327c03
Fix bug discovered by testing code
danpovey Nov 14, 2016
f37d422
Fix bug discovered by TDNN decoding script
tomkocse Dec 1, 2016
570e82f
Adding another optimization to convert row-wise to whole-matrix ops w…
danpovey Dec 9, 2016
f337886
Early parts of 'shortcut' compilation
danpovey Dec 8, 2016
d3acb9e
Add new type of optimization of per-row commands; finish some of the …
danpovey Dec 13, 2016
b1cb7d3
Getting shortcut compilation to the point where it's testable (test f…
danpovey Dec 15, 2016
e44ba77
Fix various bugs in shortcut compilation; add further testing code
danpovey Dec 15, 2016
8fd1959
Small documentation fix
danpovey Dec 15, 2016
67a8f7a
Remove no-longer-used option --cut-zero-frames from chain supervision…
danpovey Dec 15, 2016
1e215a8
Some draft code, on the way to changing egs-extraction code to allow …
danpovey Dec 16, 2016
0129565
Draft of UtteranceSplitter and related code
danpovey Dec 19, 2016
4ae5e53
Refactoring the example-extraction for nnet3, for more flexibility in…
danpovey Dec 24, 2016
8573207
Some partial work towards getting the new egs-merging process working…
danpovey Dec 27, 2016
fe1ccaa
Finish upgrades to eg-merging code for chain and discriminative examp…
danpovey Dec 28, 2016
9f833f9
[scripts][nnet3] Remove valid-{left,right}-context and priors-{left,r…
danpovey Dec 29, 2016
0d54881
Removing the --reduce-frames-per-eg option as a simplification prior …
danpovey Dec 30, 2016
ab38f33
Modify scripts to use --max-deriv-time-relative option in place of --…
danpovey Dec 30, 2016
e066c1b
Modify nnet3 python scripts to accept comma-separated alternatives in…
danpovey Dec 30, 2016
12e4eec
Modifying nnet3 scripts to accept more general form of minibatch-size…
danpovey Dec 30, 2016
72cb32a
Reverting a couple previous changes to local scripts which turned out…
danpovey Dec 30, 2016
9f5dd1e
Change nnet3 python scripts to support {left-right}-context-{initial,…
danpovey Dec 31, 2016
588bfd4
Add diagnostics to UtteranceSplitter; Various fixes.
danpovey Dec 31, 2016
00f4c41
Various code and script fixes
danpovey Jan 1, 2017
8b410f1
Change how descriptors behave in nnet3 (Issue #1125)
danpovey Jan 1, 2017
c353dc9
Various code fixes and additional diagnostics
danpovey Jan 2, 2017
d3eaea0
Various unrelated fixes to nnet3 code.
danpovey Jan 5, 2017
7da1bb1
Add code and scripts for LSTM with bounded activation
danpovey Jan 5, 2017
29c7efd
Add steps/nnet3/decode_looped.sh
danpovey Jan 5, 2017
798fb66
Updating egs-generation scripts to use new-style options for iVectors
danpovey Jan 6, 2017
1b6f7f6
Updating SetZero() to also zero stats.
danpovey Jan 6, 2017
6e648b9
tedlium script changes; renaming max-cell-value to decay-time in LSTM…
danpovey Jan 6, 2017
d128b92
Add timing info to looped compilation
danpovey Jan 6, 2017
5590773
Various bug fixes in scripts and code
danpovey Jan 7, 2017
93dcc07
Add ConstantComponent [better alternative to ConstantFunctionComponen…
danpovey Jan 8, 2017
2b66d28
Various refactoring of discriminative training; other fixes.
danpovey Jan 14, 2017
bde1355
Removing option --modify-learning-rates from example nnet3 discrimina…
danpovey Jan 15, 2017
1a8b6b2
Various script updates/fixes for discriminative training scripts; fix…
danpovey Jan 15, 2017
b29eed5
Fix minor bugs
danpovey Jan 15, 2017
e368b15
Some bug fixes to I/O code for nnet3
danpovey Jan 16, 2017
d3d89ec
Discriminative-training script fixes
danpovey Jan 16, 2017
d82fe44
Add scripts for discriminative training of TDNNs on swbd. Small bug …
danpovey Jan 17, 2017
e1db393
[src]: Removing SetZero() function in nnet3, adding SetAsGradient(). …
freewym Jan 17, 2017
f84d483
[src]: Minor updates to sequence training and adjusting priors. (#1345)
vimalmanohar Jan 17, 2017
e46cbac
Miscellaneous minor bug-fixes
danpovey Jan 18, 2017
90c88a7
Various minor fixes and script updates
danpovey Jan 19, 2017
d9b27f2
Update tools/Makefile to support OpenFst-1.5.4.
dogancan Dec 1, 2016
2bc558b
Update src/configure to support OpenFst-1.5.4.
dogancan Dec 1, 2016
b6b55d8
Fix unqualified std::vector occurrences.
dogancan Dec 1, 2016
7001487
Update fstext to support OpenFst-1.5.4.
dogancan Dec 1, 2016
f2f6dc4
Add support for API changes in OpenFst-1.5.
dogancan Dec 2, 2016
e7c3233
Rework OpenFst related preprocessor conditionals.
dogancan Dec 2, 2016
28e89e5
Fix Minimize calls.
dogancan Dec 4, 2016
c00caec
Remove OpenFst related compiler flags in src/configure.
dogancan Dec 5, 2016
3bf3b15
Update src/Makefile to enforce OpenFst >= 1.5.3.
dogancan Dec 5, 2016
091da1a
Remove code for supporting OpenFst < 1.5.3.
dogancan Dec 6, 2016
fb5b512
Remove obsolete OpenFst version checks in Darwin makefiles.
dogancan Dec 6, 2016
a41c27b
Remove support for OpenFst < 1.5.3 from tools/Makefile
dogancan Dec 6, 2016
0385303
Fix tools/Makefile to resolve travis failure.
dogancan Dec 6, 2016
c61296f
Fix src/configure and add C++11 flag to makefiles/*.mk
dogancan Dec 6, 2016
8599be4
Add check for compiler with C++11 support
dogancan Dec 7, 2016
0d7dbd6
Update installation instructions.
dogancan Dec 7, 2016
ef13e26
Remove a comment in tools/Makefile to resolve the build problem.
dogancan Dec 7, 2016
bd85a39
Add C++11 compliant compiler check and update installation instructions.
dogancan Dec 8, 2016
da7d11f
Refactor makefiles/*.mk
dogancan Dec 15, 2016
bd825b7
Clean up configure script.
dogancan Dec 15, 2016
bf8aad8
Update travis script.
dogancan Dec 15, 2016
bbb7f5d
Initialize a few variables to silence compiler warnings.
dogancan Dec 15, 2016
41c77be
Fix spacing error in configure.
dogancan Dec 16, 2016
817a314
Fix travis script.
dogancan Dec 16, 2016
20b5645
Yet another fix for the travis script.
dogancan Dec 16, 2016
b4cc589
One more fix to travis script.
dogancan Dec 16, 2016
77ec36a
Quote environment variables defined in travis script to resolve the b…
dogancan Dec 16, 2016
ec76eca
Fix the quoting in mtoken function defined in travis script.
dogancan Dec 16, 2016
e5a0120
Run tests in parallel to avoid the travis timeout.
dogancan Dec 16, 2016
75105e9
Reorganize platform specific makefiles.
dogancan Dec 21, 2016
ac480ac
Further changes to configure.
dogancan Dec 21, 2016
b37e06b
Configure script now accepts binary flags without the yes/no qualifie…
dogancan Dec 21, 2016
c386367
Small cosmetic changes to platform specific makefiles.
dogancan Dec 21, 2016
c03893f
Update installation instructions.
dogancan Dec 21, 2016
e725415
Cosmetic fixes.
dogancan Dec 21, 2016
f5b98f0
More cosmetic fixes.
dogancan Dec 21, 2016
1a512aa
Further cosmetic fixes.
dogancan Dec 21, 2016
6992617
Syncronize ppc64le configuration.
dogancan Dec 21, 2016
fed2288
Update .gitignore.
dogancan Dec 21, 2016
f31024e
Remove a few include guards that are no longer needed.
dogancan Dec 23, 2016
d037234
Upgrade codebase to support OpenFst-1.6.0.
dogancan Jan 9, 2017
5bdd430
Stop relinking dynamic libraries whenever they are updated.
dogancan Jan 9, 2017
2b6055e
Add -Wno-deprecated-declarations compiler flag to stop warnings about…
dogancan Jan 9, 2017
d94e77b
Fix test code to conform with OpenFst-1.6 API.
dogancan Jan 9, 2017
210b341
Add date/time info to travis script.
dogancan Jan 9, 2017
d5fa6cc
Testing Travis CI with different build settings.
dogancan Jan 10, 2017
9563cf1
Testing Travis CI with different build settings 2.
dogancan Jan 10, 2017
5c83a0d
Testing Travis CI with different build settings 3.
dogancan Jan 10, 2017
a32f1f8
Testing Travis CI with different build settings 4.
dogancan Jan 10, 2017
3017cbb
Testing Travis CI with different build settings 5.
dogancan Jan 10, 2017
097b825
Testing Travis CI with different build settings 6.
dogancan Jan 10, 2017
1d237a0
Add more specific compilation instructions in configure script
danpovey Jan 19, 2017
3379167
[src] Fix compilation issues on mac
danpovey Jan 19, 2017
2352321
[src] nnet3: removed the declaration of SetZero() in nnet-utils.h (#1…
freewym Jan 20, 2017
b91711c
[build] Enable Travis CI on the 'shortcut' branch (#1359)
kkm000 Jan 20, 2017
8d64376
[src] cudamatrix: modify test code to guarantee loop only run once if…
keli78 Jan 23, 2017
fd5238a
[src] nnet3: Extending nnet3-combine to support soft enforcement of s…
danpovey Jan 23, 2017
e744133
[src] Add more diagnostic output to lattice determinization programs
danpovey Jan 23, 2017
36bbf44
[src] Extend nnet3 Nnet reading code to accept .mdl files
danpovey Jan 23, 2017
6919a5a
[src][egs] Various script updates/clarifications, remove no-op option…
danpovey Jan 23, 2017
496eec5
[egs][scripts] Adding more example scripts for Tedlium and Swbd; add …
danpovey Jan 26, 2017
ac602ed
[src] nnet3: modifying nnet-combine.{h,cc} to support soft enforcemen…
danpovey Jan 23, 2017
7a69473
[src] nnet3: Changing from floating-point to double precision in nnet…
danpovey Jan 23, 2017
856db5e
[src] Adding chain version of the combination changes from the last 2…
danpovey Jan 25, 2017
b9adca3
python level implementation
GaofengCheng Jan 24, 2017
f2d999b
[egs]: fix some soft links
danpovey Jan 27, 2017
9333bcd
[src,egs,scripts]: improve use of sum-to-one penalty in combination, …
danpovey Jan 28, 2017
f583fae
Merging changes from master into upstream/shortcut
danpovey Jan 29, 2017
b78e5cc
[scripts] change default in nnet3/chain/build_tree.sh
danpovey Jan 30, 2017
d5bf81b
[scripts] various minor script fixes or extensions
danpovey Jan 30, 2017
1c8cba6
[egs] Adding various tuning scripts on tedlium.
danpovey Jan 31, 2017
3a8c0a1
[egs] egs/swbd/s5c, minor script updates and new tuning scripts.
danpovey Jan 31, 2017
7b6b690
[scripts] add utils/data/shift_feats.sh, deprecates steps/shift_feats…
hhadian Jan 31, 2017
3da8169
[egs] Small fixes/additions in Swbd/s5c chain scripts
danpovey Jan 31, 2017
7f38a5e
[src,egs,scripts]: various minor fixes: make num-epochs continuous; a…
danpovey Feb 1, 2017
ed2cedc
[egs,scripts]: add Swbd/s5c tuning scripts; simplify nnet3+chain 'com…
danpovey Feb 3, 2017
954815a
Resolve conflicts due to cross compilation changes in master (#1400)
dogancan Feb 8, 2017
e4474a8
[scripts,egs] ivector compatibility checks; minor fixes in egs (#1395)
jtrmal Feb 9, 2017
f81bc7a
[build] make the Makefile checks serial (#1409)
jtrmal Feb 9, 2017
a8cce10
Resolve merge conflicts and add "make ext" to travis build (#1407)
dogancan Feb 9, 2017
8acbbc9
[src,egs,scripts]: Replace online-nnet3 decoding setup with 'looped' …
danpovey Feb 9, 2017
a2e38fa
[scripts] Checking ivector extractor id, handle cases when ivector id…
jtrmal Feb 10, 2017
4226d50
[scripts] add empty-data checks in generate_plots.py (#1394)
keli78 Feb 11, 2017
288cf48
[scripts] Change how the --frame argument is set in non-recurrent DNN…
LvHang Feb 11, 2017
bd629d1
[scripts,build]: minor fixes only affecting error handling.
danpovey Feb 13, 2017
6e69654
[src,doc] Documentation updates; fixes to comments.
danpovey Feb 12, 2017
476c45d
[scripts] Removing tdnn-*-layer from xconfigs (redundant).
danpovey Feb 14, 2017
886e45a
[src] Add element-wise matrix min operation (#1424)
kangshiyin Feb 16, 2017
6070209
[src] Make various tests faster, especially without GPU (#1428)
danpovey Feb 17, 2017
4261d96
[src] Make chain-supervision-test.cc faster when no GPU
danpovey Feb 17, 2017
bdf205d
[build] Updating version file-- this commit marks version 5.1.0
danpovey Feb 17, 2017
0c831d6
[build] Change travis build to use clang; remove -rdynamic from CXXFL…
dogancan Feb 17, 2017
1ef3964
[scripts] fix syntax error in validate_lang.pl [thanks: daniel galvez]
danpovey Feb 19, 2017
9d34ff4
[build] Increase OpenFst version 1.6.0->1.6.1. (#1434)
dogancan Feb 20, 2017
1a7b72c
[build] Use github not sourceforge for IRSTLM [sourceforge repo dead?…
jtrmal Feb 20, 2017
e5304f8
[egs] clean up the HKUST scripts and add scoring filters (#1436)
jtrmal Feb 20, 2017
ac69d1a
[src,doc] Update version documentation with version 5.1; makefile fix…
danpovey Feb 21, 2017
dc7864a
[build] update .gitignore: ignore openfst-1.6.1 (#1439)
kangshiyin Feb 21, 2017
0e5cea8
[src,egs,scripts] Support frame-subsampling with non-chain nnet3 mode…
hhadian Feb 21, 2017
dda277d
[build] cloning IRSTLM, use https to access github, not ssh (#1441)
jtrmal Feb 21, 2017
1b5563c
[build] Update Travis configuration to get builds to complete on time…
dogancan Feb 22, 2017
fe4098c
[egs] Iban recipe: MacOSX compatibility fixes (#1448)
jtrmal Feb 22, 2017
f2b29c9
[scripts] Make it so i-vector ID is not required for steps/nnet3/deco…
danpovey Feb 23, 2017
ca3d53c
[build] Stop env.sh from crashing when set -u is active (#1451)
jtrmal Feb 23, 2017
6ed3996
upgrade the sox calls to use more compatible options (#1453)
jtrmal Feb 24, 2017
efe1074
[egs] fix typo in egs/swbd/s5c/local/nnet3/run_ivector_common.sh (#1…
megazone87 Feb 25, 2017
0f839a5
[scripts] xconfig: make scripts work when LDA-like preconditioning la…
megazone87 Feb 25, 2017
d6f6892
[build] Update OpenFst minimum version check to 1.6 in tools/Makefile…
dogancan Feb 25, 2017
5ddf62a
[src] Fix bug in decodable-online-looped.cc (prevent crash in nnet3 o…
danpovey Feb 25, 2017
21abc6f
[egs] fix MacOSX incompatibilities in calls of paste (#1457)
jtrmal Feb 26, 2017
dade1b0
[egs,scripts] Misc script fixes; refactor wsj/s5 examples; update ted…
danpovey Feb 27, 2017
8a5ee4a
[scripts,egs] Replace SGE-specific options to queue.pl (etc.) with ge…
jtrmal Feb 27, 2017
05463ac
[scripts] Fix to nnet2 script (won't change results) (#1462)
jtrmal Feb 28, 2017
65e8a96
[scripts] nnet3: remove import from tdnn in the xconfig script since …
freewym Feb 28, 2017
114abca
[egs] Add online-decoding example in some Swbd chain scripts (#1466)
hhadian Feb 28, 2017
2f9e095
[scripts] fix_data_dir.sh, prevent function returning false in case r…
jtrmal Feb 28, 2017
cceb9c1
[src] kwsbin/transcripts-to-fsts.cc, fix bug introduced in the last b…
jtrmal Mar 1, 2017
04bf5b0
[src] add CUDA kernel for backprop of NormalizeLayer (#1458)
kangshiyin Mar 3, 2017
55227f2
[src] fix kws pipeline after upgrading to openfst-1.6 (#1471)
jtrmal Mar 4, 2017
1047a5a
[egs] Add discriminative training script for WSJ/TDNN-LSTM (#1468)
hhadian Mar 4, 2017
7d78502
[src] bug-fix in gst plugin code (issue on g++ 5.4.0) (#1479)
qacollective Mar 7, 2017
02ca380
[scripts,egs] sMBR on LFR xent system with shifted feats (#1477)
hhadian Mar 7, 2017
b00b9e2
[egs] Minor cosmetic changes in voxforge example script (#1483)
baali Mar 9, 2017
0ebbc74
[egs] Fixes to URLs in vystadial example script.
baali Mar 10, 2017
7d79572
[src] nnet1: fixing issue in multi-task training (#1491)
KarelVesely84 Mar 13, 2017
7c171a5
[build] Bump OpenFst version to v1.6.2 (#1492)
dogancan Mar 14, 2017
9a61b88
[egs] swbd/chain : added blstm script using fast-LSTM; added BLSTM+TD…
vijayaditya Mar 16, 2017
68849f4
[egs] update fisher_swbd recipe (fixes to how things are installed). …
naxingyu Mar 18, 2017
bef410c
[src] sort cuda kernel function declarations to make searching easier…
kangshiyin Mar 19, 2017
041934f
[build] Android compilation, bug-fixes (#1502)
jcsilva Mar 19, 2017
f8f83ad
[doc] Add a note to README.md about Android cross compilation (#1503)
dogancan Mar 19, 2017
d38d067
[egs] ami : Added tdnn_lstm recipe with fast-lstmp layer. Added tdnn_…
vijayaditya Mar 21, 2017
0a6b38e
[scripts] prevent failure when final.ie.id doesn't exist (#1508)
jtrmal Mar 22, 2017
d954b93
[src] Fix exit code of extract-rows.cc (#1510)
happyalu Mar 23, 2017
5316e68
[egs] fixes to babel pipeline; thanks to Fred Richardson (#1509)
jtrmal Mar 23, 2017
db27674
[src,scripts]: Several unrelated cosmetic changes
danpovey Mar 23, 2017
ae82d90
[misc] remove eXecute permissions where not needed (#1515)
jtrmal Mar 27, 2017
e5d1de3
[egs] Fix to egs/wsj/s5/run.sh (unset variable) (#1517)
megazone87 Mar 28, 2017
e1b7916
[src] Adding noexcept to hashing function objects (#1519)
tomkocse Mar 29, 2017
7a73689
[src,doc] Fix several unrelated minor problems. Thanks: gaoxinglong
danpovey Mar 29, 2017
e72c15c
[src] (minor) Added missing SetZero() to NaturalGradientAffineCompone…
freewym Mar 31, 2017
c7edaec
[scripts,egs] Adding options for using PCA instead of LDA+MLLT for iv…
david-ryan-snyder Apr 2, 2017
c1e7b29
[build,src,doc] Modify get_version.sh to deal better with whitespace …
danpovey Apr 2, 2017
53d0e88
[build]: remove openfst check (#1531)
KarelVesely84 Apr 4, 2017
5ae74f1
[src] cudamatrix: speed up AddColSumMat with transfrom reduce kernel …
kangshiyin Apr 4, 2017
5ac45be
[src] Cosmetic change: remove 'train.tra' from usage messages (#1529)
hhadian Apr 4, 2017
5765819
[src] nnet1: improving the GPU diagnostics, (#1532)
KarelVesely84 Apr 5, 2017
19df56a
[src] Fix copy-feats for using the --write-num-frames and --compress …
psmit Apr 11, 2017
b1e6ec8
[scripts] fix to get_egs_targets.sh (thanks: David Pye)
danpovey Apr 12, 2017
6599c9b
[scripts] nnet3 : fix issue where LDA estimation failed for LSTMs wit…
vijayaditya Apr 12, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 9 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,14 @@ GSYMS
/tools/openfst-1.3.4/
/tools/openfst-1.4.1.tar.gz
/tools/openfst-1.4.1/
/tools/openfst-1.5.4.tar.gz
/tools/openfst-1.5.4/
/tools/openfst-1.6.0.tar.gz
/tools/openfst-1.6.0/
/tools/openfst-1.6.1.tar.gz
/tools/openfst-1.6.1/
/tools/openfst-1.6.2.tar.gz
/tools/openfst-1.6.2/
/tools/pa_stable_v19_20111121.tgz
/tools/portaudio/
/tools/sctk-2.4.0-20091110-0958.tar.bz2
Expand All @@ -114,6 +122,7 @@ GSYMS
/tools/pthreads
/tools/pthreads*.zip
/tools/sequitur
/tools/sequitur-g2p
/tools/srilm.tgz
/tools/liblbfgs-1.10.tar.gz
/tools/liblbfgs-1.10/
Expand All @@ -124,4 +133,3 @@ GSYMS
/tools/sequitur-g2p/

/kaldiwin_vs*

5 changes: 4 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,27 @@ addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-precise-3.8
packages:
- gdb
- gcc-4.9
- g++-4.9
- gfortran-4.9
- liblapack-dev
- clang-3.8

branches:
only:
- master
- shortcut

before_install:
- cat /proc/sys/kernel/core_pattern
- export XROOT=~/xroot
- tools/extras/travis_install_bindeps.sh $XROOT

script:
- CXX=g++-4.9
- CXX=clang++-3.8
CFLAGS="-march=native"
LDFLAGS="-llapack"
INCDIRS="$XROOT/usr/include"
Expand Down
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,25 +40,30 @@ Development pattern for contributors
------------------------------------

1. [Create a personal fork](https://help.github.com/articles/fork-a-repo/)
of the [main Kaldi repository] (https://github.com/kaldi-asr/kaldi) in GitHub.
of the [main Kaldi repository](https://github.com/kaldi-asr/kaldi) in GitHub.
2. Make your changes in a named branch different from `master`, e.g. you create
a branch `my-awesome-feature`.
3. [Generate a pull request](https://help.github.com/articles/creating-a-pull-request/)
through the Web interface of GitHub.
4. As a general rule, please follow [Google C++ Style Guide]
(https://google.github.io/styleguide/cppguide.html).
4. As a general rule, please follow [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).
There are a [few exceptions in Kaldi](http://kaldi-asr.org/doc/style.html).
You can use the [Google's cpplint.py]
(https://raw.githubusercontent.com/google/styleguide/gh-pages/cpplint/cpplint.py)
You can use the [Google's cpplint.py](https://raw.githubusercontent.com/google/styleguide/gh-pages/cpplint/cpplint.py)
to verify that your code is free of basic mistakes.

Platform specific notes
-----------------------

PowerPC 64bits little-endian (ppc64le):
### PowerPC 64bits little-endian (ppc64le)

- Kaldi is expected to work out of the box in RHEL >= 7 and Ubuntu >= 16.04 with
OpenBLAS, ATLAS, or CUDA.
- CUDA drivers for ppc64le can be found at [https://developer.nvidia.com/cuda-downloads]
(https://developer.nvidia.com/cuda-downloads).
- An [IBM Redbook] (https://www.redbooks.ibm.com/abstracts/redp5169.html) is
- CUDA drivers for ppc64le can be found at [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads).
- An [IBM Redbook](https://www.redbooks.ibm.com/abstracts/redp5169.html) is
available as a guide to install and configure CUDA.

### Android

- Kaldi supports cross compiling for Android using Android NDK, clang++ and
OpenBLAS.
- See [this blog post](http://jcsilva.github.io/2017/03/18/compile-kaldi-android/)
for details.
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_ihm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ sed -e 's?.*/??' -e 's?.wav??' $dir/wav.flist | \
awk '{print $2}' $dir/segments | sort -u | join - $dir/wav1.scp > $dir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# (1d) reco2file_and_channel
cat $dir/wav.scp \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_ihm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ sed -e 's?.*/??' -e 's?.wav??' $dir/wav.flist | \
awk '{print $2}' $dir/segments | sort -u | join - $dir/wav1.scp > $dir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# (1d) reco2file_and_channel
cat $dir/wav.scp \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_mdm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ awk '{print $1}' $dir/wav2.scp | join -2 2 - $dir/segments | \
awk '{print $1}' $dir/segments | join - $dir/text > $dir/t; mv $dir/t $dir/text

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

#prep reco2file_and_channel
cat $dir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_mdm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ sed -e 's?.*/??' -e 's?.wav??' $tmpdir/wav.flist | \
awk '{print $2}' $tmpdir/segments | sort -u | join - $tmpdir/wav1.scp > $tmpdir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp

#prep reco2file_and_channel
cat $tmpdir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_sdm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ awk '{print $1}' $dir/wav2.scp | join -2 2 - $dir/segments | \
awk '{print $1}' $dir/segments | join - $dir/text > $dir/t; mv $dir/t $dir/text

#replace path with an appropriate sox command that select a single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# this file reco2file_and_channel maps recording-id
cat $dir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5/local/ami_sdm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ sed -e 's?.*/??' -e 's?.wav??' $tmpdir/wav.flist | \
awk '{print $2}' $tmpdir/segments | sort -u | join - $tmpdir/wav1.scp > $tmpdir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp

#prep reco2file_and_channel
cat $tmpdir/wav.scp | \
Expand Down
14 changes: 7 additions & 7 deletions egs/ami/s5/local/online/run_nnet2_ms_perturbed.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# This example script demonstrates how speed perturbation of the data helps the nnet training in the SWB setup.

. ./cmd.sh
set -e
set -e
stage=1
train_stage=-10
use_gpu=true
Expand All @@ -27,13 +27,13 @@ fix_nnet=false

if $use_gpu; then
if ! cuda-compiled; then
cat <<EOF && exit 1
This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA
cat <<EOF && exit 1
This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA
If you want to use GPUs (and have them), go to src/, and configure and make on a machine
where "nvcc" is installed. Otherwise, call this script with --use-gpu false
EOF
fi
parallel_opts="--gpu 1"
parallel_opts="--gpu 1"
num_threads=1
minibatch_size=512
if [[ $(hostname -f) == *.clsp.jhu.edu ]]; then
Expand All @@ -52,7 +52,7 @@ else
# almost the same, but this may be a little bit slow.
num_threads=16
minibatch_size=128
parallel_opts="-pe smp $num_threads"
parallel_opts="--num-threads $num_threads"
fi

dir=exp/$mic/nnet2_online/nnet_ms_sp${affix:+_$affix}
Expand Down Expand Up @@ -155,7 +155,7 @@ fi
wait;

if [ $stage -le 12 ]; then
# do the actual online decoding with iVectors, carrying info forward from
# do the actual online decoding with iVectors, carrying info forward from
# previous utterances of the same speaker.
for decode_set in dev eval; do
(
Expand Down Expand Up @@ -191,7 +191,7 @@ if [ $stage -le 14 ]; then
steps/online/nnet2/decode.sh --config conf/decode.conf --cmd "$decode_cmd" --nj $num_jobs \
--per-utt true --online false $graph_dir data/$mic/${decode_set}_hires \
$decode_dir || exit 1;
) &
) &
done
fi
wait;
Expand Down
18 changes: 9 additions & 9 deletions egs/ami/s5/local/online/run_nnet2_ms_sp_disc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# note: this relies on having a cluster that has plenty of CPUs as well as GPUs,
# since the lattice generation runs in about real-time, so takes of the order of
# 1000 hours of CPU time.
#
#
# Note: rather than using any features we have dumped on disk, this script
# regenerates them from the wav data three times-- when we do lattice
# generation, numerator alignment and discriminative training. This made the
Expand Down Expand Up @@ -42,19 +42,19 @@ set -e

if $use_gpu; then
if ! cuda-compiled; then
cat <<EOF && exit 1
This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA
cat <<EOF && exit 1
This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA
If you want to use GPUs (and have them), go to src/, and configure and make on a machine
where "nvcc" is installed. Otherwise, call this script with --use-gpu false
EOF
fi
parallel_opts=" -l gpu=1,hostname='!g01*&!g02*' " #we want to submit to all.q as we use multiple GPUs for this
parallel_opts="--gpu 1" #we want to submit to all.q as we use multiple GPUs for this
num_threads=1
else
# Use 4 nnet jobs just like run_4d_gpu.sh so the results should be
# almost the same, but this may be a little bit slow.
num_threads=16
parallel_opts="-pe smp $num_threads"
parallel_opts="--num-threads $num_threads"
fi

if [ -z $srcdir ]; then
Expand All @@ -76,7 +76,7 @@ if [ $stage -le 1 ]; then
num_threads_denlats=6
subsplit=40 # number of jobs that run per job (but 2 run at a time, so total jobs is 80, giving
# max total slots = 80 * 6 = 480.
steps/nnet2/make_denlats.sh --cmd "$decode_cmd -l mem_free=1G,ram_free=1G -pe smp $num_threads_denlats" \
steps/nnet2/make_denlats.sh --cmd "$decode_cmd --mem 1G --num-threads $num_threads_denlats" \
--online-ivector-dir exp/$mic/nnet2_online/ivectors_train_hires_sp2 \
--nj $nj --sub-split $subsplit --num-threads "$num_threads_denlats" --config conf/decode.conf \
data/$mic/train_hires_sp data/lang $srcdir ${srcdir}_denlats || exit 1;
Expand Down Expand Up @@ -142,7 +142,7 @@ if [ $stage -le 5 ]; then
(
num_jobs=`cat data/$mic/${decode_set}_hires/utt2spk|cut -d' ' -f2|sort -u|wc -l`
decode_dir=$dir/decode_epoch${epoch}_${decode_set}_utt

steps/online/nnet2/decode.sh --config conf/decode.conf --cmd "$decode_cmd" --nj $num_jobs \
--per-utt true --iter epoch$epoch $graph_dir data/$mic/${decode_set}_hires $decode_dir || exit 1
) &
Expand All @@ -154,13 +154,13 @@ if [ $stage -le 5 ]; then
(
num_jobs=`cat data/$mic/${decode_set}_hires/utt2spk|cut -d' ' -f2|sort -u|wc -l`
decode_dir=$dir/decode_epoch${epoch}_${decode_set}_utt_offline

steps/online/nnet2/decode.sh --config conf/decode.conf --cmd "$decode_cmd" --nj $num_jobs \
--per-utt true --online false --iter epoch$epoch $graph_dir data/$mic/${decode_set}_hires $decode_dir || exit 1
) &
done
done

wait
fi

Expand Down
21 changes: 12 additions & 9 deletions egs/ami/s5b/RESULTS_ihm
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@
%WER 24.0 | 13098 94470 | 79.4 12.1 8.5 3.4 24.0 57.1 | -0.153 | exp/ihm/nnet3_cleaned/tdnn_sp/decode_dev/ascore_12/dev_hires.ctm.filt.sys
%WER 25.5 | 12643 89984 | 77.7 14.2 8.2 3.2 25.5 56.4 | -0.139 | exp/ihm/nnet3_cleaned/tdnn_sp/decode_eval/ascore_11/eval_hires.ctm.filt.sys


# local/nnet3/run_tdnn.sh --mic ihm --train-set train --gmm tri3 --nnet3-affix ""
# nnet3 xent TDNN without data cleaning [cleaning makes very small and
# inconsistent difference on this dat]
Expand All @@ -55,17 +54,21 @@
%WER 22.4 | 12643 89977 | 80.3 12.5 7.2 2.7 22.4 53.6 | -0.503 | exp/ihm/nnet3_cleaned/lstm_bidirectional_sp/decode_eval/ascore_10/eval_hires.ctm.filt.sys

############################################

# local/chain/run_tdnn.sh --mic ihm --stage 12 &
# cleanup + chain TDNN model
# for d in exp/ihm/chain_cleaned/tdnn_sp_bi/decode_*; do grep Sum $d/*sc*/*ys | utils/best_wer.sh; done
%WER 22.5 | 13098 94490 | 80.6 10.8 8.6 3.1 22.5 55.0 | 0.072 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.5 | 12643 89978 | 80.3 12.5 7.2 2.7 22.5 53.1 | 0.149 | exp/ihm/chain_cleaned/tdnn_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys

# cleanup + chain TDNN model.
# local/chain/run_tdnn.sh --mic ihm --stage 4 &
# for d in exp/ihm/chain_cleaned/tdnn1d_sp_bi/decode_*; do grep Sum $d/*sc*/*ys | utils/best_wer.sh; done
%WER 21.7 | 13098 94488 | 81.1 10.4 8.4 2.8 21.7 54.4 | 0.096 | exp/ihm/chain_cleaned/tdnn1d_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.1 | 12643 89979 | 80.5 12.1 7.4 2.6 22.1 52.8 | 0.185 | exp/ihm/chain_cleaned/tdnn1d_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys

# cleanup + chain TDNN model. Uses LDA instead of PCA for ivector features.
# local/chain/tuning/run_tdnn_1b.sh --mic ihm --stage 4 &
# for d in exp/ihm/chain_cleaned/tdnn1b_sp_bi/decode_*; do grep Sum $d/*sc*/*ys | utils/best_wer.sh; done
%WER 22.0 | 13098 94488 | 80.8 10.2 9.0 2.8 22.0 54.7 | 0.102 | exp/ihm/chain_cleaned/tdnn1b_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.2 | 12643 89968 | 80.3 12.1 7.6 2.6 22.2 52.9 | 0.170 | exp/ihm/chain_cleaned/tdnn1b_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys

# local/chain/run_tdnn.sh --mic ihm --train-set train --gmm tri3 --nnet3-affix "" --stage 12
# chain TDNN model without cleanup [note: cleanup helps very little on this IHM data.]
for d in exp/ihm/chain/tdnn_sp_bi/decode_*; do grep Sum $d/*sc*/*ys | utils/best_wer.sh; done
# for d in exp/ihm/chain/tdnn_sp_bi/decode_*; do grep Sum $d/*sc*/*ys | utils/best_wer.sh; done
%WER 22.4 | 13098 94476 | 80.4 10.4 9.2 2.8 22.4 54.6 | 0.069 | exp/ihm/chain/tdnn_sp_bi/decode_dev/ascore_10/dev_hires.ctm.filt.sys
%WER 22.5 | 12643 89974 | 80.0 12.1 7.9 2.6 22.5 52.8 | 0.157 | exp/ihm/chain/tdnn_sp_bi/decode_eval/ascore_10/eval_hires.ctm.filt.sys

Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_ihm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ sed -e 's?.*/??' -e 's?.wav??' $dir/wav.flist | \
awk '{print $2}' $dir/segments | sort -u | join - $dir/wav1.scp > $dir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# (1d) reco2file_and_channel
cat $dir/wav.scp \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_ihm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ sed -e 's?.*/??' -e 's?.wav??' $dir/wav.flist | \
awk '{print $2}' $dir/segments | sort -u | join - $dir/wav1.scp > $dir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# (1d) reco2file_and_channel
cat $dir/wav.scp \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_mdm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ awk '{print $1}' $dir/wav2.scp | join -2 2 - $dir/segments | \
awk '{print $1}' $dir/segments | join - $dir/text > $dir/t; mv $dir/t $dir/text

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

#prep reco2file_and_channel
cat $dir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_mdm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ sed -e 's?.*/??' -e 's?.wav??' $tmpdir/wav.flist | \
awk '{print $2}' $tmpdir/segments | sort -u | join - $tmpdir/wav1.scp > $tmpdir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp

#prep reco2file_and_channel
cat $tmpdir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_sdm_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ awk '{print $1}' $dir/wav2.scp | join -2 2 - $dir/segments | \
awk '{print $1}' $dir/segments | join - $dir/text > $dir/t; mv $dir/t $dir/text

#replace path with an appropriate sox command that select a single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $dir/wav2.scp > $dir/wav.scp

# this file reco2file_and_channel maps recording-id
cat $dir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/ami_sdm_scoring_data_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ sed -e 's?.*/??' -e 's?.wav??' $tmpdir/wav.flist | \
awk '{print $2}' $tmpdir/segments | sort -u | join - $tmpdir/wav1.scp > $tmpdir/wav2.scp

#replace path with an appropriate sox command that select single channel only
awk '{print $1" sox -c 1 -t wavpcm -s "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp
awk '{print $1" sox -c 1 -t wavpcm -e signed-integer "$2" -t wavpcm - |"}' $tmpdir/wav2.scp > $tmpdir/wav.scp

#prep reco2file_and_channel
cat $tmpdir/wav.scp | \
Expand Down
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/chain/run_tdnn.sh
2 changes: 1 addition & 1 deletion egs/ami/s5b/local/chain/run_tdnn_lstm.sh
Loading