Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lexiconfree beam search #101

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Lexiconfree beam search #101

wants to merge 9 commits into from

Conversation

SimBe195
Copy link
Collaborator

Simple time synchronous beam search algorithm based on the new SearchAlgorithmV2 interface. Does not use (proper) pronunciation lexicon, word-level LM or transition model. Performs special handling of blank if a blank index is set. Main purpose is open vocabulary search with CTC/Neural Transducer (or similar) models.

Supports global pruning by max beam-size and by score difference to the best hypothesis. Uses a LabelScorer to context initialization/extension and scoring.

The search requires a lexicon that represents the vocabulary. Each lemma is viewed as a token with its index in the lexicon corresponding to the associated output index of the LabelScorer.

@larissakl
Copy link
Contributor

Two general points:

  1. How should this search algorithm be used? Are you planning to put the enum SearchTypeV2, the related ParameterChoice and Module_::createSearchAlgorithm() to Module.hh/.cc later? Or do you have a different plan?
  2. Would it maybe be a good idea to factor out the struct TimeStatistic and its related code so that it can be reused in other search algorithms?

@SimBe195
Copy link
Collaborator Author

SimBe195 commented Feb 20, 2025

1. How should this search algorithm be used? Are you planning to put the `enum SearchTypeV2`, the related `ParameterChoice` and `Module_::createSearchAlgorithm()` to Module.hh/.cc later? Or do you have a different plan?

With this PR alone, the search algorithm is not usable yet. I will make PR's for an Flf node and python bindings separately. But I can include the createSearchAlgorithm() function already here.

2. Would it maybe be a good idea to factor out the `struct TimeStatistic` and its related code so that it can be reused in other search algorithms?

Yeah, probably. Maybe even into Core?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants