Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

be more careful about final LUs in analysis mode (#104) #105

Merged
merged 1 commit into from
Oct 21, 2020

Conversation

mr-martian
Copy link
Contributor

I think this fixes #104 and tests pass, but I'm not totally certain whether any weird things might happen due to, say, stepping by null.

@unhammer
Copy link
Member

There are tests for NUL flushing, so it's probably ok, but perhaps try checking that it works with the rest of the pipeline by running a local apy instance? (put your version of lt-proc in PATH, apt install apertium-apy apertium-spa-cat and curl "http://localhost:2737/translate?markUnknown=yes&langpair=spa|cat&q=sometest")

@TinoDidriksen
Copy link
Member

From testing a larger corpus, it seems to work. There's only extra words in the output - no changed ones.

@TinoDidriksen TinoDidriksen merged commit 56bbe28 into master Oct 21, 2020
@TinoDidriksen TinoDidriksen deleted the anti-swallow branch October 21, 2020 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

May swallow last lexical unit if no final newline
3 participants