Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for newlines in gettext catalogs #20815

Conversation

mzazrivec
Copy link
Contributor

@mzazrivec mzazrivec commented Nov 13, 2020

It seems that with lately added translation we have a new problem in our gettext catalogs: the translations do not honor newlines at string starts and string ends.

For example, from locale/es/manageiq.po:

msgid "Copy from Provisioning"
msgstr "Copiar desde Aprovisionamiento\\n"

Same thing shown in UI:
Screenshot from 2020-11-13 18-25-01

The rules that we need to enforce here:

  1. if original string starts with a newline, translation needs to start with a newline too
  2. if original string ends with a newline, translated string also needs to end with a newline
  3. overal amount of newlines in original and translated string needs to match

In this PR, I am introducing a new test, which tests the above rules. The exception is the 3rd rule, since it seems that in some places the newline might have been added (or omitted) intentionaly. So we're just logging these.

This PR also fixes strings in catalogs that break the above rules.

@miq-bot add_label internationalization, test, wip

@mzazrivec mzazrivec force-pushed the add_tests_for_newlines_in_gettext_catalogues branch 4 times, most recently from ded4e64 to 37af885 Compare November 13, 2020 18:47
@mzazrivec mzazrivec force-pushed the add_tests_for_newlines_in_gettext_catalogues branch from 37af885 to 13cbfc9 Compare November 20, 2020 14:38
@mzazrivec mzazrivec changed the title [WIP] Add tests for newlines in gettext catalogs Add tests for newlines in gettext catalogs Nov 20, 2020
@miq-bot miq-bot removed the wip label Nov 20, 2020
@mzazrivec mzazrivec force-pushed the add_tests_for_newlines_in_gettext_catalogues branch from 13cbfc9 to 6eb704d Compare November 20, 2020 14:40
@mzazrivec
Copy link
Contributor Author

@miq-bot remove_label unmergeable

@mzazrivec
Copy link
Contributor Author

@himdel review please?

@@ -47739,7 +47739,7 @@ msgstr "Esta funcionalidad no se admite en la versión API del proveedor"
#:
#: ../app/models/manageiq/providers/redhat/infra_manager/vm_or_template_shared/scanning.rb:51
msgid "VMs must be scanned from an EVM server whose host is attached to the same\n storage as the VM unless overridden via SmartProxy affinity.\n Please verify that:\n 1) Direct LUNs are attached to ManageIQ appliance\n 2) Management Relationship is set for the ManageIQ appliance"
msgstr "Se deben analizar las MV desde un servidor EVM cuyo host esté conectado al mismo almacenamiento que la MV, a menos que se anule mediante afinidad con SmartProxy.\n Compruebe que:\n 1) Los LUN directos estén conectados a un dispositivo ManageIQ\n 2) La relación de gestión esté configurada para el dispositivo ManageIQ"
msgstr "Se deben analizar las MV desde un servidor EVM cuyo host esté conectado al mismo\n almacenamiento que la MV, a menos que se anule mediante afinidad con SmartProxy.\n Compruebe que:\n 1) Los LUN directos estén conectados a un dispositivo ManageIQ\n 2) La relación de gestión esté configurada para el dispositivo ManageIQ"
Copy link
Contributor

@himdel himdel Nov 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random newline in the middle of a sentence? (we should probably only keep newlines after : or between sentences, right?

EDIT: never mind, this is already the case in the original msgid (newline in the middle of a sentence is rather suprrising, but it's not related to this PR)

end

expect(boundary_errors).to be_empty
# We intentionaly do not expect overal_errors to be empty, since in certain cases it may be desirable
Copy link
Contributor

@himdel himdel Nov 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to parametrize this?

If the tests don't go red on overall_errors, would we notice them? (Maybe we should skip it just in specific repos or for specific messages?)

EDIT: the issue is really that we have some newlines inside english text, that absolutely don't make sense to preserve in (shorter) japanese translations. So any parametrization would have to be per-language.

Copy link
Member

@Fryguy Fryguy Nov 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo overal_errors -> overall_errors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fryguy fixed, thanks!

Copy link
Contributor

@himdel himdel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, does what it says on the tin :)

@Fryguy
Copy link
Member

Fryguy commented Nov 23, 2020

@chessbyte Please review.

@mzazrivec mzazrivec force-pushed the add_tests_for_newlines_in_gettext_catalogues branch from 6eb704d to c88288e Compare November 24, 2020 12:58
@miq-bot
Copy link
Member

miq-bot commented Nov 24, 2020

Some comments on commits mzazrivec/manageiq@f480af2~...c88288e

spec/shared/i18n/newlines.rb

  • ⚠️ - 38 - Detected puts. Remove all debugging statements.
  • ⚠️ - 40 - Detected puts. Remove all debugging statements.
  • ⚠️ - 42 - Detected puts. Remove all debugging statements.
  • ⚠️ - 44 - Detected puts. Remove all debugging statements.

@miq-bot
Copy link
Member

miq-bot commented Nov 24, 2020

Checked commits mzazrivec/manageiq@f480af2~...c88288e with ruby 2.6.3, rubocop 0.82.0, haml-lint 0.35.0, and yamllint
2 files checked, 0 offenses detected
Everything looks fine. 🍰

@mzazrivec
Copy link
Contributor Author

@chessbyte review / merge please? Thanks.

@chessbyte chessbyte merged commit d073cd9 into ManageIQ:master Nov 30, 2020
@mzazrivec mzazrivec deleted the add_tests_for_newlines_in_gettext_catalogues branch November 30, 2020 15:17
jrafanie added a commit to jrafanie/manageiq that referenced this pull request Sep 27, 2024
Technically, we could try to fix this and get the translators to update their data to have
this fixed, but it's possible there are situations where this happens again and it's not
an actual problem for gettext.

Even the original PR that added this said this overall number of newlines check
could flag intentionally changed values.  Since we've never reacted to this output,
we can remove it.

From that PR (we care about the 3rd check below):

1. if original string starts with a newline, translation needs to start with a newline too
2. if original string ends with a newline, translated string also needs to end with a newline
3. overal amount of newlines in original and translated string needs to match

In this PR, I am introducing a new test, which tests the above rules. The exception is the
3rd rule, since it seems that in some places the newline might have been added (or omitted)
intentionaly. So we're just logging these.

Reference:: ManageIQ#20815
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants