Merge pull request #10 from richirisu/feature-autodetect-language

Auto-detect language from filename

docs/language support.md

@@ -1,31 +1,90 @@
 # Language Support
 
 - Original Japanese Release
-- English Translation V1.01 by Neill Corlett 
-- French Translation RC1 by Terminus Traduction
-- German Translation V1.00 RC3 to V3.0 by G-Trans
-- Italian Translation V1.00b  by Clomax, Ombra, Chester
+- English Translation V1.01 by Neill Corlett, SoM2Freak
+- French Translation RC1 by Terminus Traduction (Copernic)
+- German Translation V1.00 RC3 to V3.0 by G-Trans (Special-Man, LavosSpawn)
+- Italian Translation V1.00b by Mumble Translations (Clomax, Ombra, Chester)
 - Spanish Translation V1.03 by Magno, Vegetal Gibber
 
+## Automatic Language Detection
+
+The filename is being used to determine the translation patch. Unfortunately, there is no way to detect the translation patch from the save file data directly, so the filename has to suffice.
+
+Please refer to the source code for the actual regular expressions.
+
+### Detection Order
+
+1. Original Japanese ROM name without any translation information (Japanese)
+
+ | | |
+ | ---| --- |
+ | **Japanese** | Seiken Densetsu 3 (J) <br/> Seiken Densetsu 3 (Japan) |
+
+2. Patch name (English, French, German, Italian, Spanish, incl. Japanese)
+
+ | | |
+ | ---| --- |
+ | **English** | SD3EN101.IPS |
+ | **French** | SEIKEN3F.IPS |
+ | **German** | SEIKEN3D.IPS <br/> SD3GER203.IPS <br/> SD3DE30.IPS |
+ | **Italian** | SD3_JAP_ITA_V100_BETA_6A93E9F.IPS <br/> SD3_ENG_ITA_V100_BETA_6A93E9F.IPS |
+ | **Spanish** | SOM2SP.IPS |
+
+ So, either one of `SD3` or `SOM2` or `SEIKEN3`, followed by a one to three letter language code, followed by an optional version number.
+
+ Only Italian differs from this pattern slightly.
+
+3. Translator / translation team (English, French, German, Italian, Spanish)
+
+ | | |
+ | ---| --- |
+ | **English** | Neill Corlett, SoM2Freak |
+ | **French** | Terminus Traduction, Copernic |
+ | **German** | G-Trans, Special-Man, LavosSpawn |
+ | **Italian** | Mumble Translations, Clomax, Ombra, Chester |
+ | **Spanish** | Magno, Vegetal Gibber |
+
+4. Language / language code (English, French, German, Italian, Spanish, incl. Japanese)
+
+ | | |
+ | ---| --- |
+ | **English** | en, eng, english |
+ | **French** | fr, fra, french, français, francais |
+ | **German** | de, deu, ger, german, deutsch |
+ | **Italian** | it, ita, italian, italiano |
+ | **Japanese** | ja, jp, jap, japanese |
+ | **Spanish** | es, sp, esp, spa, spanish, español, espanol, castellano |
+
+ Only two and three letter language codes are taken into account here, no single letter codes.
+
+5. Fallback to English
+
+### Remarks
+
+- File names are treated case-insensitive.
+- There is also a pre-patched ROM circulating named `Seiken Densetsu 3 (Japan) [En by LNF+Neill Corlett+SoM2Freak v1.01]`. This is covered by rule 3 and 4.
+- The mere name `SEIKEN3` does not provide enough information to determine the language. Most likely it will be English. In any case, according to the above rule-set, the last rule will apply with English as fallback.
+
 ## UI Changes
 
-You can select the language of the cartridge from the combo box at the top. Unfortunately, there is no way to automatically detect the language from just the save file.
+When loading a save file, the program will automatically try to determine the used translation patch. In addition, you can also select the translation patch manually from the combo box at the top.
 
-It is recommended to choose the right language before loading the save file. If you change the language afterwards, the current player names will automatically be mapped to the new encoding, leading to a possible drop of unsupported characters. If this happens, just load the save file again.
+If you change the translation patch after having loaded a save file, the current player names will automatically be mapped to the new encoding, thus, leading to a possible drop of unsupported characters. If this happens, just load the save file again.
 
-Also, the program will check for unsupported characters on input, which means you can only input valid characters for the currently selected language. For compatibility both half-width and full-width characters are supported.
+Also, the program will check for unsupported characters on input, which means you can only enter valid characters from the currently selected translation patch. Although, most ASCII characters are present in all language encodings.
 
-Besides, most ASCII characters are present in all language encodings.
+For compatibility both half-width and full-width characters are supported.
 
 A player's name is limited to a maximum length of 6 characters.
 
 ## Remarks
 
-There are currently two encoding files for Japanese to Unicode: One with full-width encoding and the other with half-width encoding in Unicode. Both work equally well. For now I went with full-width.
+There are currently two encoding files to choose from when converting from Japanese to Unicode: One with full-width encoding in Unicode and the other one with half-width encoding in Unicode. Both work equally well. For now I went with full-width.
 
 You can change this by simply overwriting `encoding_japanese_to_unicode.json` with either `encoding_japanese_to_unicode_halfwidth.json` or `encoding_japanese_to_unicode_fullwidth.json`.
 
-Besides, when converting from Unicode to Japanese, both half-width and full-width characters are supported regardlessly.
+Despite, when converting from Unicode to Japanese, both half-width and full-width characters are supported regardlessly.
 
 ## Known Issues
 

sd3save_editor/gui/mainwindow.py

@@ -6,9 +6,13 @@
 from sd3save_editor.gui.datatype import (ComboBoxElement, LineEditElement,
  SpinboxElement)
 from sd3save_editor.save import NameTooLongException
+from sd3save_editor.save import Language
 import sd3save_editor.save as save
 import sd3save_editor.game_data as game_data
 
+from pathlib import Path
+import re
+
 
 class MainWindow(QMainWindow):
  def __init__(self):
@@ -18,6 +22,8 @@ def __init__(self):
  self.initFileOpenEvents()
  self.initChangeNameInput()
  self.initLanguageComboBox()
+ self.autoLanguage = Language.ENGLISH;
+ self.updateLanguageComboBox();
  self.initSaveEvent()
  self.saveIndex = None
  self.guiData = {
@@ -133,8 +139,15 @@ def initChangeNameInput(self):
  def initLanguageComboBox(self):
  self.ui.languageComboBox.activated.connect(self.languageChanged)
 
+ def updateLanguageComboBox(self):
+ text = self.ui.languageComboBox.itemText(self.autoLanguage.value);
+ self.ui.languageComboBox.setItemText(0, "Auto -- {}".format(text))
+
  def languageChanged(self, index):
- save.char_name_language = save.Language(index + 1)
+ if (index == 0):
+ save.char_name_language = self.autoLanguage
+ else:
+ save.char_name_language = Language(index)
  self.validateCharacterNames()
 
  def validateCharacterNames(self):
@@ -195,6 +208,13 @@ def openFileDialog(self):
  'Open file',
  filter="Seiken3 Save (*.srm)"
  ))[0]
+ if self.filename:
+ self.autoLanguage = self.detectLanguage(Path(self.filename).stem)
+ self.updateLanguageComboBox()
+ if self.ui.languageComboBox.currentIndex() == 0:
+ save.char_name_language = Language(self.autoLanguage.value)
+ else:
+ save.char_name_language = Language(self.ui.languageComboBox.currentIndex())
  if self.filename:
  try:
  self.saveData = save.read_save(self.filename)
@@ -207,4 +227,55 @@ def openFileDialog(self):
  self.setTableData()
  self.initSaveEntryComboBox()
  except Exception as ex:
+ self.ui.saveButton.setEnabled(False)
+ self.ui.actionSave.setEnabled(False)
  QMessageBox.warning(self, "Can't open Seiken3 save", str(ex))
+
+ @staticmethod
+ def detectLanguage(name):
+ # orignal Japanese release
+ if (re.search(r"\ASeiken Densetsu 3 \((J|Japan)\)\Z", name, re.IGNORECASE)):
+ return Language.JAPANESE
+ # translation patches
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(E|EN|ENG)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.ENGLISH
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(F|FR|FRA)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.FRENCH
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(D|G|DE|DEU|GER)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.GERMAN
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(I|IT|ITA)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.ITALIAN
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(J|JA|JP|JAP)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.JAPANESE
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)(S|ES|SP|ESP|SPA)[0-9]*\Z", name, re.IGNORECASE)):
+ return Language.SPANISH
+ # translation patches (Italian exception)
+ if (re.search(r"\A(SD3|SOM2|SEIKEN3)_(JAP|ENG)_ITA_", name, re.IGNORECASE)):
+ return Language.ITALIAN
+ # translation authors
+ if (re.search(r"(\A|\W)(neill|corlett|som2freak)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.ENGLISH
+ if (re.search(r"(\A|\W)(terminus|copernic)\W", name, re.IGNORECASE)):
+ return Language.FRENCH
+ if (re.search(r"(\A|\W)(g-trans|special-man|lavosspawn)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.GERMAN
+ if (re.search(r"(\A|\W)(mumble|clomax|ombra|chester)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.ITALIAN
+ if (re.search(r"(\A|\W)(magno|vegetal|gibber)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.SPANISH
+ # language codes
+ if (re.search(r"(\A|\W)(en|eng|english)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.ENGLISH
+ if (re.search(r"(\A|\W)(fr|fra|french|français|francais)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.FRENCH
+ if (re.search(r"(\A|\W)(de|deu|ger|german|deutsch)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.GERMAN
+ if (re.search(r"(\A|\W)(it|ita|italian|italiano)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.ITALIAN
+ if (re.search(r"(\A|\W)(ja|jp|jap|japanese)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.JAPANESE
+ if (re.search(r"(\A|\W)(es|sp|esp|spa|spanish|español|espanol|castellano)(\Z|\W)", name, re.IGNORECASE)):
+ return Language.SPANISH
+ # fallback
+ return Language.ENGLISH
+

sd3save_editor/gui/mainwindow.ui

@@ -399,40 +399,45 @@
  <item row="0" column="0">
  <widget class="QLabel" name="languageLabel">
  <property name="text">
- <string>Language</string>
+ <string>Translation Patch</string>
  </property>
  </widget>
  </item>
  <item row="0" column="1">
  <widget class="QComboBox" name="languageComboBox">
  <item>
  <property name="text">
- <string>English</string>
+ <string>Auto</string>
  </property>
  </item>
  <item>
  <property name="text">
- <string>French</string>
+ <string>English by Neill Corlett V1.01</string>
  </property>
  </item>
  <item>
  <property name="text">
- <string>German</string>
+ <string>French by Terminus Traduction RC1</string>
  </property>
  </item>
  <item>
  <property name="text">
- <string>Italian</string>
+ <string>German by G-Trans V1.00 RC3 to V3.0</string>
  </property>
  </item>
  <item>
  <property name="text">
- <string>Japanese</string>
+ <string>Italian by Mumble Translations V1.00b</string>
  </property>
  </item>
  <item>
  <property name="text">
- <string>Spanish</string>
+ <string>Japanese (Original Release)</string>
+ </property>
+ </item>
+ <item>
+ <property name="text">
+ <string>Spanish by Magno and Vegetal Gibber</string>
  </property>
  </item>
  </widget>

sd3save_editor/gui/mainwindow_ui.py

@@ -236,6 +236,7 @@ def setupUi(self, MainWindow):
  self.languageComboBox.addItem("")
  self.languageComboBox.addItem("")
  self.languageComboBox.addItem("")
+ self.languageComboBox.addItem("")
  self.formLayout_2.setWidget(0, QtWidgets.QFormLayout.FieldRole, self.languageComboBox)
  self.saveEntryLabel = QtWidgets.QLabel(self.centralwidget)
  self.saveEntryLabel.setObjectName("saveEntryLabel")
@@ -303,13 +304,14 @@ def retranslateUi(self, MainWindow):
  item.setText(_translate("MainWindow", "Amount"))
  self.tabsOverview.setTabText(self.tabsOverview.indexOf(self.tab_5), _translate("MainWindow", "Item Storage"))
  self.saveButton.setText(_translate("MainWindow", "Save"))
- self.languageLabel.setText(_translate("MainWindow", "Language"))
- self.languageComboBox.setItemText(0, _translate("MainWindow", "English"))
- self.languageComboBox.setItemText(1, _translate("MainWindow", "French"))
- self.languageComboBox.setItemText(2, _translate("MainWindow", "German"))
- self.languageComboBox.setItemText(3, _translate("MainWindow", "Italian"))
- self.languageComboBox.setItemText(4, _translate("MainWindow", "Japanese"))
- self.languageComboBox.setItemText(5, _translate("MainWindow", "Spanish"))
+ self.languageLabel.setText(_translate("MainWindow", "Translation Patch"))
+ self.languageComboBox.setItemText(0, _translate("MainWindow", "Auto"))
+ self.languageComboBox.setItemText(1, _translate("MainWindow", "English by Neill Corlett V1.01"))
+ self.languageComboBox.setItemText(2, _translate("MainWindow", "French by Terminus Traduction RC1"))
+ self.languageComboBox.setItemText(3, _translate("MainWindow", "German by G-Trans V1.00 RC3 to V3.0"))
+ self.languageComboBox.setItemText(4, _translate("MainWindow", "Italian by Mumble Translations V1.00b"))
+ self.languageComboBox.setItemText(5, _translate("MainWindow", "Japanese (Original Release)"))
+ self.languageComboBox.setItemText(6, _translate("MainWindow", "Spanish by Magno and Vegetal Gibber"))
  self.saveEntryLabel.setText(_translate("MainWindow", "Save Entry"))
  self.menuFile.setTitle(_translate("MainWindow", "File"))
  self.actionOpen.setText(_translate("MainWindow", "Open"))

test/test_language.py

@@ -0,0 +1,65 @@
+import os
+import pytest
+
+from sd3save_editor.gui.mainwindow import MainWindow
+from sd3save_editor.save import Language
+
+
+def test_detect_language_from_filename():
+ assert MainWindow.detectLanguage("") == Language.ENGLISH # fallback
+ assert MainWindow.detectLanguage("Something") == Language.ENGLISH # fallback
+ assert MainWindow.detectLanguage("SEIKEN3") == Language.ENGLISH # fallback
+ assert MainWindow.detectLanguage("SD3EN101") == Language.ENGLISH
+ assert MainWindow.detectLanguage("SEIKEN3F") == Language.FRENCH
+ assert MainWindow.detectLanguage("SEIKEN3D") == Language.GERMAN
+ assert MainWindow.detectLanguage("SD3GER203") == Language.GERMAN
+ assert MainWindow.detectLanguage("SD3DE30") == Language.GERMAN
+ assert MainWindow.detectLanguage("SD3_JAP_ITA_V100_BETA_6A93E9F") == Language.ITALIAN
+ assert MainWindow.detectLanguage("SD3_ENG_ITA_V100_BETA_6A93E9F") == Language.ITALIAN
+ assert MainWindow.detectLanguage("SOM2SP") == Language.SPANISH
+ assert MainWindow.detectLanguage("Seiken Densetsu 3 (J)") == Language.JAPANESE
+ assert MainWindow.detectLanguage("Seiken Densetsu 3 (Japan)") == Language.JAPANESE
+ assert MainWindow.detectLanguage("Seiken Densetsu 3 (Japan) [En by LNF+Neill Corlett+SoM2Freak v1.01]") == Language.ENGLISH
+ assert MainWindow.detectLanguage("english") == Language.ENGLISH
+ assert MainWindow.detectLanguage("french") == Language.FRENCH
+ assert MainWindow.detectLanguage("français") == Language.FRENCH
+ assert MainWindow.detectLanguage("francais") == Language.FRENCH
+ assert MainWindow.detectLanguage("german") == Language.GERMAN
+ assert MainWindow.detectLanguage("deutsch") == Language.GERMAN
+ assert MainWindow.detectLanguage("italian") == Language.ITALIAN
+ assert MainWindow.detectLanguage("italiano") == Language.ITALIAN
+ assert MainWindow.detectLanguage("japanese") == Language.JAPANESE
+ assert MainWindow.detectLanguage("spanish") == Language.SPANISH
+ assert MainWindow.detectLanguage("español") == Language.SPANISH
+ assert MainWindow.detectLanguage("espanol") == Language.SPANISH
+ assert MainWindow.detectLanguage("castellano") == Language.SPANISH
+ assert MainWindow.detectLanguage("ENGLISH") == Language.ENGLISH
+ assert MainWindow.detectLanguage("FRENCH") == Language.FRENCH
+ assert MainWindow.detectLanguage("FRANÇAIS") == Language.FRENCH
+ assert MainWindow.detectLanguage("FRANCAIS") == Language.FRENCH
+ assert MainWindow.detectLanguage("GERMAN") == Language.GERMAN
+ assert MainWindow.detectLanguage("DEUTSCH") == Language.GERMAN
+ assert MainWindow.detectLanguage("ITALIAN") == Language.ITALIAN
+ assert MainWindow.detectLanguage("ITALIANO") == Language.ITALIAN
+ assert MainWindow.detectLanguage("JAPANESE") == Language.JAPANESE
+ assert MainWindow.detectLanguage("SPANISH") == Language.SPANISH
+ assert MainWindow.detectLanguage("ESPAÑOL") == Language.SPANISH
+ assert MainWindow.detectLanguage("ESPANOL") == Language.SPANISH
+ assert MainWindow.detectLanguage("CASTELLANO") == Language.SPANISH
+ assert MainWindow.detectLanguage("Something [EN]") == Language.ENGLISH
+ assert MainWindow.detectLanguage("Something [FR]") == Language.FRENCH
+ assert MainWindow.detectLanguage("Something [DE]") == Language.GERMAN
+ assert MainWindow.detectLanguage("Something [IT]") == Language.ITALIAN
+ assert MainWindow.detectLanguage("Something [JA]") == Language.JAPANESE
+ assert MainWindow.detectLanguage("Something [JP]") == Language.JAPANESE
+ assert MainWindow.detectLanguage("Something [ES]") == Language.SPANISH
+ assert MainWindow.detectLanguage("Something [SP]") == Language.SPANISH
+ assert MainWindow.detectLanguage("Something [ENG]") == Language.ENGLISH
+ assert MainWindow.detectLanguage("Something [FRA]") == Language.FRENCH
+ assert MainWindow.detectLanguage("Something [DEU]") == Language.GERMAN
+ assert MainWindow.detectLanguage("Something [GER]") == Language.GERMAN
+ assert MainWindow.detectLanguage("Something [ITA]") == Language.ITALIAN
+ assert MainWindow.detectLanguage("Something [JAP]") == Language.JAPANESE
+ assert MainWindow.detectLanguage("Something [ESP]") == Language.SPANISH
+ assert MainWindow.detectLanguage("Something [SPA]") == Language.SPANISH
+