Merge r9y9 0.3.0 (r9y9#16)

* Workaround the installation issue r9y9#27 Not sure about the exact root cause, but the error messages suggested that there's issues with setuptools/pip. I can confirm we can fix the issue by changing the build-time setuptools requirement to <v60.0. At least v59.8.0 should work. https://github.com/pypa/setuptools/releases/tag/v59.8.0 * parepare for v0.1.6 * Start a new dev cycle * Update hts_engine_API * Update open_jtalk https://github.com/r9y9/open_jtalk/releases/tag/v1.11.2 * Update version to v0.2.0 * [ci skip] update changelog * Start new dev cycle * Fixees for Python 3.10 * bump version for dev * Add a workaround for Numpy's ABI issue * add codes * update tests * add test for frontend * add run_marine option to tts() * update the README for run_marin_option * update error message for importing marine * fix for lint * add marine's license * update readme * add new API in docs/pyopenjtalk.rst * add chage log * fix typo * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * add link * prep for release --------- Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> Co-authored-by: park.byeongseon <park.byeongseon@linecorp.com> Co-authored-by: bgsn.pk <disloretomail@gmail.com>
Hiroshiba · Feb 2, 2023 · 827a3fc · 827a3fc
1 parent f4ade29
commit 827a3fc
Show file tree

Hide file tree

Showing 11 changed files with 275 additions and 72 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -15,8 +15,19 @@ jobs:
  runs-on: ${{ matrix.os }}
  strategy:
  matrix:
- os: [ubuntu-latest, windows-latest]
- python-version: [3.7, 3.8, 3.9]
+ include:
+ - os: ubuntu-latest
+ python-version: 3.7
+ - os: ubuntu-latest
+ python-version: 3.8
+ - os: ubuntu-latest
+ python-version: 3.9
+ - os: ubuntu-latest
+ python-version: '3.10'
+ - os: macos-latest
+ python-version: 3.9
+ - os: windows-latest
+ python-version: 3.9
 
  steps:
  - uses: actions/checkout@v2
@@ -36,4 +47,4 @@ jobs:
  - name: Test with pytest
  run: |
  pip install pytest
- pytest
+ pytest
diff --git a/README.md b/README.md
@@ -105,6 +105,24 @@ In [3]: pyopenjtalk.g2p("こんにちは", kana=True)
 Out[3]: 'コンニチワ'
 ```
 
+### About `run_marine` option
+
+After v0.3.0, the `run_marine` option has been available for estimating the Japanese accent with the DNN-based method (see [marine](https://github.com/6gsn/marine)). If you want to use the feature, please install pyopenjtalk as below;
+
+```shell
+pip install pyopenjtalk[marine]
+```
+
+And then, you can use the option as the following examples;
+
+```python
+In [1]: import pyopenjtalk
+
+In [2]: x, sr = pyopenjtalk.tts("おめでとうございます", run_marine=True) # for TTS
+
+In [3]: label = pyopenjtalk.extract_fullcontext("こんにちは", run_marine=True) # for text processing frontend only
+```
+
 ### Create/Apply user dictionary
 
 1. Create a CSV file (e.g. `user.csv`) and write custom words like below:
@@ -139,7 +157,8 @@ done!
 - pyopenjtalk: MIT license ([LICENSE.md](LICENSE.md))
 - Open JTalk: Modified BSD license ([COPYING](https://github.com/r9y9/open_jtalk/blob/1.10/src/COPYING))
 - htsvoice in this repository: Please check [pyopenjtalk/htsvoice/README.md](pyopenjtalk/htsvoice/README.md).
+- marine: Apache 2.0 license ([LICENSE](https://github.com/6gsn/marine/blob/main/LICENSE))
 
 ## Acknowledgements
 
-HTS Working Group for their dedicated efforts to develop and maintain Open JTalk.
+HTS Working Group for their dedicated efforts to develop and maintain Open JTalk.
diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -1,6 +1,14 @@
 Change log
 ==========
 
+v0.3.0 <2022-09-20>
+-------------------
+
+Newer numpy (>v1.20.0) is required to avoid ABI compatibility issues. Please check the updated installation guide.
+
+* `#40`_: Introduce marine for Japanese accent estimation. Note that there could be a breakpoint regarding `run_frontend` because this PR changed the behavior of the API.
+* `#35`_: Fixes for Python 3.10.
+
 v0.2.0 <2022-02-06>
 -------------------
 
@@ -82,3 +90,5 @@ Initial release with OpenJTalk's text processsing functionality
 .. _#25: https://github.com/r9y9/pyopenjtalk/pull/25
 .. _#27: https://github.com/r9y9/pyopenjtalk/issues/27
 .. _#29: https://github.com/r9y9/pyopenjtalk/pull/29
+.. _#35: https://github.com/r9y9/pyopenjtalk/pull/35
+.. _#40: https://github.com/r9y9/pyopenjtalk/pull/40
diff --git a/docs/index.rst b/docs/index.rst
@@ -24,6 +24,24 @@ The latest release is availabe on pypi. You can install it by:
 
  pip install pyopenjtalk
 
+
+Workaround for ``ValueError: numpy.ndarray size changed, may indicate binary incompatibility``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This type of errors comes from the Numpys' ABI breaking changes. If you see ``ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject`` or similar, please make sure to install numpy first, and then install pyopenjtalk by:
+
+.. code::
+
+ pip install pyopenjtalk --no-build-isolation
+
+or:
+
+.. code::
+
+ pip install git+https://github.com/r9y9/pyopenjtalk --no-build-isolation
+
+The option ``--no-build-isolation`` tells pip not to create a build environment, so the pre-installed numpy is used to build the packge. Hense there should be no Numpy's ABI issues.
+
 .. toctree::
  :maxdepth: 1
  :caption: Notebooks

diff --git a/docs/pyopenjtalk.rst b/docs/pyopenjtalk.rst
@@ -25,3 +25,5 @@ Misc
 ----
 
 .. autofunction:: run_frontend
+.. autofunction:: make_label
+.. autofunction:: estimate_accent
diff --git a/pyopenjtalk/__init__.py b/pyopenjtalk/__init__.py
@@ -19,6 +19,7 @@
 
 from .htsengine import HTSEngine
 from .openjtalk import CreateUserDict, OpenJTalk
+from .utils import merge_njd_marine_features
 
 # Dictionary directory
 # defaults to the package directory where the dictionary will be automatically downloaded
@@ -39,6 +40,8 @@
 # Global instance of HTSEngine
 # mei_normal.voice is used as default
 _global_htsengine = None
+# Global instance of Marine
+_global_marine = None
 
 
 # https://github.com/tqdm/tqdm#hooks-and-callbacks
@@ -98,18 +101,53 @@ def g2p(*args, **kwargs):
  return _global_jtalk.g2p(*args, **kwargs)
 
 
-def extract_fullcontext(text):
+def estimate_accent(njd_features):
+ """Accent estimation using marine
+
+ This function requires marine (https://github.com/6gsn/marine)
+
+ Args:
+ njd_result (list): features generated by OpenJTalk.
+
+ Returns:
+ list: features for NJDNode with estimation results by marine.
+ """
+ global _global_marine
+ if _global_marine is None:
+ try:
+ from marine.predict import Predictor
+ except BaseException:
+ raise ImportError(
+ "Please install marine by `pip install pyopenjtalk[marine]`"
+ )
+ _global_marine = Predictor()
+ from marine.utils.openjtalk_util import convert_njd_feature_to_marine_feature
+
+ marine_feature = convert_njd_feature_to_marine_feature(njd_features)
+ marine_results = _global_marine.predict(
+ [marine_feature], require_open_jtalk_format=True
+ )
+ njd_features = merge_njd_marine_features(njd_features, marine_results)
+ return njd_features
+
+
+def extract_fullcontext(text, run_marine=False):
  """Extract full-context labels from text
 
  Args:
  text (str): Input text
+ run_marine (bool): Whether to estimate accent using marine.
+ Default is False. If you want to activate this option, you need to install marine
+ by `pip install pyopenjtalk[marine]`
 
  Returns:
  list: List of full-context labels
  """
- # note: drop first return
- _, labels = run_frontend(text)
- return labels
+
+ njd_features = run_frontend(text)
+ if run_marine:
+ njd_features = estimate_accent(njd_features)
+ return make_label(njd_features)
 
 
 def synthesize(labels, speed=1.0, half_tone=0.0):
@@ -136,37 +174,56 @@ def synthesize(labels, speed=1.0, half_tone=0.0):
  return _global_htsengine.synthesize(labels), sr
 
 
-def tts(text, speed=1.0, half_tone=0.0):
+def tts(text, speed=1.0, half_tone=0.0, run_marine=False):
  """Text-to-speech
 
  Args:
  text (str): Input text
  speed (float): speech speed rate. Default is 1.0.
  half_tone (float): additional half-tone. Default is 0.
+ run_marine (bool): Whether to estimate accent using marine.
+ Default is False. If you want activate this option, you need to install marine
+ by `pip install pyopenjtalk[marine]`
 
  Returns:
  np.ndarray: speech waveform (dtype: np.float64)
  int: sampling frequency (defualt: 48000)
  """
- return synthesize(extract_fullcontext(text), speed, half_tone)
+ return synthesize(
+ extract_fullcontext(text, run_marine=run_marine), speed, half_tone
+ )
 
 
-def run_frontend(text, verbose=0):
+def run_frontend(text):
  """Run OpenJTalk's text processing frontend
 
  Args:
  text (str): Unicode Japanese text.
- verbose (int): Verbosity. Default is 0.
 
  Returns:
- tuple: Pair of 1) NJD_print and 2) JPCommon_make_label.
- The latter is the full-context labels in HTS-style format.
+ list: features for NJDNode.
+ """
+ global _global_jtalk
+ if _global_jtalk is None:
+ _lazy_init()
+ _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
+ return _global_jtalk.run_frontend(text)
+
+
+def make_label(njd_features):
+ """Make full-context label using features
+
+ Args:
+ njd_features (list): features for NJDNode.
+
+ Returns:
+ list: full-context labels.
  """
  global _global_jtalk
  if _global_jtalk is None:
  _lazy_init()
  _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
- return _global_jtalk.run_frontend(text, verbose)
+ return _global_jtalk.make_label(njd_features)
 
 
 def create_user_dict(path, out_path):
@@ -205,4 +262,4 @@ def unset_user_dict():
  global _global_jtalk
  if _global_jtalk is None:
  _lazy_init()
- _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
+ _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)