Merge r9y9 0.3.0 (#16)

* Workaround the installation issue r9y9#27 Not sure about the exact root cause, but the error messages suggested that there's issues with setuptools/pip. I can confirm we can fix the issue by changing the build-time setuptools requirement to <v60.0. At least v59.8.0 should work. https://github.com/pypa/setuptools/releases/tag/v59.8.0 * parepare for v0.1.6 * Start a new dev cycle * Update hts_engine_API * Update open_jtalk https://github.com/r9y9/open_jtalk/releases/tag/v1.11.2 * Update version to v0.2.0 * [ci skip] update changelog * Start new dev cycle * Fixees for Python 3.10 * bump version for dev * Add a workaround for Numpy's ABI issue * add codes * update tests * add test for frontend * add run_marine option to tts() * update the README for run_marin_option * update error message for importing marine * fix for lint * add marine's license * update readme * add new API in docs/pyopenjtalk.rst * add chage log * fix typo * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * Update pyopenjtalk/__init__.py Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> * add link * prep for release --------- Co-authored-by: Ryuichi Yamamoto <zryuichi@gmail.com> Co-authored-by: park.byeongseon <park.byeongseon@linecorp.com> Co-authored-by: bgsn.pk <disloretomail@gmail.com>
VOICEVOX · Feb 2, 2023 · 827a3fc · 827a3fc
1 parent f4ade29
commit 827a3fc
Show file tree

Hide file tree

Showing 11 changed files with 275 additions and 72 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -15,8 +15,19 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [ubuntu-latest, windows-latest]
-        python-version: [3.7, 3.8, 3.9]
+        include:
+          - os: ubuntu-latest
+            python-version: 3.7
+          - os: ubuntu-latest
+            python-version: 3.8
+          - os: ubuntu-latest
+            python-version: 3.9
+          - os: ubuntu-latest
+            python-version: '3.10'
+          - os: macos-latest
+            python-version: 3.9
+          - os: windows-latest
+            python-version: 3.9
 
     steps:
     - uses: actions/checkout@v2
@@ -36,4 +47,4 @@ jobs:
     - name: Test with pytest
       run: |
         pip install pytest
-        pytest
+        pytest
diff --git a/README.md b/README.md
@@ -105,6 +105,24 @@ In [3]: pyopenjtalk.g2p("こんにちは", kana=True)
 Out[3]: 'コンニチワ'
 ```
 
+### About `run_marine` option
+
+After v0.3.0, the `run_marine` option has been available for estimating the Japanese accent with the DNN-based method (see [marine](https://github.com/6gsn/marine)). If you want to use the feature, please install pyopenjtalk as below;
+
+```shell
+pip install pyopenjtalk[marine]
+```
+
+And then, you can use the option as the following examples;
+
+```python
+In [1]: import pyopenjtalk
+
+In [2]: x, sr = pyopenjtalk.tts("おめでとうございます", run_marine=True) # for TTS
+
+In [3]: label = pyopenjtalk.extract_fullcontext("こんにちは", run_marine=True) # for text processing frontend only
+```
+
 ### Create/Apply user dictionary
 
 1. Create a CSV file (e.g. `user.csv`) and write custom words like below:
@@ -139,7 +157,8 @@ done!
 - pyopenjtalk: MIT license ([LICENSE.md](LICENSE.md))
 - Open JTalk: Modified BSD license ([COPYING](https://github.com/r9y9/open_jtalk/blob/1.10/src/COPYING))
 - htsvoice in this repository: Please check [pyopenjtalk/htsvoice/README.md](pyopenjtalk/htsvoice/README.md).
+- marine: Apache 2.0 license ([LICENSE](https://github.com/6gsn/marine/blob/main/LICENSE))
 
 ## Acknowledgements
 
-HTS Working Group for their dedicated efforts to develop and maintain Open JTalk.
+HTS Working Group for their dedicated efforts to develop and maintain Open JTalk.
diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -1,6 +1,14 @@
 Change log
 ==========
 
+v0.3.0 <2022-09-20>
+-------------------
+
+Newer numpy  (>v1.20.0) is required to avoid ABI compatibility issues. Please check the updated installation guide.
+
+* `#40`_: Introduce marine for Japanese accent estimation. Note that there could be a breakpoint regarding `run_frontend` because this PR changed the behavior of the API.
+* `#35`_: Fixes for Python 3.10.
+
 v0.2.0 <2022-02-06>
 -------------------
 
@@ -82,3 +90,5 @@ Initial release with OpenJTalk's text processsing functionality
 .. _#25: https://github.com/r9y9/pyopenjtalk/pull/25
 .. _#27: https://github.com/r9y9/pyopenjtalk/issues/27
 .. _#29: https://github.com/r9y9/pyopenjtalk/pull/29
+.. _#35: https://github.com/r9y9/pyopenjtalk/pull/35
+.. _#40: https://github.com/r9y9/pyopenjtalk/pull/40
diff --git a/docs/index.rst b/docs/index.rst
@@ -24,6 +24,24 @@ The latest release is availabe on pypi. You can install it by:
 
    pip install pyopenjtalk
 
+
+Workaround for ``ValueError: numpy.ndarray size changed, may indicate binary incompatibility``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This type of errors comes from the Numpys' ABI breaking changes. If you see ``ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject`` or similar, please make sure to install numpy first, and then install pyopenjtalk by:
+
+.. code::
+
+   pip install pyopenjtalk --no-build-isolation
+
+or:
+
+.. code::
+
+   pip install git+https://github.com/r9y9/pyopenjtalk --no-build-isolation
+
+The option ``--no-build-isolation`` tells pip not to create a build environment, so the pre-installed numpy is used to build the packge. Hense there should be no Numpy's ABI issues.
+
 .. toctree::
    :maxdepth: 1
    :caption: Notebooks

diff --git a/docs/pyopenjtalk.rst b/docs/pyopenjtalk.rst
@@ -25,3 +25,5 @@ Misc
 ----
 
 .. autofunction:: run_frontend
+.. autofunction:: make_label
+.. autofunction:: estimate_accent
diff --git a/pyopenjtalk/__init__.py b/pyopenjtalk/__init__.py
@@ -19,6 +19,7 @@
 
 from .htsengine import HTSEngine
 from .openjtalk import CreateUserDict, OpenJTalk
+from .utils import merge_njd_marine_features
 
 # Dictionary directory
 # defaults to the package directory where the dictionary will be automatically downloaded
@@ -39,6 +40,8 @@
 # Global instance of HTSEngine
 # mei_normal.voice is used as default
 _global_htsengine = None
+# Global instance of Marine
+_global_marine = None
 
 
 # https://github.com/tqdm/tqdm#hooks-and-callbacks
@@ -98,18 +101,53 @@ def g2p(*args, **kwargs):
     return _global_jtalk.g2p(*args, **kwargs)
 
 
-def extract_fullcontext(text):
+def estimate_accent(njd_features):
+    """Accent estimation using marine
+
+    This function requires marine (https://github.com/6gsn/marine)
+
+    Args:
+        njd_result (list): features generated by OpenJTalk.
+
+    Returns:
+        list: features for NJDNode with estimation results by marine.
+    """
+    global _global_marine
+    if _global_marine is None:
+        try:
+            from marine.predict import Predictor
+        except BaseException:
+            raise ImportError(
+                "Please install marine by `pip install pyopenjtalk[marine]`"
+            )
+        _global_marine = Predictor()
+    from marine.utils.openjtalk_util import convert_njd_feature_to_marine_feature
+
+    marine_feature = convert_njd_feature_to_marine_feature(njd_features)
+    marine_results = _global_marine.predict(
+        [marine_feature], require_open_jtalk_format=True
+    )
+    njd_features = merge_njd_marine_features(njd_features, marine_results)
+    return njd_features
+
+
+def extract_fullcontext(text, run_marine=False):
     """Extract full-context labels from text
 
     Args:
         text (str): Input text
+        run_marine (bool): Whether to estimate accent using marine.
+          Default is False. If you want to activate this option, you need to install marine
+          by `pip install pyopenjtalk[marine]`
 
     Returns:
         list: List of full-context labels
     """
-    # note: drop first return
-    _, labels = run_frontend(text)
-    return labels
+
+    njd_features = run_frontend(text)
+    if run_marine:
+        njd_features = estimate_accent(njd_features)
+    return make_label(njd_features)
 
 
 def synthesize(labels, speed=1.0, half_tone=0.0):
@@ -136,37 +174,56 @@ def synthesize(labels, speed=1.0, half_tone=0.0):
     return _global_htsengine.synthesize(labels), sr
 
 
-def tts(text, speed=1.0, half_tone=0.0):
+def tts(text, speed=1.0, half_tone=0.0, run_marine=False):
     """Text-to-speech
 
     Args:
         text (str): Input text
         speed (float): speech speed rate. Default is 1.0.
         half_tone (float): additional half-tone. Default is 0.
+        run_marine (bool): Whether to estimate accent using marine.
+          Default is False. If you want activate this option, you need to install marine
+          by `pip install pyopenjtalk[marine]`
 
     Returns:
         np.ndarray: speech waveform (dtype: np.float64)
         int: sampling frequency (defualt: 48000)
     """
-    return synthesize(extract_fullcontext(text), speed, half_tone)
+    return synthesize(
+        extract_fullcontext(text, run_marine=run_marine), speed, half_tone
+    )
 
 
-def run_frontend(text, verbose=0):
+def run_frontend(text):
     """Run OpenJTalk's text processing frontend
 
     Args:
         text (str): Unicode Japanese text.
-        verbose (int): Verbosity. Default is 0.
 
     Returns:
-        tuple: Pair of 1) NJD_print and 2) JPCommon_make_label.
-        The latter is the full-context labels in HTS-style format.
+        list: features for NJDNode.
+    """
+    global _global_jtalk
+    if _global_jtalk is None:
+        _lazy_init()
+        _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
+    return _global_jtalk.run_frontend(text)
+
+
+def make_label(njd_features):
+    """Make full-context label using features
+
+    Args:
+        njd_features (list): features for NJDNode.
+
+    Returns:
+        list: full-context labels.
     """
     global _global_jtalk
     if _global_jtalk is None:
         _lazy_init()
         _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
-    return _global_jtalk.run_frontend(text, verbose)
+    return _global_jtalk.make_label(njd_features)
 
 
 def create_user_dict(path, out_path):
@@ -205,4 +262,4 @@ def unset_user_dict():
     global _global_jtalk
     if _global_jtalk is None:
         _lazy_init()
-    _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)
+    _global_jtalk = OpenJTalk(dn_mecab=OPEN_JTALK_DICT_DIR)