Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

エンジンのマニフェストを追加 #387

Merged
merged 3 commits into from
May 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,9 @@ jobs:
- name: Generate licenses.json
shell: bash
run: |
python generate_licenses.py > licenses.json
python generate_licenses.py > manifest_assets/dependency_licenses.json
# FIXME: VOICEVOX (editor) cannot build without licenses.json
cp manifest_assets/dependency_licenses.json licenses.json

# Download ONNX Runtime
- name: Export ONNX Runtime url to calc hash
Expand Down Expand Up @@ -129,6 +131,7 @@ jobs:
set -eux
rm -r speaker_info
cp -r download/resource/character_info speaker_info
cp -r download/resource/manifest_assets manifest_assets

- name: Prepare VOICEVOX Core release cache
uses: actions/cache@v2
Expand Down Expand Up @@ -197,6 +200,7 @@ jobs:
--include-data-file=${{ env.pythonLocation }}/lib/python*/site-packages/scipy/.dylibs/*.dylib=./scipy/.dylibs/ \
--include-data-file=${{ env.pythonLocation }}/lib/python*/site-packages/_soundfile_data/*=./_soundfile_data/ \
--include-data-dir=../speaker_info=./speaker_info \
--include-data-dir=../manifest_assets=./manifest_assets \
--follow-imports \
--no-prefer-source-code \
../run.py
Expand Down Expand Up @@ -298,6 +302,7 @@ jobs:
set -eux
rm -r speaker_info
cp -r download/resource/character_info speaker_info
cp -r download/resource/manifest_assets manifest_assets

# NOTE: `load: true` may silently fail when the GitHub Actions disk (14GB) is full.
# https://docs.github.com/ja/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources
Expand Down Expand Up @@ -641,6 +646,7 @@ jobs:
set -eux
rm -r speaker_info
cp -r download/resource/character_info speaker_info
cp -r download/resource/manifest_assets manifest_assets

# Download VOICEVOX Core
- name: Prepare VOICEVOX Core cache
Expand All @@ -660,7 +666,10 @@ jobs:

- name: Generate licenses.json
shell: bash
run: python generate_licenses.py > licenses.json
run: |
python generate_licenses.py > manifest_assets/dependency_licenses.json
# FIXME: VOICEVOX (editor) cannot build without licenses.json
cp manifest_assets/dependency_licenses.json licenses.json

- name: Cache Nuitka (ccache, module-cache)
uses: actions/cache@v2
Expand Down Expand Up @@ -705,6 +714,7 @@ jobs:
--include-data-file="download/onnxruntime/lib/onnxruntime.dll=./" \
--include-data-file="download/core/${{ matrix.voicevox_core_dll_name }}=./" \
--include-data-dir="speaker_info=./speaker_info" \
--include-data-dir="manifest_assets=./manifest_assets" \
--msvc=14.2 \
--follow-imports \
--no-prefer-source-code \
Expand Down
10 changes: 8 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ ADD ./voicevox_engine /opt/voicevox_engine/voicevox_engine
ADD ./docs /opt/voicevox_engine/docs
ADD ./run.py ./generate_licenses.py ./presets.yaml ./default.csv /opt/voicevox_engine/
ADD ./speaker_info /opt/voicevox_engine/speaker_info
ADD ./manifest_assets /opt/voicevox_engine/manifest_assets

# Replace version
ARG VOICEVOX_ENGINE_VERSION=latest
Expand All @@ -218,7 +219,9 @@ RUN <<EOF
export PATH="/home/user/.local/bin:${PATH:-}"

gosu user /opt/python/bin/pip3 install pip-licenses
gosu user /opt/python/bin/python3 generate_licenses.py > /opt/voicevox_engine/licenses.json
gosu user /opt/python/bin/python3 generate_licenses.py > /opt/voicevox_engine/manifest_assets/dependency_licenses.json
# FIXME: VOICEVOX (editor) cannot build without licenses.json
cp /opt/voicevox_engine/manifest_assets/dependency_licenses.json /opt/voicevox_engine/licenses.json
EOF

# Keep this layer separated to use layer cache on download failed in local build
Expand Down Expand Up @@ -294,7 +297,9 @@ RUN <<EOF
export PATH="/home/user/.local/bin:${PATH:-}"

gosu user /opt/python/bin/pip3 install pip-licenses
gosu user /opt/python/bin/python3 generate_licenses.py > /opt/voicevox_engine/licenses.json
gosu user /opt/python/bin/python3 generate_licenses.py > /opt/voicevox_engine/manifest_assets/dependency_licenses.json
# FIXME: VOICEVOX (editor) cannot build without licenses.json
cp /opt/voicevox_engine/manifest_assets/dependency_licenses.json /opt/voicevox_engine/licenses.json
EOF

# Create build script
Expand Down Expand Up @@ -333,6 +338,7 @@ RUN <<EOF
--include-data-file=/opt/voicevox_core/*.so=./ \
--include-data-file=/opt/onnxruntime/lib/libonnxruntime.so=./ \
--include-data-dir=/opt/voicevox_engine/speaker_info=./speaker_info \
--include-data-dir=/opt/voicevox_engine/manifest_assets=./manifest_assets \
--follow-imports \
--no-prefer-source-code \
/opt/voicevox_engine/run.py
Expand Down
9 changes: 9 additions & 0 deletions manifest_assets/base_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"manifest_version": "0.12.0",
"name": "dummy name",
"icon": "",
"default_sampling_rate": 24000,
"terms_of_service": "",
"update_infos": [],
"dependency_licenses": []
}
8 changes: 8 additions & 0 deletions manifest_assets/dependency_licenses.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[
{
"name": "dummy library",
"version": "0.0.1",
"license": "dummy license",
"text": "dummy license text"
}
]
Binary file added manifest_assets/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions manifest_assets/terms_of_service.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dummy teams of service
7 changes: 7 additions & 0 deletions manifest_assets/update_infos.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[
{
"version": "0.0.1",
"descriptions": "dummpy descriptions",
"contributors": ["dummy contributor"]
}
]
7 changes: 7 additions & 0 deletions run.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@

from voicevox_engine import __version__
from voicevox_engine.cancellable_engine import CancellableEngine
from voicevox_engine.engine_manifest import EngineManifestLoader
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

他のクラスはついてないので、短くするためにEngineなくても良いかも?(あまり強い意見じゃないです)

from voicevox_engine.engine_manifest.EngineManifest import EngineManifest
from voicevox_engine.kana_parser import create_kana, parse_kana
from voicevox_engine.model import (
AccentPhrase,
Expand Down Expand Up @@ -87,6 +89,7 @@ def generate_app(
preset_loader = PresetLoader(
preset_path=root_dir / "presets.yaml",
)
engine_manifest_loader = EngineManifestLoader(root_dir / "manifest_assets")

# キャッシュを有効化
# モジュール側でlru_cacheを指定するとキャッシュを制御しにくいため、HTTPサーバ側で指定する
Expand Down Expand Up @@ -868,6 +871,10 @@ def supported_devices(
media_type="application/json",
)

@app.get("/engine_manifest", response_model=EngineManifest, tags=["その他"])
def engine_manifest():
return engine_manifest_loader.load_manifest()

return app


Expand Down
41 changes: 41 additions & 0 deletions voicevox_engine/engine_manifest/EngineManifest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from typing import List, Optional
Hiroshiba marked this conversation as resolved.
Show resolved Hide resolved

from pydantic import BaseModel, Field


class UpdateInfo(BaseModel):
"""
エンジンのアップデート情報
"""

version: str = Field(title="エンジンのバージョン名")
descriptions: str = Field(title="アップデートの詳細についての説明")
contributors: Optional[List[str]] = Field(title="貢献者名")


class LicenseInfo(BaseModel):
"""
依存ライブラリのライセンス情報
"""

name: str = Field(title="依存ライブラリ名")
version: Optional[str] = Field(title="依存ライブラリのバージョン")
license: Optional[str] = Field(title="依存ライブラリのライセンス名")
text: str = Field(title="依存ライブラリのライセンス本文")


class EngineManifest(BaseModel):
"""
エンジン自体に関する情報
"""

manifest_version: str = Field(title="マニフェストのバージョン")
name: str = Field(title="エンジン名")
icon: str = Field(title="エンジンのアイコンをBASE64エンコードしたもの")
default_sampling_rate: int = Field(title="デフォルトのサンプリング周波数")
terms_of_service: str = Field(title="エンジンの利用規約")
update_infos: List[UpdateInfo] = Field(title="エンジンのアップデート情報")
dependency_licenses: List[LicenseInfo] = Field(title="依存関係のライセンス情報")

class Config:
validate_assignment = True
34 changes: 34 additions & 0 deletions voicevox_engine/engine_manifest/EngineManifestLoader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import json
from base64 import b64encode
from pathlib import Path

from .EngineManifest import EngineManifest, LicenseInfo, UpdateInfo


class EngineManifestLoader:
def __init__(self, assets_dir: Path):
self.assets_dir = assets_dir

def load_manifest(self) -> EngineManifest:
manifest = EngineManifest(
**json.load((self.assets_dir / "base_info.json").open(encoding="utf-8"))
)
manifest.icon = b64encode((self.assets_dir / "icon.png").read_bytes()).decode(
"utf-8"
)
manifest.terms_of_service = (self.assets_dir / "terms_of_service.md").read_text(
encoding="utf-8"
)
manifest.update_infos = [
UpdateInfo(**update_info)
for update_info in json.load(
(self.assets_dir / "update_infos.json").open(encoding="utf-8")
)
]
manifest.dependency_licenses = [
LicenseInfo(**license_info)
for license_info in json.load(
(self.assets_dir / "dependency_licenses.json").open(encoding="utf-8")
)
]
return manifest
7 changes: 7 additions & 0 deletions voicevox_engine/engine_manifest/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from .EngineManifest import EngineManifest
from .EngineManifestLoader import EngineManifestLoader

__all__ = [
"EngineManifest",
"EngineManifestLoader",
]
35 changes: 0 additions & 35 deletions voicevox_engine/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,38 +276,3 @@ class SupportedFeaturesInfo(BaseModel):
support_adjusting_silence_scale: bool = Field(title="前後の無音時間が調節可能かどうか")
support_interrogative_upspeak: bool = Field(title="疑似疑問文に対応しているかどうか")
support_switching_device: bool = Field(title="CPU/GPUの切り替えが可能かどうか")


class UpdateInfo(BaseModel):
"""
エンジンのアップデート情報
"""

version: str = Field(title="エンジンのバージョン名")
descriptions: List[str] = Field(title="アップデートの詳細についての説明")
contributors: Optional[List[str]] = Field(title="貢献者名")


class LicenseInfo(BaseModel):
"""
依存ライブラリのライセンス情報
"""

name: str = Field(title="依存ライブラリ名")
version: Optional[str] = Field(title="依存ライブラリのバージョン")
license: Optional[str] = Field(title="依存ライブラリのライセンス名")
text: str = Field(title="依存ライブラリのライセンス本文")


class EngineManifest(BaseModel):
"""
エンジン自体に関する情報
"""

manifest_version: str = Field(title="マニフェストのバージョン")
name: str = Field(title="エンジン名")
icon: str = Field(title="エンジンのアイコンをBASE64エンコードしたもの")
default_sampling_rate: int = Field(title="デフォルトのサンプリング周波数")
terms_of_service: str = Field(title="エンジンの利用規約")
update_infos: List[UpdateInfo] = Field(title="エンジンのアップデート情報")
dependency_licenses: List[LicenseInfo] = Field(title="依存関係のライセンス情報")