Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

マルチエンジン:VVPPに7zを使えるように #1253

Merged
merged 31 commits into from
Apr 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
84345ca
Add: 7zを同梱
sevenc-nanashi Mar 13, 2023
216a8f1
Add: 7zを同梱
sevenc-nanashi Mar 15, 2023
d7958ae
Add: 分割7zの対応を追加
sevenc-nanashi Mar 15, 2023
ee8e953
Merge branch 'main' of github.com:voicevox/voicevox into add/7z-vvpp
sevenc-nanashi Mar 15, 2023
cf05938
Add: コメントを追加
sevenc-nanashi Mar 15, 2023
cf1d170
Fix: productionビルド時のバグを修正
sevenc-nanashi Mar 15, 2023
6996708
Code: コメントを追記
sevenc-nanashi Mar 15, 2023
1a30be6
Change: zipも7zで解凍するように
sevenc-nanashi Mar 16, 2023
93e7003
Merge: main -> add/7z-vvpp
sevenc-nanashi Mar 16, 2023
9b50042
Fix: installer.nshを修正
sevenc-nanashi Mar 16, 2023
b1405e3
Update: 7zipの埋め込みを更新
sevenc-nanashi Mar 16, 2023
cb593ef
Change: 7zaを埋め込まず、落とすように
sevenc-nanashi Mar 17, 2023
cc9208f
Fix: バージョン取得周りを修正
sevenc-nanashi Mar 17, 2023
ae15d8e
Fix: electron-builderのコンフィグを修正
sevenc-nanashi Mar 17, 2023
3522d81
Revert: installer.nshの変更を取り消し
sevenc-nanashi Mar 17, 2023
6f916f8
Fix: 消し忘れ
sevenc-nanashi Mar 17, 2023
773f1f7
Delete: 不要なrequireを削除
sevenc-nanashi Mar 17, 2023
5a16dbf
Delete: 不要なimportとパッケージを削除
sevenc-nanashi Mar 17, 2023
769122f
Delete: whichの依存を削除
sevenc-nanashi Mar 17, 2023
7b1956d
Fix: multistreamは必要だった
sevenc-nanashi Mar 17, 2023
9eba6f6
Change: ダウンローダーからもdllを削除
sevenc-nanashi Mar 17, 2023
4fad9aa
Change: レビューを反映
sevenc-nanashi Mar 18, 2023
abb11ed
Update: installer.nshをファイルの変更に追従
sevenc-nanashi Mar 18, 2023
d31d005
Fix: スラッシュを修正
sevenc-nanashi Mar 19, 2023
b3a6648
Code: sevenZipFiles -> sevenZipFile
sevenc-nanashi Mar 19, 2023
a4be2ad
Refactor: ファイルのフォーマット周りを共通化
sevenc-nanashi Mar 19, 2023
82320dc
Code: フォーマットを変更
sevenc-nanashi Mar 19, 2023
5a5cfdc
Code: FIXMEを追加
sevenc-nanashi Mar 19, 2023
0b3a679
Change: ファイル名を全部指定するように
sevenc-nanashi Mar 19, 2023
de13f80
Code: 7zrを落とす理由を追加
sevenc-nanashi Mar 19, 2023
22b2a7c
Code: 概要と型チェックを追加
sevenc-nanashi Mar 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed build/7zr.exe
Binary file not shown.
91 changes: 91 additions & 0 deletions build/download7z.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// @ts-check
/**
* OSに合った7-Zipのバイナリとライセンスをダウンロードするスクリプト。
*/
const path = require("path");
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
const fs = require("fs");
const { spawnSync } = require("child_process");

(async () => {
// node-fetchはESModuleなので、import()で読み込む
const { default: fetch } = await import("node-fetch");
const distPath = path.resolve(__dirname, "vendored", "7z");
let url;
let filesToExtract;
switch (process.platform) {
case "win32": {
// 7za.exeは7z形式で圧縮されているので、7zr.exeが必要になる。
// Mac/Linuxと違い、インストーラー以外には7z形式でしか配布されていない。
// Actionsでインストーラーを動かすことはできないので、単独で配布されている7zr.exeを使い、
// 7z形式で圧縮されている7za.exeを展開する。
const sevenzrUrl = "https://www.7-zip.org/a/7zr.exe";
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved
const sevenzrPath = path.resolve(distPath, "7zr.exe");
if (!fs.existsSync(sevenzrPath)) {
console.log("Downloading 7zr from " + sevenzrUrl);
const res = await fetch(sevenzrUrl);
const buffer = await res.arrayBuffer();

await fs.promises.writeFile(sevenzrPath, Buffer.from(buffer));
}

url = "https://www.7-zip.org/a/7z2201-extra.7z";
// 7za.dll、7zxa.dllはなくても動くので、除外する
// filesToExtract = ["7za.exe", "7za.dll", "7zxa.dll", "License.txt"];
filesToExtract = ["7za.exe", "License.txt"];
sevenc-nanashi marked this conversation as resolved.
Show resolved Hide resolved

break;
}
case "linux": {
url = "https://www.7-zip.org/a/7z2201-linux-x64.tar.xz";
filesToExtract = ["7zzs", "License.txt"];
break;
}
case "darwin": {
url = "https://www.7-zip.org/a/7z2107-mac.tar.xz";
filesToExtract = ["7zz", "License.txt"];
break;
}
default: {
throw new Error("Unsupported platform");
}
}

const existingFiles = await fs.promises.readdir(distPath);

const notDownloaded = filesToExtract.filter(
(file) => !existingFiles.includes(file)
);

if (notDownloaded.length === 0) {
console.log("7z already downloaded");
return;
}

console.log("Downloading 7z from " + url);
const res = await fetch(url);
const buffer = await res.arrayBuffer();
const sevenZipPath = path.resolve(distPath, path.basename(url));
await fs.promises.writeFile(sevenZipPath, Buffer.from(buffer));

console.log("Extracting 7z");
const extractor = url.endsWith(".7z")
? spawnSync(
path.resolve(distPath, "7zr.exe"),
["x", "-y", "-o" + distPath, sevenZipPath, ...filesToExtract],
{
stdio: ["ignore", "inherit", "inherit"],
}
)
: spawnSync(
"tar",
["xvf", sevenZipPath, "-v", "-C", distPath, ...filesToExtract],
{
stdio: ["ignore", "inherit", "inherit"],
}
);

if (extractor.status !== 0) {
console.error("Failed to extract 7z");
process.exit(1);
}
})();
41 changes: 35 additions & 6 deletions build/generateLicenses.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
const process = require("process");
const { execFileSync } = require("child_process");
const fs = require("fs");
const path = require("path");
const yargs = require("yargs/yargs");
const { hideBin } = require("yargs/helpers");
const argv = yargs(hideBin(process.argv))
Expand Down Expand Up @@ -57,12 +58,40 @@ const licenseJson = execFileSync(

const checkerLicenses = JSON.parse(licenseJson);

const licenses = Object.entries(checkerLicenses).map(([, license]) => ({
name: license.name,
version: license.version,
license: license.licenses,
text: license.licenseText,
}));
const externalLicenses = [];

externalLicenses.push({
name: "7-Zip",
version: execFileSync(
path.join(
__dirname,
"vendored",
"7z",
{
win32: "7za.exe",
linux: "7zzs",
darwin: "7zz",
}[process.platform]
),

{
encoding: "utf-8",
}
).match(/7-Zip\s+(?:\(.\))?\s*([0-9.]+)/)[1],
license: "LGPL-2.1",
text: fs.readFileSync(path.join(__dirname, "vendored", "7z", "License.txt"), {
encoding: "utf-8",
}),
});

const licenses = Object.entries(checkerLicenses)
.map(([, license]) => ({
name: license.name,
version: license.version,
license: license.licenses,
text: license.licenseText,
}))
.concat(externalLicenses);

const outputPath = argv.output_path;
fs.writeFileSync(outputPath, JSON.stringify(licenses));
4 changes: 2 additions & 2 deletions build/installer.nsh
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,9 @@
StrCpy $archiveSize $0

; 展開後の合計サイズを取得
File /oname=$PLUGINSDIR\7zr.exe "${BUILD_RESOURCES_DIR}\7zr.exe"
File /oname=$PLUGINSDIR\7za.exe "${BUILD_RESOURCES_DIR}\vendored\7z\7za.exe"
${getUncompressedSizeFrom7z} $0 $1
${If} $0 == "Failed to execute 7zr.exe"
${If} $0 == "Failed to execute 7za.exe"
${OrIf} $0 == "Failed to open file list"
StrCpy $0 "Failed"
Goto updateDefinedVariables7z_finish
Expand Down
4 changes: 4 additions & 0 deletions build/vendored/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
*
!README.md
!.gitignore
!*/.gitkeep
Empty file added build/vendored/7z/.gitkeep
Empty file.
7 changes: 7 additions & 0 deletions build/vendored/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# build/vendored

このディレクトリはダウンロードしたファイルを格納するためのものです。

| ディレクトリ名 | 内容 | ダウンローダー |
| -------------- | ------------------------------ | --------------------- |
| `7z` | [7-Zip](http://www.7-zip.org/) | `build/download7z.js` |
18 changes: 18 additions & 0 deletions electron-builder.config.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @ts-check
const path = require("path");
const fs = require("fs");

Expand All @@ -24,6 +25,19 @@ const isMac = process.platform === "darwin";
// cf: https://k-hyoda.hatenablog.com/entry/2021/10/23/000349#%E8%BF%BD%E5%8A%A0%E5%B1%95%E9%96%8B%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB%E5%85%88%E3%81%AE%E8%A8%AD%E5%AE%9A
const extraFilePrefix = isMac ? "MacOS/" : "";

const sevenZipFile = fs
.readdirSync(path.resolve(__dirname, "build", "vendored", "7z"))
.find(
// Windows: 7za.exe, Linux: 7zzs, macOS: 7zz
(fileName) => ["7za.exe", "7zzs", "7zz"].includes(fileName)
);

if (!sevenZipFile) {
throw new Error(
"7z binary file not found. Run `node ./build/download7z.js` first."
);
}

/** @type {import("electron-builder").Configuration} */
const builderOptions = {
beforeBuild: async () => {
Expand Down Expand Up @@ -72,6 +86,10 @@ const builderOptions = {
from: VOICEVOX_ENGINE_DIR,
to: extraFilePrefix,
},
{
from: path.resolve(__dirname, "build", "vendored", "7z", sevenZipFile),
to: extraFilePrefix + sevenZipFile,
},
],
// electron-builder installer
productName: "VOICEVOX",
Expand Down
Loading