Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve: improve support for pyannote.audio #688

Merged
merged 8 commits into from
Mar 23, 2022
Merged

improve: improve support for pyannote.audio #688

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5eb6481
improve: improve support for pyannote.audio
hbredin Feb 16, 2022
81a0d5c
Merge branch 'main' into patch-1
hbredin Mar 11, 2022
d7e3ecb
Merge branch 'main' into patch-1
hbredin Mar 13, 2022
98ac740
Merge branch 'main' into patch-1
hbredin Mar 14, 2022
b61ae65
feat: use "pyannote-audio-{model | pipeline}" as secondary tags
hbredin Mar 23, 2022
600dbe5
Merge branch 'main' into patch-1
hbredin Mar 23, 2022
c838742
fix: revert unrelated changes
hbredin Mar 23, 2022
0d991a3
fix: revert unrelated changes
hbredin Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 33 additions & 9 deletions js/src/lib/interfaces/Libraries.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export enum ModelLibrary {
"fairseq" = "Fairseq",
"flair" = "Flair",
"keras" = "Keras",
"pyannote" = "Pyannote",
"pyannote-audio" = "pyannote.audio",
"sentence-transformers" = "Sentence Transformers",
"sklearn" = "Scikit-learn",
"spacy" = "spaCy",
Expand Down Expand Up @@ -138,18 +138,42 @@ const keras = (model: ModelData) =>
model = from_pretrained_keras("${model.id}")
`;

const pyannote = (model: ModelData) =>
`from pyannote.audio.core.inference import Inference
const pyannote_audio_pipeline = (model: ModelData) =>
`from pyannote.audio import Pipeline

model = Inference("${model.id}")
pipeline = Pipeline.from_pretrained("${model.id}")

# inference on the whole file
model("file.wav")
pipeline("file.wav")

# inference on an excerpt
from pyannote.core import Segment
excerpt = Segment(start=2.0, end=5.0)
model.crop("file.wav", excerpt)`;

from pyannote.audio import Audio
waveform, sample_rate = Audio().crop("file.wav", excerpt)
pipeline({"waveform": waveform, "sample_rate": sample_rate})`;

const pyannote_audio_model = (model: ModelData) =>
`from pyannote.audio import Model, Inference

model = Model.from_pretrained("${model.id}")
inference = Inference(model)

# inference on the whole file
inference("file.wav")

# inference on an excerpt
from pyannote.core import Segment
excerpt = Segment(start=2.0, end=5.0)
inference.crop("file.wav", excerpt)`;

const pyannote_audio = (model: ModelData) => {
if (model.tags?.includes("pyannote-audio-pipeline")) {
hbredin marked this conversation as resolved.
Show resolved Hide resolved
return pyannote_audio_pipeline(model);
}
return pyannote_audio_model(model);
};

const tensorflowttsTextToMel = (model: ModelData) =>
`from tensorflow_tts.inference import AutoProcessor, TFAutoModel
Expand Down Expand Up @@ -338,11 +362,11 @@ export const MODEL_LIBRARIES_UI_ELEMENTS: { [key in keyof typeof ModelLibrary]?:
repoUrl: "https://github.com/keras-team/keras",
snippet: keras,
},
"pyannote": {
btnLabel: "pyannote",
"pyannote-audio": {
btnLabel: "pyannote.audio",
repoName: "pyannote-audio",
repoUrl: "https://github.com/pyannote/pyannote-audio",
snippet: pyannote,
snippet: pyannote_audio,
},
"sentence-transformers": {
btnLabel: "sentence-transformers",
Expand Down