-
|
It might have been a slip in the crack or a happy accident which got unknowingly patched in a latter Commit, but when transcribing an English video with language XYZ (Most European languages and a few Asian ones) and --transcribe, it generated a PERFECTLY translated files. And it translated while being asked to transcribe. I don't know which commit killed it, as I've done a handful of Git Checkout ID . sadly to no avail. Anyone had a similar experience, or better yet, is still able to trick Whisper into translating to non-officially supported languages? |
Beta Was this translation helpful? Give feedback.
Replies: 14 comments 18 replies
-
|
Translating into languages other than English wasn't a part of the training objective, so that behavior may not be always reliable. I suspect the new If that's the case, you can specify |
Beta Was this translation helpful? Give feedback.
-
|
hey @nicolasraj, thanks a lot, it works ! I was using translate instead of transcribe... I guess I was not really well awake when I read this first post... |
Beta Was this translation helpful? Give feedback.
-
|
this whisper audio.mp3 --model medium --task transcribe --language French works perfectly, only bad deal is that without gpu delays eons to translate, and you may need to pay for premium gpus after some time, that or manually translate your files, that may be faster if you know english-your language |
Beta Was this translation helpful? Give feedback.
-
|
@nicolasraj Sorry for the perhaps trivial question, do you know how I should do if I want to integrate the translation into any language from a python file? Thank you in advance |
Beta Was this translation helpful? Give feedback.
-
|
Is Audio.mp3 English, and the output is French? "this whisper audio.mp3 --model medium --task transcribe --language French works perfectly" |
Beta Was this translation helpful? Give feedback.
-
|
So Whisper does translate from English to French for example now? What is the command then, because I can't make it work. |
Beta Was this translation helpful? Give feedback.
-
|
If the translations were as accurate as you say, it would be great to be able to use Whisper for translation. I hope OpenAI releases an open-source translation model, the cost of translation keeps going up it's an industry that could use some disruption in a big way. With Whisper the cost of producing high quality transcriptions is driven down significantly but to translate an hour of content say with Google Translate (fixed rate of $20/million characters) costs about $0.80/h of transcribed content, which ends up being way higher than the actual cost of transcription itself, while translation (based on my amateur guess) should be much less computationally expensive. |
Beta Was this translation helpful? Give feedback.
-
|
Whisper is translating for me from English to French and Spanish. Using big model, and a '5' wait time. |
Beta Was this translation helpful? Give feedback.
-
|
Indeed.
Way back when, Whisper did DeepL level translations in languages it wasn't
documented to support.
Le dim. 9 juil. 2023 à 10:18, ShanghaiTimes ***@***.***> a
écrit :
… Is Audio.mp3 English, and the output is French?
"this whisper audio.mp3 --model medium --task transcribe --language French
works perfectly"
—
Reply to this email directly, view it on GitHub
<#649 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4THCS33FUYJDMJRBYMN2Y3XPJSMXANCNFSM6AAAAAASVXZJ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Not *NOW*, months ago before a new release which broke the functionality.
I've raised it and they replied that as it was never an intended feature,
it's not broken and work as it should.
No more translation outside of English.
Le dim. 9 juil. 2023 à 11:24, ShanghaiTimes ***@***.***> a
écrit :
… So Whisper does translate from English to French for example now?
What is the command then, because I can't make it work.
—
Reply to this email directly, view it on GitHub
<#649 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4THCSZXNQEWE6A6G2U3CKDXPJ2DZANCNFSM6AAAAAASVXZJ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
. However, it does sometimes translate. Especially short pieces. For example. (Except for the last line for some reason?) % whisper /Users/robertchalmers/Music/PlayHT/TheWitchesStoryIntroduction.wav --model medium --language fr --task transcribe |
Beta Was this translation helpful? Give feedback.
-
|
A second run through using the "small" model. whisper /Users/robertchalmers/Music/PlayHT/TheWitchesStoryIntroduction.wav --model small --language fr --task transcribe |
Beta Was this translation helpful? Give feedback.
-
|
It's nice to know it's working somewhat, insyead of not at all.
But the results are not too French, Mesium yielded way better results.
Since I only have a 2060 with 6gb VRAM, I've never been able to run Large.
Le dim. 9 juil. 2023 à 12:23, ShanghaiTimes ***@***.***> a
écrit :
… A second run through using the "small" model.
whisper /Users/robertchalmers/Music/PlayHT/TheWitchesStoryIntroduction.wav
--model small --language fr --task transcribe
/usr/local/lib/python3.9/site-packages/whisper/*init*.py:63: UserWarning:
/Users/robertchalmers/.cache/whisper/small.pt exists, but the SHA256
checksum does not match; re-downloading the file
warnings.warn(
100%|███████████████████████████████████████| 461M/461M [04:02<00:00,
1.99MiB/s]
/usr/local/lib/python3.9/site-packages/whisper/transcribe.py:114:
UserWarning: FP16 is not supported on CPU; using FP32 instead
warnings.warn("FP16 is not supported on CPU; using FP32 instead")
[00:00.000 --> 00:04.640] Lili était une jeune ordinaire qui vivait dans
un petit village sur les outskirts de la ville.
[00:04.640 --> 00:08.640] Elle aimait explorer le côté du pays et avait
une faune particulière pour son jardin,
[00:08.640 --> 00:12.000] où elle a pris beaucoup de bonheur, tendant à
ses fleurs et les herbes.
[00:12.640 --> 00:16.080] Un jour, en jouant dans le jardin, Lili s'est
arrêté sur un petit gâteau d'une petite
[00:16.080 --> 00:21.840] peau dans les bouches. Intrigue, elle s'est
arrêtée plus loin et a senti une énergie étrange
[00:21.840 --> 00:27.200] emanant de la peau. Sans hésitation, Lili s'est
arrêté au gâteau et a trouvé un monde
[00:27.280 --> 00:30.880] unlike qu'elle n'avait jamais vu avant. Elle ne
connait rien.
[00:30.880 --> 00:33.840] C'était juste le début de sa voyage magique
comme une jeune fille.
—
Reply to this email directly, view it on GitHub
<#649 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4THCS2YZWSGO2HFE4V4CNTXPKBAJANCNFSM6AAAAAASVXZJ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Yes, only somewhat... I'm on an M2 Mac Mini, with 26GB Ram. The 'large' model only gives me English on a long file of 14 minutes. I'm trying it again now but it takes ages to download the model. whisper "/Users/robertchalmers/Movies/FCPX Movies/Removing and Replacing Selected Colours From An Image.wav" --model large --language French |
Beta Was this translation helpful? Give feedback.
-
|
So it seems very short files translate pretty well. Long ones don't. ackages/whisper/transcribe.py:114: UserWarning: FP16 is not supported on CPU; using FP32 instead |
Beta Was this translation helpful? Give feedback.
-
|
Ok, it seems to depend on the model. This works, all the way through. The Spanish version has dropped into English after 5 sentences. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @ShanghaiTimes, what was your command for this task because I’m trying too to translate an audio to a text in French using whisper but it keeps doing it in English even if I specify that I want the large-v1 model (by the way I’m running my code on google collab if that makes any difference). |
Beta Was this translation helpful? Give feedback.
-
|
hi, have you found a way to translate a video or audio file into french? thanks |
Beta Was this translation helpful? Give feedback.
-
|
90 minutes for 15' of source file?!
I'd have bet the M2 would outperform my RTX.
Last time I've tried with a similar input, I thought it was awfully long,
but lasted way less than 60 minutes of a 2060.
I guess there are no substitute to CUDA cores.
Le dim. 9 juil. 2023 à 14:43, ShanghaiTimes ***@***.***> a
écrit :
… Ok, it seems to depend on the model.
whisper "/Users/robertchalmers/Movies/FCPX Movies/Removing and Replacing
Selected Colours From An Image.wav" --model large --language French
This works, all the way through.
This is a 15:42 wav file. The Mac is an M2 Mac Miini, 24GB RAM.
The French translation took 1 hour 30 minutes.
The Spanish version has dropped into English after 5 sentences.
—
Reply to this email directly, view it on GitHub
<#649 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4THCS2JHU5IVDHZW52TNETXPKRPHANCNFSM6AAAAAASVXZJ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Good to know, as I hope to switch back to Mac.
Le dim. 9 juil. 2023 à 17:00, ShanghaiTimes ***@***.***> a
écrit :
… Whisper is translating for me from English to French and Spanish. Using
big model, and a '5' wait time.
Nice
—
Reply to this email directly, view it on GitHub
<#649 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4THCS35EMDQWHHPKFFRWQLXPLBR3ANCNFSM6AAAAAASVXZJ2I>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
for the time being if u want to use on apple silicon better use https://github.com/ggerganov/whisper.cpp |
Beta Was this translation helpful? Give feedback.
-
|
HiAs I mention in my post, simply using a 5 wait time I think.I haven’t used it in a while now. I gave up.-----Robert ***@***.*** 7 Jan 2024, at 7:47 am, G444-4 ***@***.***> wrote:
Hi @ShanghaiTimes, what was your command for this task because I’m trying too to translate an audio to a text in French using whisper but it keeps doing it in English even if I specify that I want the large-v1 model (by the way I’m running my code on google collab if that makes any difference).
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Is there some news about this feature? |
Beta Was this translation helpful? Give feedback.
-
|
Well, I personally gave up on this method. And used Whisper to transcribe and translate into English and then I other models from that I found on Huggingface or google translate to translate into the other language (results weren't as good as I would hoped to tho, but it might be enough for some). Since GPT4o is supposed to be able to translate by design I think it would be possible to use it from the API (but I think tou would have to pay for it). If you're in one of the listed countries maybe try to use Gemini 🤷♂️ |
Beta Was this translation helpful? Give feedback.
-
|
Hi I've just seen that a company called Gladia Ia could help you |
Beta Was this translation helpful? Give feedback.
-
|
This just worked |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
Translating into languages other than English wasn't a part of the training objective, so that behavior may not be always reliable. I suspect the new
"large-v2"model added in the latest commit 4179ed2 might be the causing the change you are experiencing.If that's the case, you can specify
"large-v1"to use the original large model.