Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding prompt when transcribe with Whisper #462

Merged
merged 16 commits into from
Oct 3, 2023
Merged

Adding prompt when transcribe with Whisper #462

merged 16 commits into from
Oct 3, 2023

Conversation

minhquoc0712
Copy link
Collaborator

@minhquoc0712 minhquoc0712 commented Sep 13, 2023 •
edited
Loading

  • Add initial_prompt to WhisperLearner.infer. initial_prompt is a string that suggests the context of the transcription. For example names of people that will appear in the transcription.
  • The ROS and ROS2 node, documents, and demo are updated accordingly.

@minhquoc0712 minhquoc0712 marked this pull request as ready for review September 25, 2023 21:10
@tsampazk tsampazk added test sources Run style checks test tools Test the toolkit methods labels Sep 26, 2023
@tsampazk tsampazk self-requested a review September 26, 2023 07:01
@tsampazk
Copy link
Collaborator

The tests on speech_transcription/vosk fail. It seems that the certificates of the url used to download vosk models have expired. I think it's this one which indeed gives out a warning of expired certificates when visiting from the browser. It seems they expired today:

Websites prove their identity via certificates, which are valid for a set time period. 
The certificate for alphacephei.com expired on 9/26/2023.
 
Error code: SEC_ERROR_EXPIRED_CERTIFICATE

Found a similar issue here from last year.

One quick fix would be to disable verification here by adding the verify=False argument. However, it still fails down the line on this one. Following a quick search, i couldn't find an easy way to disable verification on that one. @minhquoc0712 could you please take a look at potential fixes and maybe include them on this PR?

@tsampazk tsampazk mentioned this pull request Sep 26, 2023
Merged
@minhquoc0712
Copy link
Collaborator Author

@tsampazk , I run the test_voks.py, and it doesn't have any errors. I run it from my room's WiFi. Can you try again? Perhaps, Vosk developers made some fixes?

@tsampazk
Copy link
Collaborator

tsampazk commented Oct 2, 2023

@minhquoc0712 thanks for testing, it seems that indeed they seem to have fixed the expired certificates. I will provide a review soon.

tsampazk
tsampazk requested changes Oct 2, 2023
View reviewed changes
Copy link
Collaborator

@tsampazk tsampazk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @minhquoc0712, i have added some minor comments.

docs/reference/speech-transcription-whisper.md Outdated Show resolved Hide resolved
projects/opendr_ws/src/opendr_perception/README.md Outdated Show resolved Hide resolved
projects/opendr_ws/src/opendr_perception/README.md Outdated Show resolved Hide resolved
projects/opendr_ws_2/src/opendr_perception/README.md Outdated Show resolved Hide resolved
projects/opendr_ws_2/src/opendr_perception/README.md Outdated Show resolved Hide resolved
minhquoc0712 and others added 5 commits October 2, 2023 11:12
@minhquoc0712 @tsampazk
Update docs/reference/speech-transcription-whisper.md
ed10fdf 
Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
@minhquoc0712 @tsampazk
Update projects/opendr_ws/src/opendr_perception/README.md
84ba044 
Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
@minhquoc0712 @tsampazk
Update projects/opendr_ws/src/opendr_perception/README.md
3668796 
Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
@minhquoc0712 @tsampazk
Update projects/opendr_ws_2/src/opendr_perception/README.md
e244c9d 
Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
@minhquoc0712 @tsampazk
Update projects/opendr_ws_2/src/opendr_perception/README.md
2797c80 
Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
tsampazk
tsampazk approved these changes Oct 2, 2023
View reviewed changes
Copy link
Collaborator

@tsampazk tsampazk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you @minhquoc0712!

omichel
omichel approved these changes Oct 3, 2023
View reviewed changes
Copy link
Collaborator

@omichel omichel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks good to me as well.
Thank you.

@minhquoc0712 minhquoc0712 merged commit 154df38 into develop Oct 3, 2023
47 checks passed
@minhquoc0712 minhquoc0712 deleted the speech-transcription-adding-prompt branch October 3, 2023 10:18
lucamarchionni pushed a commit to lucamarchionni/opendr that referenced this pull request Jun 10, 2024
@minhquoc0712 @tsampazk
Adding prompt when transcribe with Whisper (opendr-eu#462)
b9dc555 
* Adding prompt when transcribe with Whisper

* Update parser argument for ROS1 speech transcription node

* Fix: Change from '-' to '_'

* Add initial prompt to ROS and ROS2 node

* Update documents

* Update demo live with initial prompt

* Update docs/reference/speech-transcription-whisper.md

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>

* Update projects/opendr_ws/src/opendr_perception/README.md

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>

* Update projects/opendr_ws/src/opendr_perception/README.md

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>

* Update projects/opendr_ws_2/src/opendr_perception/README.md

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>

* Update projects/opendr_ws_2/src/opendr_perception/README.md

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>

---------

Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tsampazk tsampazk tsampazk approved these changes

@omichel omichel omichel approved these changes

@passalis passalis Awaiting requested review from passalis passalis is a code owner

@stefaniapedrazzi stefaniapedrazzi Awaiting requested review from stefaniapedrazzi stefaniapedrazzi is a code owner

Assignees
No one assigned
Labels
test sources Run style checks test tools Test the toolkit methods
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@minhquoc0712 @tsampazk @omichel