Added agent to get video transcripts. #72

sarfarazsiddiquii · 2024-11-19T20:20:30Z

Added an agent (in reference to issue #70) to retrieve video transcripts in raw format or with timestamps.

Summary by CodeRabbit

New Features
- Introduced the TranscriptionAgent, enabling video transcription with optional timestamp formatting.
- Integrated the TranscriptionAgent into the ChatHandler for enhanced functionality.
Bug Fixes
- Improved error handling for cases where transcripts are not found, ensuring a smoother user experience.

ashish-spext · 2024-11-21T15:49:06Z

Getting the error:

[BACKEND]     "parameters": self.parameters,
[BACKEND]                   ^^^^^^^^^^^^^^^
[BACKEND] AttributeError: 'VideoTranscriptionAgent' object has no attribute 'parameters'. Did you mean: 'get_parameters'?

ashish-spext · 2024-11-21T15:52:07Z

backend/director/agents/transcript_agent.py

+
+logger = logging.getLogger(__name__)
+
+class VideoTranscriptionAgent(BaseAgent):


Naming should be consistent.

1 - Existing agent file doesn't have _agent
transcript_agent -> transcription.py

2 - Agent name VideoTranscriptionAgent -> TranscriptionAgent (since in future we may reuse this for audio transcription as well)

3 - agent_name video_transcription -> transcription

@sarfarazsiddiquii agent name is still video_transcription can you make it simply transcription.

This has been resolved in the recent commit. Sorry for overlooking it.

backend/director/agents/transcript_agent.py

sarfarazsiddiquii · 2024-11-22T00:09:10Z

Hey @ashish-spext, the error 'VideoTranscriptionAgent' object has no attribute 'parameters', naming conventions, and conflicts are resolved in the latest commit.
Thanks.

ashish-spext · 2024-11-22T08:34:57Z

Awesome! Let me test it in a while.

ashish-spext · 2024-11-22T17:36:57Z

backend/director/agents/transcription.py

+            data={"video_id": video_id, "transcript": output_text},
+        )
+
+    def _group_transcript_with_timestamps(self, transcript_text: str, time_range: int) -> str:


The grouping logic is not correct.

If you will test it you will get only one block like this:

Reason: There are no new lines in transcription text, and even if they were new line representing the given range (2 minutes in case of default) is wrong.

Correct way would be to use the transcription dictionary that VideoDB tool is sending it has timing information unlike the transcription text that is being used.

@ashish-spext I’m having a hard time understanding the structure of the transcription dictionary returned by the VideoDB tool.

I think the output of the get_transcript() method, when called with text=False, will give us transcript details.

However, I’m unable to see any changes I’ve made or test the app due to API key limitations:

can you please provide more information about how timing information is stored in transcription dictionary?

This issue can be resolved by adding free LLM models. Merging this PR will fix the problem and it would be helpful for solving similar issues in the future.

@sarfarazsiddiquii We have resolved this issue by adding an OpenAI proxy. An OpenAI key is no longer required. Please pull the latest changes from the main branch, ensure that no OpenAI key is present in the .env file, and test the transcript.

@ankit-v2-3, Thank you for the update, the code is now testable.

I’ve fixed the grouping logic in the latest commit, the agent will now properly group the transcription text into 2 minute intervals by default unless time interval is defined.

Let me know if any change is required.

ashish-spext · 2024-11-22T17:37:45Z

backend/director/agents/transcription.py

+
+        output_text_content.text = output_text
+        output_text_content.status = MsgStatus.success
+        output_text_content.status_message = "Transcription completed successfully."


Message like "Here is your transcription" would be better since we are using that for title of the trascription.

makes sense. I’ve corrected the changes in the recent commit.

…i/Director into TranscriptAgent

…into TranscriptAgent

…i/Director into TranscriptAgent

sarfarazsiddiquii · 2024-11-28T22:25:02Z

Hi @ashish-spext, I've made the requested changes.
Could you please review them and let me know if there's anything else you'd like me to add to this PR?

ashish-spext · 2024-12-06T06:32:05Z

Thanks for the changes.
Can you please pull to resolve the conflicts.
@ankit-v2-3 from our team will take up the review.

coderabbitai · 2024-12-06T10:44:16Z

Walkthrough

The changes introduce a new TranscriptionAgent class in transcription.py, which extends BaseAgent and is responsible for transcribing videos. It includes functionality for optional timestamp formatting and error handling during the transcription process. Additionally, the ChatHandler class in handler.py is updated to incorporate the new agent, ensuring it is part of the agent management system.

Changes

File Path	Change Summary
backend/director/agents/transcription.py	- Added `TranscriptionAgent` class extending `BaseAgent`. - Implemented `run` method for transcription processing. - Added `_group_transcript_with_timestamps` method for formatting transcripts. - Included error handling and logging for the transcription process.
backend/director/handler.py	- Imported `TranscriptionAgent` and added it to the `self.agents` list in `ChatHandler`.

Possibly related PRs

Ashu/fix launch v1 #96: This PR is unrelated as it focuses on modifications to the README.md file, enhancing documentation rather than any code changes or functionality related to the TranscriptionAgent class or its methods.

Suggested reviewers

ashish-spext

Poem

🐰 In the realm of code where agents play,
A new friend hops in to brighten the day.
With transcripts to gather, in timestamps they flow,
Our TranscriptionAgent is ready to show!
So let’s raise a cheer for this coding delight,
Hopping through videos, making words bright! 🌟

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 88b61b0 and f28a1a8.

📒 Files selected for processing (2)

backend/director/agents/transcription.py (1 hunks)
backend/director/handler.py (2 hunks)

🔇 Additional comments (1)

backend/director/handler.py (1)

25-25: Integration of TranscriptionAgent is implemented correctly

The TranscriptionAgent is properly imported (line 25) and added to the agents list (line 61) in the ChatHandler class. This allows the agent to be utilized within the application as intended.

Also applies to: 61-61

backend/director/agents/transcription.py

ankit-v2-3 · 2024-12-17T06:56:02Z

backend/director/agents/transcription.py

+        self.output_message.actions.append("Trying to get the video transcription...")
+        output_text_content = TextContent(
+            agent_name=self.agent_name,
+            status_message="Processing the transcription...",


Could you please keep the ellipses (..) to a max of two in all status messages and actions?

Added agent to get video transcripts

d90ca52

ashish-spext requested review from ankit-v2-3 and ashish-spext November 20, 2024 08:58

ashish-spext requested changes Nov 21, 2024

View reviewed changes

sarfarazsiddiquii added 5 commits November 22, 2024 04:13

added self parameters and naming conventions

28d1248

Added agent to get video transcripts

2555fab

added self parameters and naming conventions

a720e6f

added parameters

91137b6

resolved conflicts

e9c02f6

ashish-spext requested changes Nov 22, 2024

View reviewed changes

sarfarazsiddiquii and others added 8 commits November 23, 2024 00:59

Merge branch 'video-db:main' into TranscriptAgent

4a76907

changes agent name and complete message

7f87786

Merge branch 'TranscriptAgent' of https://github.com/sarfarazsiddiqui…

df887b4

…i/Director into TranscriptAgent

Merge branch 'video-db:main' into TranscriptAgent

ae236a0

Merge branch 'main' of https://github.com/sarfarazsiddiquii/Director …

4e486b8

…into TranscriptAgent

fixed timestamps grouping logic

8211670

Merge branch 'TranscriptAgent' of https://github.com/sarfarazsiddiqui…

a4bd25e

…i/Director into TranscriptAgent

Added timestamp capping.

80df72f

Merge branch 'main' into TranscriptAgent

f28a1a8

coderabbitai bot reviewed Dec 6, 2024

View reviewed changes

backend/director/agents/transcription.py Show resolved Hide resolved

backend/director/agents/transcription.py Show resolved Hide resolved

backend/director/agents/transcription.py Show resolved Hide resolved

ankit-v2-3 requested changes Dec 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added agent to get video transcripts. #72

Added agent to get video transcripts. #72

sarfarazsiddiquii commented Nov 19, 2024 •

edited by coderabbitai bot

Loading

ashish-spext commented Nov 21, 2024

ashish-spext Nov 21, 2024 •

edited

Loading

ashish-spext Nov 22, 2024

sarfarazsiddiquii Nov 22, 2024

sarfarazsiddiquii commented Nov 22, 2024 •

edited

Loading

ashish-spext commented Nov 22, 2024

ashish-spext Nov 22, 2024

sarfarazsiddiquii Nov 22, 2024

sarfarazsiddiquii Nov 23, 2024

ankit-v2-3 Nov 25, 2024

sarfarazsiddiquii Nov 25, 2024 •

edited

Loading

ashish-spext Nov 22, 2024

sarfarazsiddiquii Nov 22, 2024

sarfarazsiddiquii commented Nov 28, 2024

ashish-spext commented Dec 6, 2024

coderabbitai bot commented Dec 6, 2024 •

edited

Loading

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

coderabbitai bot left a comment

ankit-v2-3 Dec 17, 2024


		logger = logging.getLogger(__name__)

		class VideoTranscriptionAgent(BaseAgent):

Added agent to get video transcripts. #72

Are you sure you want to change the base?

Added agent to get video transcripts. #72

Conversation

sarfarazsiddiquii commented Nov 19, 2024 • edited by coderabbitai bot Loading

Summary by CodeRabbit

ashish-spext commented Nov 21, 2024

ashish-spext Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarfarazsiddiquii commented Nov 22, 2024 • edited Loading

ashish-spext commented Nov 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarfarazsiddiquii Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarfarazsiddiquii commented Nov 28, 2024

ashish-spext commented Dec 6, 2024

coderabbitai bot commented Dec 6, 2024 • edited Loading

Walkthrough

Changes

Possibly related PRs

Suggested reviewers

Poem

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

coderabbitai bot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarfarazsiddiquii commented Nov 19, 2024 •

edited by coderabbitai bot

Loading

ashish-spext Nov 21, 2024 •

edited

Loading

sarfarazsiddiquii commented Nov 22, 2024 •

edited

Loading

sarfarazsiddiquii Nov 25, 2024 •

edited

Loading

coderabbitai bot commented Dec 6, 2024 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)