Skip to content

Commit

Permalink
Typos and licence
Browse files Browse the repository at this point in the history
  • Loading branch information
@Giom-V
Giom-V committed Dec 18, 2024
1 parent 4bd929d commit 47fc37d
Showing 1 changed file with 17 additions and 109 deletions.
126 changes: 17 additions & 109 deletions gemini-2/video_understanding.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "lb5yiH5h8x3h"
},
"source": [
"##### Copyright 2024 Google LLC."
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "VGj9OhLBPVD8"
"id": "906e07f6e562"
},
"outputs": [],
"source": [
Expand Down Expand Up @@ -50,7 +59,7 @@
"id": "3w14yjWnPVD-"
},
"source": [
"Gemini has from the begining been a multidal model, capable of analyzing all sorts of medias using its [long context window](https://developers.googleblog.com/en/new-features-for-the-gemini-api-and-google-ai-studio/).\n",
"Gemini has from the begining been a multimodal model, capable of analyzing all sorts of medias using its [long context window](https://developers.googleblog.com/en/new-features-for-the-gemini-api-and-google-ai-studio/).\n",
"\n",
"[Gemini 2.0](https://ai.google.dev/gemini-api/docs/models/gemini-v2) bring video analysis to a whole new level as illustrated in [this video](https://www.youtube.com/watch?v=Mot-JEU26GQ):\n"
]
Expand Down Expand Up @@ -103,7 +112,7 @@
"\n",
"This section install the SDK, set it up using your [API key](../quickstarts/Authentication.ipynb), imports the relevant libs, downloads the sample videos and upload them to Gemini.\n",
"\n",
"Expand the section if you are curious, but you can also just run it (ity should take a couple of minutes since tehre are large files) and go straight to the examples."
"Expand the section if you are curious, but you can also just run it (it should take a couple of minutes since tehre are large files) and go straight to the examples."
]
},
{
Expand Down Expand Up @@ -306,7 +315,6 @@
}
],
"source": [
"from ctypes import pointer\n",
"import time\n",
"\n",
"def upload_video(video_file_name):\n",
Expand Down Expand Up @@ -373,80 +381,7 @@
"outputs": [
{
"data": {
"text/markdown": [
"```json\n",
"[\n",
" {\n",
" \"timecode\": \"00:00:00\",\n",
" \"caption\": \"A close up of brown fur with visible plant life in the background.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:00:00\",\n",
" \"caption\": \"A gray fox walks into the scene, followed by another fox. The background is a rocky area with some undergrowth. A high pitched sound effect can be heard.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:00:16\",\n",
" \"caption\": \"A grayscale shot of a mountain lion in the forest, walking on some leaves.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:00:35\",\n",
" \"caption\": \"A nighttime grayscale shot of two foxes in the forest. They appear to be digging.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:00:50\",\n",
" \"caption\": \"An overexposed, blurry shot that is most of one solid color, followed by\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:00:51\",\n",
" \"caption\": \"A nighttime shot of some foxes by rocks in a forest. A sound effect can be heard.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:01:04\",\n",
" \"caption\": \"A nighttime shot of a mountain lion in the forest with visible rock formations.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:01:28\",\n",
" \"caption\": \"A night time shot of a bobcat standing by a tree in the forest.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:01:51\",\n",
" \"caption\": \"A medium shot of a black bear in the forest during the day. It turns to its left and moves past the camera.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:01:56\",\n",
" \"caption\": \"A grayscale shot of a mountain lion moving quickly across the screen in a forest.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:02:05\",\n",
" \"caption\":\"A close up of brown fur. The camera moves to reveal a black bear.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:02:22\",\n",
" \"caption\": \"A night time grayscale shot of a fox looking at a lit up city in the background.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:02:35\",\n",
" \"caption\": \"A nighttime grayscale shot of a bear walking near the edge of a cliff.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:02:51\",\n",
" \"caption\": \"A nighttime shot of a mountain lion in the forest.\"\n",
" },\n",
"{\n",
" \"timecode\": \"00:03:04\",\n",
" \"caption\":\"A black bear is standing in the forest looking directly at the camera. The bear is moving. Then a lighter brown bear enters the frame from the right.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:03:22\",\n",
" \"caption\": \"A lighter brown bear stands in the forest, looking to the left.\"\n",
" },\n",
" {\n",
" \"timecode\": \"00:04:21\",\n",
" \"caption\": \"A nighttime shot of a bobcat with glowing eyes standing near a fallen tree limb.\"\n",
" }\n",
"]\n",
"```"
],
"text/markdown": "```json\n[\n {\n \"timecode\": \"00:00:00\",\n \"caption\": \"A close up of brown fur with visible plant life in the background.\"\n },\n {\n \"timecode\": \"00:00:00\",\n \"caption\": \"A gray fox walks into the scene, followed by another fox. The background is a rocky area with some undergrowth. A high pitched sound effect can be heard.\"\n },\n {\n \"timecode\": \"00:00:16\",\n \"caption\": \"A grayscale shot of a mountain lion in the forest, walking on some leaves.\"\n },\n {\n \"timecode\": \"00:00:35\",\n \"caption\": \"A nighttime grayscale shot of two foxes in the forest. They appear to be digging.\"\n },\n {\n \"timecode\": \"00:00:50\",\n \"caption\": \"An overexposed, blurry shot that is most of one solid color, followed by\"\n },\n {\n \"timecode\": \"00:00:51\",\n \"caption\": \"A nighttime shot of some foxes by rocks in a forest. A sound effect can be heard.\"\n },\n {\n \"timecode\": \"00:01:04\",\n \"caption\": \"A nighttime shot of a mountain lion in the forest with visible rock formations.\"\n },\n {\n \"timecode\": \"00:01:28\",\n \"caption\": \"A night time shot of a bobcat standing by a tree in the forest.\"\n },\n {\n \"timecode\": \"00:01:51\",\n \"caption\": \"A medium shot of a black bear in the forest during the day. It turns to its left and moves past the camera.\"\n },\n {\n \"timecode\": \"00:01:56\",\n \"caption\": \"A grayscale shot of a mountain lion moving quickly across the screen in a forest.\"\n },\n {\n \"timecode\": \"00:02:05\",\n \"caption\":\"A close up of brown fur. The camera moves to reveal a black bear.\"\n },\n {\n \"timecode\": \"00:02:22\",\n \"caption\": \"A night time grayscale shot of a fox looking at a lit up city in the background.\"\n },\n {\n \"timecode\": \"00:02:35\",\n \"caption\": \"A nighttime grayscale shot of a bear walking near the edge of a cliff.\"\n },\n {\n \"timecode\": \"00:02:51\",\n \"caption\": \"A nighttime shot of a mountain lion in the forest.\"\n },\n{\n \"timecode\": \"00:03:04\",\n \"caption\":\"A black bear is standing in the forest looking directly at the camera. The bear is moving. Then a lighter brown bear enters the frame from the right.\"\n },\n {\n \"timecode\": \"00:03:22\",\n \"caption\": \"A lighter brown bear stands in the forest, looking to the left.\"\n },\n {\n \"timecode\": \"00:04:21\",\n \"caption\": \"A nighttime shot of a bobcat with glowing eyes standing near a fallen tree limb.\"\n }\n]\n```",
"text/plain": [
"<IPython.core.display.Markdown object>"
]
Expand Down Expand Up @@ -486,7 +421,7 @@
"source": [
"The prompt used is quite a generic one, but you can get even better results if you cutomize it to your needs (like asking specifically for foxes).\n",
"\n",
"The [live demo on AI Studio](https://aistudio.google.com/starter-apps/video) shows how you can postprocess this output to just directly to the the specific part of the video by clicking on the timecodes. If you are interested, you can check the [code of that demo on Github](https://github.com/google-gemini/starter-applets/tree/main/video)."
"The [live demo on AI Studio](https://aistudio.google.com/starter-apps/video) shows how you can postprocess this output to jump directly to the the specific part of the video by clicking on the timecodes. If you are interested, you can check the [code of that demo on Github](https://github.com/google-gemini/starter-applets/tree/main/video)."
]
},
{
Expand All @@ -511,34 +446,7 @@
"outputs": [
{
"data": {
"text/markdown": [
"Certainly! Here's a table of the sticky notes, organized into categories based on their subject matter, along with a few ideas for additional considerations:\n",
"\n",
"**Project Name Brainstorm Sticky Notes**\n",
"\n",
"| Category | Sticky Note Names |\n",
"|--------------------|--------------------|\n",
"| **Constellations** | Canis Major, Leo Minor, Lynx, Delphinus, Serpens, Centaurus, Lyra |\n",
"| **Celestial Objects**| Lunar Eclipse, Stellar Nexus, Comets Tail, Supernova Echo, Orion's Belt, Celestial Drift|\n",
"| **Mythological** | Prometheus Rising, Chimera Dream, Perseus Shield, Zephyr, Odin, Aether, Athena's Eye, Athena, Medusa, Hera, Cerberus, |\n",
"| **Scientific Concepts** | Convergence, Astral Forge, Draco, Euler's Path, Equilibrium, Chaos Theory, Golden Ratio, Infinity Loop, Taylor Series, Stokes Theorem, Bayes Theorem, Riemann's Hypothesis, Fractal, Symmetry, Vector |\n",
"| **Other** | Galactic Core, Titan, Echo, Phoenix, Pandora's Box, Sagitta|\n",
"\n",
"\n",
"**Ideas for What We May Have Missed**\n",
"\n",
"1. **Themes & Subcategories**: Is there a desired theme or subcategory (e.g., \"geometric\", \"mythological\", \"technical\", etc.)? If so, some additional terms or more targeted brainstorming in that area might be beneficial.\n",
" * **Greek Mythology**: Expand on Greek/Roman myths. Consider names like: *Icarus's Fall*, *Pegasus*, *Hades' Gate*, *Poseidon's Trident*.\n",
" * **Astronomy**: More technical names from astronomy like *Solar Flare*, *Accretion Disk*, *Gravitational Lens*, *Quasar Jet*, *Event Horizon*, etc.\n",
"2. **Style**:\n",
" * **Simplicity vs. Complexity**: Do you want a name that is straightforward or something more complex and evocative? Consider a combination of scientific and mythological names.\n",
" * **Acronyms/Initialisms**: Some project names are short acronyms (e.g., ALICE, ATLAS). Perhaps brainstorm some of those if you need a shorter title.\n",
"3. **Audience**: Who is the audience for this project? The name could be more or less technical, depending on whether it's an expert audience or a general public one.\n",
"4. **Uniqueness**: It's often valuable to research if any of these project names are already in use to avoid confusion.\n",
"5. **Team Name/Internal vs. External Name**: Is this a team name or is this a name for a product that the public will see?\n",
"\n",
"Let me know if you’d like me to explore any of these ideas or categories in more detail!"
],
"text/markdown": "Certainly! Here's a table of the sticky notes, organized into categories based on their subject matter, along with a few ideas for additional considerations:\n\n**Project Name Brainstorm Sticky Notes**\n\n| Category | Sticky Note Names |\n|--------------------|--------------------|\n| **Constellations** | Canis Major, Leo Minor, Lynx, Delphinus, Serpens, Centaurus, Lyra |\n| **Celestial Objects**| Lunar Eclipse, Stellar Nexus, Comets Tail, Supernova Echo, Orion's Belt, Celestial Drift|\n| **Mythological** | Prometheus Rising, Chimera Dream, Perseus Shield, Zephyr, Odin, Aether, Athena's Eye, Athena, Medusa, Hera, Cerberus, |\n| **Scientific Concepts** | Convergence, Astral Forge, Draco, Euler's Path, Equilibrium, Chaos Theory, Golden Ratio, Infinity Loop, Taylor Series, Stokes Theorem, Bayes Theorem, Riemann's Hypothesis, Fractal, Symmetry, Vector |\n| **Other** | Galactic Core, Titan, Echo, Phoenix, Pandora's Box, Sagitta|\n\n\n**Ideas for What We May Have Missed**\n\n1. **Themes & Subcategories**: Is there a desired theme or subcategory (e.g., \"geometric\", \"mythological\", \"technical\", etc.)? If so, some additional terms or more targeted brainstorming in that area might be beneficial.\n * **Greek Mythology**: Expand on Greek/Roman myths. Consider names like: *Icarus's Fall*, *Pegasus*, *Hades' Gate*, *Poseidon's Trident*.\n * **Astronomy**: More technical names from astronomy like *Solar Flare*, *Accretion Disk*, *Gravitational Lens*, *Quasar Jet*, *Event Horizon*, etc.\n2. **Style**:\n * **Simplicity vs. Complexity**: Do you want a name that is straightforward or something more complex and evocative? Consider a combination of scientific and mythological names.\n * **Acronyms/Initialisms**: Some project names are short acronyms (e.g., ALICE, ATLAS). Perhaps brainstorm some of those if you need a shorter title.\n3. **Audience**: Who is the audience for this project? The name could be more or less technical, depending on whether it's an expert audience or a general public one.\n4. **Uniqueness**: It's often valuable to research if any of these project names are already in use to avoid confusion.\n5. **Team Name/Internal vs. External Name**: Is this a team name or is this a name for a product that the public will see?\n\nLet me know if you’d like me to explore any of these ideas or categories in more detail!",
"text/plain": [
"<IPython.core.display.Markdown object>"
]
Expand All @@ -549,7 +457,7 @@
}
],
"source": [
"prompt = \"Transcribe the sticky notes, organize them and put it in a table. Do you think we missed anything?\" # @param {type:\"string\"}\n",
"prompt = \"Transcribe the sticky notes, organize them and put it in a table. Do you think anything is missing?\" # @param {type:\"string\"}\n",
"\n",
"video = post_its_video # @param [\"trailcam_video\", \"pottery_video\", \"post_its_video\", \"user_study_video\"] {\"type\":\"raw\",\"allow-input\":true}\n",
"\n",
Expand Down

0 comments on commit 47fc37d

Please sign in to comment.