app/Keyword Extraction with KeyBERT and WordWise.ipynb

{"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"source":"<a href=\"https://www.kaggle.com/code/manishkr1754/keyword-extraction-with-keybert-and-wordwise?scriptVersionId=145452236\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"markdown","source":"<center><h1> Keyword Extraction with KeyBERT and WordWise</h1></center>\n\n---\n\n<center><h3>Comparison, Parameter Tuning & Evaluation</h3></center>","metadata":{"id":"UipK8A-O6_La"}},{"cell_type":"markdown","source":"# Process Flow of the Keyword Extraction\n---\n\n- 1. Getting System Ready\n- 2. Data Collection\n- 3. Data Eyeballing\n- 4. Data Preprocessing\n- 5. Keyword Extraction\n    - 5.1 KeyBERT\n    - 5.2 WordWise\n- 6. Exporting Output as Excel File","metadata":{}},{"cell_type":"markdown","source":"## 1) Getting System Ready\n---\n\nImporting Required Packages","metadata":{}},{"cell_type":"code","source":"!pip install xlsxwriter -q\n!pip install keybert -q\n!pip install wordwise -q","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:31:30.943431Z","iopub.execute_input":"2023-10-06T09:31:30.944219Z","iopub.status.idle":"2023-10-06T09:32:01.96209Z","shell.execute_reply.started":"2023-10-06T09:31:30.944118Z","shell.execute_reply":"2023-10-06T09:32:01.960467Z"},"trusted":true},"execution_count":1,"outputs":[]},{"cell_type":"markdown","source":"1. `!pip install xlsxwriter -q`: This installs the `xlsxwriter` library, which is used for creating Excel files in Python. The `-q` flag suppresses verbose output, so you won't see detailed installation messages.\n\n2. `!pip install keybert -q`: This installs the `keybert` library, which is a Python library for keyword extraction using BERT embeddings. Again, the `-q` flag suppresses verbose output during installation.\n\n3. `!pip install wordwise -q`: This installs the `wordwise` library, which is another library for keyword extraction. The `-q` flag is used to keep the installation process quiet.","metadata":{}},{"cell_type":"code","source":"import requests\nfrom bs4 import BeautifulSoup\nimport pandas as pd\nimport json\nfrom pprint import pprint\nimport xlsxwriter\nfrom keybert import KeyBERT\nfrom wordwise import Extractor","metadata":{"id":"h3_tnXMcQUlQ","execution":{"iopub.status.busy":"2023-10-06T09:32:01.964618Z","iopub.execute_input":"2023-10-06T09:32:01.965015Z","iopub.status.idle":"2023-10-06T09:32:10.309646Z","shell.execute_reply.started":"2023-10-06T09:32:01.964955Z","shell.execute_reply":"2023-10-06T09:32:10.308384Z"},"trusted":true},"execution_count":2,"outputs":[]},{"cell_type":"markdown","source":"- `requests`: This library allows you to send HTTP requests to web servers and retrieve data from websites. It's commonly used for web scraping and accessing web APIs.\n\n- `BeautifulSoup`: BeautifulSoup is a popular Python library for parsing and navigating HTML and XML documents. It helps you extract and manipulate data from web pages easily.\n\n- `pandas`: Pandas is a powerful data manipulation and analysis library. It provides data structures like DataFrames and Series, making it efficient for data cleaning, transformation, and analysis.\n\n- `json`: The `json` module in Python is used for working with JSON (JavaScript Object Notation) data. It allows you to parse JSON strings and serialize Python objects into JSON format.\n\n- `pprint`: The `pprint` module provides a way to pretty-print Python data structures, making complex data easier to read and understand when printed to the console.\n\n- `xlsxwriter`: XlsxWriter is a Python library for creating Excel files (XLSX format). It allows you to generate Excel spreadsheets with various formatting options.\n\n- `KeyBERT`: KeyBERT is a Python library for keyword extraction that leverages BERT embeddings. It's useful for extracting meaningful keywords from text data.\n\n- `wordwise`: Wordwise is another Python library for keyword extraction. It provides a different approach to keyword extraction compared to BERT-based methods.","metadata":{"id":"va0BeOBHDvnm","execution":{"iopub.status.busy":"2023-10-05T17:58:35.15183Z","iopub.execute_input":"2023-10-05T17:58:35.152831Z","iopub.status.idle":"2023-10-05T17:58:43.61255Z","shell.execute_reply.started":"2023-10-05T17:58:35.152794Z","shell.execute_reply":"2023-10-05T17:58:43.611697Z"}}},{"cell_type":"markdown","source":"## 2) Data Collection\n---\nWe aim to gather the necessary data from the **Google Cloud Skill Boost Course** website like module and submodule names along with their descriptions, which will serve as our input for keyword extraction.\n\n- **Data Source:** Our primary data source is the \"Google Cloud Skill Boost Course\" website which contains information about various modules and submodules within the course. These modules are accompanied by descriptions that provide insights into their content.\n\n- **Data Extraction Technique:** We will utilize **Web Scraping** techniques to extract the relevant data from the website. Specifically, we will use the Python library **BeautifulSoup** to parse the HTML content of the web pages and extract module and submodule names along with their descriptions.","metadata":{"id":"BVnuD6Rn6xPU"}},{"cell_type":"markdown","source":"#### Step-1: Define url of the Course Page","metadata":{}},{"cell_type":"code","source":"url = \"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&search_id=25346338\"\nurl","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:10.311011Z","iopub.execute_input":"2023-10-06T09:32:10.311872Z","iopub.status.idle":"2023-10-06T09:32:10.320268Z","shell.execute_reply.started":"2023-10-06T09:32:10.311828Z","shell.execute_reply":"2023-10-06T09:32:10.319089Z"},"trusted":true},"execution_count":3,"outputs":[{"execution_count":3,"output_type":"execute_result","data":{"text/plain":"'https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&search_id=25346338'"},"metadata":{}}]},{"cell_type":"markdown","source":"#### Step-2: Send an HTTP GET request to fetch the page content","metadata":{}},{"cell_type":"code","source":"response_from_page = requests.get(url)\nresponse_from_page","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:10.322169Z","iopub.execute_input":"2023-10-06T09:32:10.322607Z","iopub.status.idle":"2023-10-06T09:32:11.601293Z","shell.execute_reply.started":"2023-10-06T09:32:10.322569Z","shell.execute_reply":"2023-10-06T09:32:11.60006Z"},"trusted":true},"execution_count":4,"outputs":[{"execution_count":4,"output_type":"execute_result","data":{"text/plain":"<Response [200]>"},"metadata":{}}]},{"cell_type":"markdown","source":"The **<Response [200]>** is standard HTTP response code indicating a successful request. It means that the request to the website was successful and the server has returned the expected response. We can now proceed to parse the content of the web page using BeautifulSoup or another HTML parsing library.\n\n**Common Response Code**\n- `200 OK`: The request was successful and the server has returned the requested data.\n- `404 Not Found`: The requested resource was not found on the server.\n- `403 Forbidden`: Access to the requested resource is forbidden.\n- `500 Internal Server Error`: An error occurred on the server while processing the request.","metadata":{}},{"cell_type":"markdown","source":"#### Step-3: Parse the HTML content with BeautifulSoup","metadata":{}},{"cell_type":"code","source":"soup = BeautifulSoup(response_from_page.content, \"html.parser\")","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.603904Z","iopub.execute_input":"2023-10-06T09:32:11.604277Z","iopub.status.idle":"2023-10-06T09:32:11.655504Z","shell.execute_reply.started":"2023-10-06T09:32:11.604247Z","shell.execute_reply":"2023-10-06T09:32:11.654245Z"},"trusted":true},"execution_count":5,"outputs":[]},{"cell_type":"markdown","source":"- `soup = BeautifulSoup(response_from_page.content, \"html.parser\")`: This line of code parses the HTML content of the page using BeautifulSoup's HTML parser and stores the parsed data in the `soup` variable. Now, we can navigate and extract information from the parsed HTML.","metadata":{}},{"cell_type":"markdown","source":"#### Step-4: Extracting Information from Parsed HTML Data","metadata":{}},{"cell_type":"code","source":"ql_course = soup.find(\"ql-course\")\nmodules = ql_course.attrs['modules']","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"YbJm6CLnxJX9","outputId":"d6687d17-ee4a-470a-fb5f-b1045fcbceb9","execution":{"iopub.status.busy":"2023-10-06T09:32:11.65706Z","iopub.execute_input":"2023-10-06T09:32:11.657713Z","iopub.status.idle":"2023-10-06T09:32:11.663124Z","shell.execute_reply.started":"2023-10-06T09:32:11.65768Z","shell.execute_reply":"2023-10-06T09:32:11.661824Z"},"trusted":true},"execution_count":6,"outputs":[]},{"cell_type":"markdown","source":"- `ql_course = soup.find(\"ql-course\")`: Here `find()` method is used to locate the first occurrence of an HTML element with the tag name \"ql-course\" within the parsed HTML. The result is stored in the `ql_course` variable.\n\n- `modules = ql_course.attrs['modules']`: Assuming that the \"ql-course\" element has an attribute named \"modules,\" this line extracts the value of that attribute and stores it in the `modules` variable.","metadata":{}},{"cell_type":"markdown","source":"## 3) Data Eyeballing\n---","metadata":{"execution":{"iopub.status.busy":"2023-10-05T18:31:49.240779Z","iopub.execute_input":"2023-10-05T18:31:49.241126Z","iopub.status.idle":"2023-10-05T18:31:49.248066Z","shell.execute_reply.started":"2023-10-05T18:31:49.241098Z","shell.execute_reply":"2023-10-05T18:31:49.246986Z"}}},{"cell_type":"markdown","source":"#### Parsed HTML Data","metadata":{}},{"cell_type":"code","source":"print(type(soup))\nprint(\"\\n\\n\")\nprint(soup)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.66493Z","iopub.execute_input":"2023-10-06T09:32:11.665375Z","iopub.status.idle":"2023-10-06T09:32:11.69135Z","shell.execute_reply.started":"2023-10-06T09:32:11.665345Z","shell.execute_reply":"2023-10-06T09:32:11.690474Z"},"trusted":true},"execution_count":7,"outputs":[{"name":"stdout","text":"<class 'bs4.BeautifulSoup'>\n\n\n\n\n<!DOCTYPE html>\n\n<html lang=\"en\">\n<head>\n<title>Building Batch Data Pipelines on Google Cloud | Google Cloud Skills Boost</title>\n<meta content=\"/cable\" name=\"action-cable-url\"/>\n<script>\n//<![CDATA[\nwindow.gon={};gon.deployment=\"google-run\";\n//]]>\n</script>\n<script>\n  (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':\n  new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],\n  j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=\n  'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);\n  })(window,document,'script','dataLayer',\"GTM-5XSKHDX\");\n</script>\n<script async=\"async\" src=\"https://www.googletagmanager.com/gtag/js?id=G-2X30ZRBDSG\"></script>\n<script>\n  window.dataLayer = window.dataLayer || [];\n  function gtag(){dataLayer.push(arguments);}\n  gtag('js', new Date());\n  gtag('config', \"G-2X30ZRBDSG\", {\n    user_id: \"\"\n  });\n</script>\n<script src=\"https://cdn.qwiklabs.com/assets/hallofmirrors/polyfills/webcomponents-loader-8583b47b01e40e6ea7bf1ad443a20b7e79bbdd5eafac009806a228c91a97282c.js\"></script>\n<script src=\"https://cdn.qwiklabs.com/assets/vendor-613520545532a2795a401a8168ca8b698b9ee765a66f5ad9ead806b5c32eabe8.js\"></script>\n<script src=\"https://cdn.qwiklabs.com/assets/application-c2770981ed24312fa69ad807ef5d06802f708af0e74810f52be4c583f7ed041b.js\"></script>\n<script src=\"https://cdn.qwiklabs.com/assets/hallofmirrors/hallofmirrors-a6c0129b2af190dd61671b901bc50cb81175f5b4b8df491a433759be034ea9ab.js\"></script>\n<!--[if lt IE 9]>  <![endif]-->\n<script src=\"http://html5shim.googlecode.com/svn/trunk/html5.js\" type=\"text/javascript\"></script>\n<!--[endif]>  <![endif]-->\n<script type=\"application/ld+json\">\n{\"@context\":\"https://schema.org/\",\"@id\":\"https://www.cloudskillsboost.google/course_templates/53\",\"@type\":\"Course\",\"name\":\"Building Batch Data Pipelines on Google Cloud\",\"description\":\"\\u003cp\\u003eData pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. This course describes which paradigm should be used and when for batch data. Furthermore, this course covers several technologies on Google Cloud for data transformation including BigQuery, executing Spark on Dataproc, pipeline graphs in Cloud Data Fusion and serverless data processing with Dataflow. Learners will get hands-on experience building data pipeline components o...\",\"educationalLevel\":\"Beginner\",\"image\":[\"https://cdn.qwiklabs.com/8uLByazcRHEALOwxBVZqyJgmaD8%2FQTvrCDwP1c07dmo%3D\"],\"provider\":{\"@type\":\"Organization\",\"name\":\"Google Cloud Skills Boost\",\"url\":\"https://cloudskillsboost.google\"},\"publisher\":{\"@type\":\"Organization\",\"name\":\"Google Cloud\",\"url\":\"https://cloud.google.com/learn\"},\"datePublished\":\"2023-05-17\",\"inLanguage\":\"en\",\"availableLanguage\":[\"en\"],\"offers\":[{\"@type\":\"Offer\",\"category\":\"Partially Free\"},{\"@type\":\"Offer\",\"category\":\"Subscription\"}],\"hasCourseInstance\":[{\"@type\":\"CourseInstance\",\"courseMode\":\"Online\",\"courseWorkload\":\"P1D\"}]}\n</script>\n<meta content=\"authenticity_token\" name=\"csrf-param\"/>\n<meta content=\"cS05F54Wq3Qqi7oyPTjKx3sZBhGsuSekUP10p3TL/+WmQsquX86FiHK0FH1beJA8El050uSwG1uUjfbT3n1Mog==\" name=\"csrf-token\"/>\n<meta content=\"width=device-width, initial-scale=1.0, user-scalable=yes\" name=\"viewport\"/>\n<meta content=\"1rRsY0INj8RvwB5EF5pwdxt2A2P9aDgAlsICaJ0d5w0\" name=\"google-site-verification\"/>\n<meta content=\"#3681E4\" property=\"msapplication-TileColor\"/>\n<meta content=\"/favicon-144.png\" property=\"msapplication-TileImage\"/>\n<meta content='[{\"id\":\"recaptcha_experiment\",\"optimize_id\":\"dpViOcLkT3qS4TvL2mRojA\",\"title\":\"No Recaptcha shown for trusted users\",\"variant_index\":0,\"variant\":\"original\"}]' name=\"active-experiments\"/>\n<meta content='{\"userId\":null,\"experimentIds\":[\"course_upgrade\",\"canonical_domain_redirect\",\"program_resources\",\"program_learning_assignment\",\"program_announcements\",\"peer_assignment\",\"teams\",\"oauth_risc_shutoff\",\"search_updates\",\"show_annual_purchase_now\",\"programs\",\"used_in\",\"barker\",\"program_groups\",\"enforce_codebuild_verdicts\",\"chat_off_for_signed_out_users\",\"community_forum\",\"alexandria_show_bundle_errors\"]}' name=\"help-api-product-data\"/>\n<meta content='{\"groupIds\":[]}' name=\"help-api-custom-data\"/>\n<meta content=\"&amp;lt;p&amp;gt;Data pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. This course describes which paradigm should be used and when for batch data. Furthermore, this course covers several technologies on Google Cloud for data transformation including BigQuery, executing Spark on Dataproc, pipeline graphs in Cloud Data Fusion and serverless data processing with Dataflow. Learners will get hands-on experience building data pipeline components on Google Cloud using Qwiklabs.&amp;lt;/p&amp;gt;\" name=\"description\"/>\n<meta content=\"Qwiklabs\" name=\"author\"/>\n<meta content=\"Building Batch Data Pipelines on Google Cloud | Google Cloud Skills Boost\" property=\"og:title\"/>\n<meta content=\"website\" property=\"og:type\"/>\n<meta content=\"/favicon-144.png\" property=\"og:image\"/>\n<meta content=\"Qwiklabs\" property=\"og:site_name\"/>\n<meta content=\"&amp;lt;p&amp;gt;Data pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. This course describes which paradigm should be used and when for batch data. Furthermore, this course covers several technologies on Google Cloud for data transformation including BigQuery, executing Spark on Dataproc, pipeline graphs in Cloud Data Fusion and serverless data processing with Dataflow. Learners will get hands-on experience building data pipeline components on Google Cloud using Qwiklabs.&amp;lt;/p&amp;gt;\" property=\"og:description\"/>\n<meta content=\"/qwiklabs_logo_900x887.png\" property=\"og:logo\" size=\"900x887\"/>\n<meta content=\"/qwiklabs_logo_994x187.png\" property=\"og:logo\" size=\"994x187\"/>\n<meta content=\"https://www.cloudskillsboost.google/course_templates/53\" property=\"og:url\"><link href=\"https://www.cloudskillsboost.google/course_templates/53\" rel=\"canonical\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ar&amp;search_id=25346338\" hreflang=\"ar\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=de&amp;search_id=25346338\" hreflang=\"de\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=es&amp;search_id=25346338\" hreflang=\"es\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=fr&amp;search_id=25346338\" hreflang=\"fr\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=fr_CA&amp;search_id=25346338\" hreflang=\"fr-CA\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=he&amp;search_id=25346338\" hreflang=\"he\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=id&amp;search_id=25346338\" hreflang=\"id\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=it&amp;search_id=25346338\" hreflang=\"it\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ja&amp;search_id=25346338\" hreflang=\"ja\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ko&amp;search_id=25346338\" hreflang=\"ko\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pl&amp;search_id=25346338\" hreflang=\"pl\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pt_BR&amp;search_id=25346338\" hreflang=\"pt-BR\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pt_PT&amp;search_id=25346338\" hreflang=\"pt-PT\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ru&amp;search_id=25346338\" hreflang=\"ru\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=tr&amp;search_id=25346338\" hreflang=\"tr\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=uk&amp;search_id=25346338\" hreflang=\"uk\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=zh&amp;search_id=25346338\" hreflang=\"zh\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=zh_TW&amp;search_id=25346338\" hreflang=\"zh-TW\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;search_id=25346338\" hreflang=\"en\" rel=\"alternate\"/>\n<link href=\"https://www.cloudskillsboost.google/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;search_id=25346338\" hreflang=\"x-default\" rel=\"alternate\"/>\n<link color=\"#3681E4\" href=\"/favicon-svg.svg\" rel=\"mask-icon\"/>\n<link href=\"/favicon-180.png\" rel=\"apple-touch-icon-precomposed\"/>\n<link href=\"https://cdn.qwiklabs.com/X46FrQX4iLxHW5MxL8jICvgZM0evMEKscCeQO%2BazGdo%3D\" rel=\"shortcut icon\" type=\"image/x-icon\"/>\n<link href=\"https://fonts.googleapis.com/css?family=Oswald:400|Roboto+Mono:400,700|Roboto:300,400,500,700|Google+Sans:300,400,500,700|Google+Sans+Display:400|Material+Icons|Google+Material+Icons|Google+Sans+Text:400,500,700\" media=\"screen\" rel=\"stylesheet\">\n<link href=\"https://cdn.qwiklabs.com/assets/application-e9a2156b865ab64b2dad2836aa309fedf85704d2ab19d4c8a3cdd85cabbd8364.css\" media=\"all\" rel=\"stylesheet\">\n<link href=\"https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.css\" media=\"screen\" rel=\"stylesheet\">\n<style>\n  :root {\n    --primary-text-on-surface-color: #1a73e8;\n    --primary-text-on-surface-color-dark: #1568d6;\n    --primary-text-on-surface-color-darker: #135ec1;\n    --primary-text-on-surface-color-darkest: #1154ac;\n    --primary-surface-color: #1a73e8;\n    --primary-surface-color-rgb: 26,115,232;\n    --primary-surface-color-light: #d1e3fa;\n    --primary-surface-color-lightest: #e8f1fd;\n    --text-on-primary-color: #ffffff;\n    --accent-text-on-surface-color: #f29900;\n    --accent-surface-color: #f9ab00;\n    --accent-surface-color-rgb: 249,171,0;\n    --accent-surface-color-light: #ffefcc;\n    --text-on-accent-color: #202124;\n  }\n</style>\n<style>\n  :root {\n    --navbar-height: 112px;\n  }\n</style>\n</link></link></link></meta></head>\n<body class=\"course-template-show-body course-layout-body l-full no-nav learner-layout-body\">\n<ql-drawer-container class=\"body-container\">\n<ql-drawer id=\"app-nav\" mode=\"over\" slot=\"drawer\" width=\"288\">\n<div class=\"nav-panel__logo\">\n<div class=\"custom-logo\"><img alt=\"Google Cloud Skills Boost\" aria-label=\"Google Cloud Skills Boost\" height=\"24\" src=\"https://cdn.qwiklabs.com/PGyhmgS3zZncIEGywnx5UXsKwepRRFQ9BhAg%2FWHNrlQ%3D\"/></div>\n</div>\n<nav class=\"ql-sidenav\">\n<ql-sidenav-item href=\"/\" icon=\"home\" label=\"Paths\"></ql-sidenav-item>\n<ql-sidenav-item href=\"/catalog\" icon=\"school\" label=\"Explore\"></ql-sidenav-item>\n</nav>\n</ql-drawer>\n<ql-drawer-content class=\"body-content\" slot=\"drawer-content\">\n<ql-toolbar class=\"sticky app-toolbar\" jumpenabled=\"\" role=\"banner\">\n<div class=\"toolbar-navigation\" slot=\"navigation\">\n<ql-toggle-button class=\"\" for=\"app-nav\" icon=\"menu\" label=\"Toggle navigation drawer\" tip=\"Menu\"></ql-toggle-button>\n</div>\n<div class=\"toolbar-title ql-headline-6\" slot=\"title\"><a aria-label=\"Google Cloud Skills Boost\" class=\"custom-logo\" href=\"/\"><div class=\"custom-logo\"><img alt=\"Google Cloud Skills Boost\" aria-label=\"Google Cloud Skills Boost\" height=\"24\" src=\"https://cdn.qwiklabs.com/PGyhmgS3zZncIEGywnx5UXsKwepRRFQ9BhAg%2FWHNrlQ%3D\"/></div></a>\n</div>\n<div class=\"toolbar-tabs\" role=\"tablist\" slot=\"tabs\"><ql-tab href=\"/\" label=\"Paths\"></ql-tab>\n<ql-tab active=\"\" href=\"/catalog\" label=\"Explore\"></ql-tab>\n<ql-tab href=\"/subscriptions\" label=\"Subscriptions\"></ql-tab>\n</div>\n<div class=\"toolbar-actions\" slot=\"action\">\n<ql-icon-button class=\"mobile-hide\" icon=\"help_outline\" id=\"help-menu-button\" label=\"Open help menu\" tip=\"Help\"></ql-icon-button>\n<ql-menu for=\"help-menu-button\" id=\"help-menu\">\n<ql-menu-item data-analytics-action=\"opened_help\" data-analytics-label=\"course\" label=\"Help Center\" onclick='hallofmirrors.helpService.startHelp({\"productData\":{\"userId\":null},\"context\":\"course\"})'></ql-menu-item>\n<ql-menu-item href=\"mailto:support@qwiklabs.com\" label=\"Email support\"></ql-menu-item>\n</ql-menu>\n<ql-icon-button class=\"mobile-hide\" icon=\"language\" id=\"language\" label=\"Select your language preference\" tip=\"Language\"></ql-icon-button>\n<ql-menu for=\"language\">\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"ar\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ar&amp;search_id=25346338\" label=\"العربية‬‎\" lang=\"ar\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"de\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=de&amp;search_id=25346338\" label=\"Deutsch\" lang=\"de\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"en\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=en&amp;search_id=25346338\" label=\"English\" lang=\"en\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"es\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=es&amp;search_id=25346338\" label=\"español (Latinoamérica)\" lang=\"es\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"fr\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=fr&amp;search_id=25346338\" label=\"français\" lang=\"fr\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"fr_CA\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=fr_CA&amp;search_id=25346338\" label=\"français (Canada)\" lang=\"fr-CA\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"he\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=he&amp;search_id=25346338\" label=\"עברית\" lang=\"he\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"id\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=id&amp;search_id=25346338\" label=\"bahasa Indonesia\" lang=\"id\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"it\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=it&amp;search_id=25346338\" label=\"Italiano\" lang=\"it\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"ja\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ja&amp;search_id=25346338\" label=\"日本語\" lang=\"ja\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"ko\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ko&amp;search_id=25346338\" label=\"한국어\" lang=\"ko\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"pl\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pl&amp;search_id=25346338\" label=\"Polski\" lang=\"pl\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"pt_BR\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pt_BR&amp;search_id=25346338\" label=\"português (Brasil)\" lang=\"pt-BR\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"pt_PT\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=pt_PT&amp;search_id=25346338\" label=\"português (Portugal)\" lang=\"pt-PT\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"ru\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=ru&amp;search_id=25346338\" label=\"русский\" lang=\"ru\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"tr\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=tr&amp;search_id=25346338\" label=\"Türkçe\" lang=\"tr\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"uk\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=uk&amp;search_id=25346338\" label=\"український\" lang=\"uk\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"zh\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=zh&amp;search_id=25346338\" label=\"简体中文\" lang=\"zh\"></ql-menu-item>\n<ql-menu-item data-analytics-action=\"changed_locale\" data-analytics-label=\"zh_TW\" href=\"/course_templates/53?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A0%2C%22has_search%22%3Atrue%7D&amp;locale=zh_TW&amp;search_id=25346338\" label=\"繁體中文\" lang=\"zh-TW\"></ql-menu-item>\n</ql-menu>\n<ql-button data-analytics-action=\"clicked_header_sign_in\" href=\"/users/sign_in\" label=\"Sign in\" text=\"\"></ql-button>\n<ql-button data-analytics-action=\"clicked_header_join\" href=\"/users/sign_up\" label=\"Join\"></ql-button>\n</div>\n</ql-toolbar>\n<ql-toolbar class=\"sticky page-toolbar\">\n<h1 class=\"toolbar-title ql-headline-6\" slot=\"title\">Google Cloud Skills Boost</h1>\n<div class=\"toolbar-actions\" slot=\"action\"></div>\n</ql-toolbar>\n<main id=\"jump-content\">\n<ql-drawer-container class=\"course-container\">\n<ql-drawer class=\"course-drawer course-drawer-left\" open=\"\" slot=\"drawer\" width=\"240\">\n<div class=\"course-outline-header\">\n<div class=\"course-info\">\n<h2 class=\"ql-title-medium\">\nBuilding Batch Data Pipelines on Google Cloud\n</h2>\n</div>\n</div>\n<ql-course-outline coursepath=\"/course_templates/53\" modules='[{\"id\":\"59338\",\"title\":\"Introduction\",\"description\":\"\\u003cp\\u003eIn this module, we introduce the course and agenda\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386567\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379215\",\"href\":null,\"isLocked\":false,\"duration\":55000,\"title\":\"Course Introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59339\",\"title\":\"Introduction to Building Batch Data Pipelines\",\"description\":\"\\u003cp\\u003eThis module reviews different methods of data loading: EL, ELT and ETL and when to use what\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386568\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379216\",\"href\":null,\"isLocked\":false,\"duration\":69000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386569\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379217\",\"href\":null,\"isLocked\":false,\"duration\":220000,\"title\":\"EL, ELT, ETL\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386570\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379218\",\"href\":null,\"isLocked\":false,\"duration\":168000,\"title\":\"Quality considerations\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386571\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379219\",\"href\":null,\"isLocked\":false,\"duration\":180000,\"title\":\"How to carry out operations in BigQuery\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386572\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379220\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Shortcomings\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386573\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379221\",\"href\":null,\"isLocked\":false,\"duration\":428000,\"title\":\"ETL to solve data quality issues\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386574\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379222\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Introduction to Building Batch Data Pipelines\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59340\",\"title\":\"Executing Spark on Dataproc\",\"description\":\"\\u003cp\\u003eThis module shows how to run Hadoop on Dataproc, how to leverage Cloud Storage, and how to optimize your Dataproc jobs.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386575\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379223\",\"href\":null,\"isLocked\":false,\"duration\":27000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386576\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379224\",\"href\":null,\"isLocked\":false,\"duration\":286000,\"title\":\"The Hadoop ecosystem\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386577\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379225\",\"href\":null,\"isLocked\":false,\"duration\":602000,\"title\":\"Running Hadoop on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386578\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379226\",\"href\":null,\"isLocked\":false,\"duration\":379000,\"title\":\"Cloud Storage instead of HDFS\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386579\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379227\",\"href\":null,\"isLocked\":false,\"duration\":172000,\"title\":\"Optimizing Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386580\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379228\",\"href\":null,\"isLocked\":false,\"duration\":567000,\"title\":\"Optimizing Dataproc storage\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386581\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379229\",\"href\":null,\"isLocked\":false,\"duration\":312000,\"title\":\"Optimizing Dataproc templates and autoscaling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386582\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379230\",\"href\":null,\"isLocked\":false,\"duration\":186000,\"title\":\"Optimizing Dataproc monitoring\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386583\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379231\",\"href\":null,\"isLocked\":false,\"duration\":28000,\"title\":\"Lab Intro: Running Apache Spark jobs on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386584\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379232\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Running Apache Spark jobs on Cloud Dataproc\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386585\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379233\",\"href\":null,\"isLocked\":false,\"duration\":34000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386586\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379234\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Executing Spark on Dataproc\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59341\",\"title\":\"Serverless Data Processing with Dataflow\",\"description\":\"\\u003cp\\u003eThis module covers using Dataflow to build your data processing pipelines\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386587\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379235\",\"href\":null,\"isLocked\":false,\"duration\":53000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386588\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379236\",\"href\":null,\"isLocked\":false,\"duration\":342000,\"title\":\"Introduction to Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386589\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379237\",\"href\":null,\"isLocked\":false,\"duration\":171000,\"title\":\"Why customers value Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386590\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379238\",\"href\":null,\"isLocked\":false,\"duration\":228000,\"title\":\"Building Dataflow pipelines in code\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386591\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379239\",\"href\":null,\"isLocked\":false,\"duration\":127000,\"title\":\"Key considerations with designing pipelines\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386592\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379240\",\"href\":null,\"isLocked\":false,\"duration\":188000,\"title\":\"Transforming data with PTransforms\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386593\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379241\",\"href\":null,\"isLocked\":false,\"duration\":10000,\"title\":\"Lab Intro: Building a Simple Dataflow Pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386594\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379242\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"A Simple Dataflow Pipeline (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379243\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: A Simple Dataflow Pipeline (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386595\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379244\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Aggregate with GroupByKey and Combine\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386596\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379245\",\"href\":null,\"isLocked\":false,\"duration\":21000,\"title\":\"Lab Intro: MapReduce in Beam\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386597\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379246\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"MapReduce in Beam (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379247\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Beam: MapReduce in Beam (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386598\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379248\",\"href\":null,\"isLocked\":false,\"duration\":283000,\"title\":\"Side inputs and windows of data\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386599\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379249\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Practicing Pipeline Side Inputs\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386600\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379250\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Python)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379251\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386601\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379252\",\"href\":null,\"isLocked\":false,\"duration\":210000,\"title\":\"Creating and re-using pipeline templates\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386602\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379253\",\"href\":null,\"isLocked\":false,\"duration\":126000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386603\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379254\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Serverless Data Processing with Dataflow\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59342\",\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"description\":\"\\u003cp\\u003eThis module shows how to manage data pipelines with Cloud Data Fusion and Cloud Composer.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386604\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379255\",\"href\":null,\"isLocked\":false,\"duration\":19000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386605\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379256\",\"href\":null,\"isLocked\":false,\"duration\":219000,\"title\":\"Introduction to Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386606\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379257\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Components of Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386607\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379258\",\"href\":null,\"isLocked\":false,\"duration\":95000,\"title\":\"Cloud Data Fusion UI\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386608\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379259\",\"href\":null,\"isLocked\":false,\"duration\":291000,\"title\":\"Build a pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386609\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379260\",\"href\":null,\"isLocked\":false,\"duration\":108000,\"title\":\"Explore data using wrangler\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386610\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379261\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Building and executing a pipeline graph in Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386611\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379262\",\"href\":null,\"isLocked\":false,\"duration\":9000000,\"title\":\"Building and Executing a Pipeline Graph with Data Fusion 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386612\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379263\",\"href\":null,\"isLocked\":false,\"duration\":64000,\"title\":\"Orchestrate work between Google Cloud services with Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386613\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379264\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Apache Airflow environment\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386614\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379265\",\"href\":null,\"isLocked\":false,\"duration\":470000,\"title\":\"DAGs and Operators\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386615\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379266\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Workflow scheduling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386616\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379267\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Monitoring and Logging\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386617\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379268\",\"href\":null,\"isLocked\":false,\"duration\":22000,\"title\":\"Lab Intro: An Introduction to Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386618\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379269\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"An Introduction to Cloud Composer 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386619\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379270\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59343\",\"title\":\"Course Summary\",\"description\":\"\\u003cp\\u003eCourse Summary\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386620\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379271\",\"href\":null,\"isLocked\":false,\"duration\":197000,\"title\":\"Course Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59344\",\"title\":\"Course Resources\",\"description\":\"\\u003cp\\u003ePDF links to all modules\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386621\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379272\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Building Batch Data Pipelines on Google Cloud\",\"type\":\"document\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false}]'></ql-course-outline>\n</ql-drawer>\n<ql-drawer-content class=\"course-activity\" slot=\"drawer-content\">\n<div class=\"course-activity-header\">\n<div class=\"breadcrumbs ql-label-medium\">\n<a href=\"/course_templates/53\">Building Batch Data Pipelines on Google Cloud</a>\n<ql-icon>navigate_next</ql-icon>\nOverview\n</div>\n</div>\n<div class=\"course-activity-content\">\n<div class=\"course-wrapper\">\n<div class=\"course-top-matter\">\n<div class=\"course-title\">\n<ql-activity-label activity=\"course\"></ql-activity-label>\n<div class=\"title-text\">\n<h1 class=\"ql-headline-1\">Building Batch Data Pipelines on Google Cloud</h1>\n</div>\n</div>\n<div class=\"course-details\">\n<span class=\"course-detail\">\n<ql-icon icon=\"date_range\"></ql-icon>\n3 days\n</span>\n<span class=\"course-detail\">\n<ql-icon icon=\"show_chart\"></ql-icon>\nIntroductory\n</span>\n<span class=\"course-detail\">\n<ql-icon>universal_currency_alt</ql-icon>\n45 Credits\n</span>\n</div>\n<div class=\"course-description\"><p>Data pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. This course describes which paradigm should be used and when for batch data. Furthermore, this course covers several technologies on Google Cloud for data transformation including BigQuery, executing Spark on Dataproc, pipeline graphs in Cloud Data Fusion and serverless data processing with Dataflow. Learners will get hands-on experience building data pipeline components on Google Cloud using Qwiklabs.</p></div>\n<div class=\"course-badge\">\n<img alt=\"Badge for Building Batch Data Pipelines on Google Cloud\" src=\"https://cdn.qwiklabs.com/1hVzGcLHWb4EK2Ikakf2Gbd%2FxmXnp7%2BGzoGQfihCAfg%3D\"/>\n<p>When you complete this course, you can earn the badge displayed above! View all the badges you have earned by visiting your profile page. Boost your cloud career by showing the world the skills you have developed!</p>\n</div>\n<div class=\"course-registration\">\n<ql-button href=\"/users/sign_up\" label=\"Join to enroll in this course\"></ql-button>\n</div>\n</div>\n<div class=\"course-curriculum\">\n<ql-course courseid=\"12305\" modules='[{\"id\":\"59338\",\"title\":\"Introduction\",\"description\":\"\\u003cp\\u003eIn this module, we introduce the course and agenda\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386567\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379215\",\"href\":null,\"isLocked\":false,\"duration\":55000,\"title\":\"Course Introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59339\",\"title\":\"Introduction to Building Batch Data Pipelines\",\"description\":\"\\u003cp\\u003eThis module reviews different methods of data loading: EL, ELT and ETL and when to use what\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386568\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379216\",\"href\":null,\"isLocked\":false,\"duration\":69000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386569\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379217\",\"href\":null,\"isLocked\":false,\"duration\":220000,\"title\":\"EL, ELT, ETL\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386570\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379218\",\"href\":null,\"isLocked\":false,\"duration\":168000,\"title\":\"Quality considerations\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386571\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379219\",\"href\":null,\"isLocked\":false,\"duration\":180000,\"title\":\"How to carry out operations in BigQuery\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386572\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379220\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Shortcomings\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386573\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379221\",\"href\":null,\"isLocked\":false,\"duration\":428000,\"title\":\"ETL to solve data quality issues\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386574\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379222\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Introduction to Building Batch Data Pipelines\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59340\",\"title\":\"Executing Spark on Dataproc\",\"description\":\"\\u003cp\\u003eThis module shows how to run Hadoop on Dataproc, how to leverage Cloud Storage, and how to optimize your Dataproc jobs.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386575\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379223\",\"href\":null,\"isLocked\":false,\"duration\":27000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386576\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379224\",\"href\":null,\"isLocked\":false,\"duration\":286000,\"title\":\"The Hadoop ecosystem\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386577\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379225\",\"href\":null,\"isLocked\":false,\"duration\":602000,\"title\":\"Running Hadoop on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386578\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379226\",\"href\":null,\"isLocked\":false,\"duration\":379000,\"title\":\"Cloud Storage instead of HDFS\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386579\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379227\",\"href\":null,\"isLocked\":false,\"duration\":172000,\"title\":\"Optimizing Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386580\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379228\",\"href\":null,\"isLocked\":false,\"duration\":567000,\"title\":\"Optimizing Dataproc storage\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386581\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379229\",\"href\":null,\"isLocked\":false,\"duration\":312000,\"title\":\"Optimizing Dataproc templates and autoscaling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386582\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379230\",\"href\":null,\"isLocked\":false,\"duration\":186000,\"title\":\"Optimizing Dataproc monitoring\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386583\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379231\",\"href\":null,\"isLocked\":false,\"duration\":28000,\"title\":\"Lab Intro: Running Apache Spark jobs on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386584\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379232\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Running Apache Spark jobs on Cloud Dataproc\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386585\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379233\",\"href\":null,\"isLocked\":false,\"duration\":34000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386586\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379234\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Executing Spark on Dataproc\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59341\",\"title\":\"Serverless Data Processing with Dataflow\",\"description\":\"\\u003cp\\u003eThis module covers using Dataflow to build your data processing pipelines\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386587\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379235\",\"href\":null,\"isLocked\":false,\"duration\":53000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386588\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379236\",\"href\":null,\"isLocked\":false,\"duration\":342000,\"title\":\"Introduction to Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386589\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379237\",\"href\":null,\"isLocked\":false,\"duration\":171000,\"title\":\"Why customers value Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386590\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379238\",\"href\":null,\"isLocked\":false,\"duration\":228000,\"title\":\"Building Dataflow pipelines in code\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386591\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379239\",\"href\":null,\"isLocked\":false,\"duration\":127000,\"title\":\"Key considerations with designing pipelines\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386592\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379240\",\"href\":null,\"isLocked\":false,\"duration\":188000,\"title\":\"Transforming data with PTransforms\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386593\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379241\",\"href\":null,\"isLocked\":false,\"duration\":10000,\"title\":\"Lab Intro: Building a Simple Dataflow Pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386594\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379242\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"A Simple Dataflow Pipeline (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379243\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: A Simple Dataflow Pipeline (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386595\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379244\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Aggregate with GroupByKey and Combine\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386596\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379245\",\"href\":null,\"isLocked\":false,\"duration\":21000,\"title\":\"Lab Intro: MapReduce in Beam\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386597\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379246\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"MapReduce in Beam (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379247\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Beam: MapReduce in Beam (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386598\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379248\",\"href\":null,\"isLocked\":false,\"duration\":283000,\"title\":\"Side inputs and windows of data\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386599\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379249\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Practicing Pipeline Side Inputs\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386600\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379250\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Python)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379251\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386601\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379252\",\"href\":null,\"isLocked\":false,\"duration\":210000,\"title\":\"Creating and re-using pipeline templates\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386602\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379253\",\"href\":null,\"isLocked\":false,\"duration\":126000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386603\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379254\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Serverless Data Processing with Dataflow\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59342\",\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"description\":\"\\u003cp\\u003eThis module shows how to manage data pipelines with Cloud Data Fusion and Cloud Composer.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386604\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379255\",\"href\":null,\"isLocked\":false,\"duration\":19000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386605\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379256\",\"href\":null,\"isLocked\":false,\"duration\":219000,\"title\":\"Introduction to Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386606\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379257\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Components of Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386607\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379258\",\"href\":null,\"isLocked\":false,\"duration\":95000,\"title\":\"Cloud Data Fusion UI\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386608\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379259\",\"href\":null,\"isLocked\":false,\"duration\":291000,\"title\":\"Build a pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386609\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379260\",\"href\":null,\"isLocked\":false,\"duration\":108000,\"title\":\"Explore data using wrangler\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386610\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379261\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Building and executing a pipeline graph in Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386611\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379262\",\"href\":null,\"isLocked\":false,\"duration\":9000000,\"title\":\"Building and Executing a Pipeline Graph with Data Fusion 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386612\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379263\",\"href\":null,\"isLocked\":false,\"duration\":64000,\"title\":\"Orchestrate work between Google Cloud services with Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386613\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379264\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Apache Airflow environment\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386614\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379265\",\"href\":null,\"isLocked\":false,\"duration\":470000,\"title\":\"DAGs and Operators\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386615\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379266\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Workflow scheduling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386616\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379267\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Monitoring and Logging\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386617\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379268\",\"href\":null,\"isLocked\":false,\"duration\":22000,\"title\":\"Lab Intro: An Introduction to Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386618\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379269\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"An Introduction to Cloud Composer 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386619\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379270\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59343\",\"title\":\"Course Summary\",\"description\":\"\\u003cp\\u003eCourse Summary\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386620\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379271\",\"href\":null,\"isLocked\":false,\"duration\":197000,\"title\":\"Course Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59344\",\"title\":\"Course Resources\",\"description\":\"\\u003cp\\u003ePDF links to all modules\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386621\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379272\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Building Batch Data Pipelines on Google Cloud\",\"type\":\"document\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false}]'></ql-course>\n<ql-collapsible class=\"course-info\">\n<div class=\"course-info-header\" slot=\"header\">\n<ql-icon>info</ql-icon>\n<div class=\"ql-subhead-1\">Course Info</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\n<ql-expandable-list>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Objectives</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\n<ul>\n<li>\n    Review different methods of data loading: EL, ELT and ETL and when to use what\n  </li>\n<li>\n    Run Hadoop on Dataproc, leverage Cloud Storage, and optimize Dataproc jobs\n  </li>\n<li>\n    Build your data processing pipelines using Dataflow\n  </li>\n<li>\n    Manage data pipelines with Data Fusion and Cloud Composer\n  </li>\n</ul>\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Prerequisites</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nTo benefit from this course, participants should have completed “Google Cloud Big Data and Machine Learning Fundamentals” or have equivalent experience. Participant should also have: • Basic proficiency with a common query language such as SQL. • Experience with data modeling and ETL (extract, transform, load) activities. • Experience with developing applications using a common programming language such as Python. • Familiarity with machine learning and/or statistics\n\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Audience</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nDevelopers responsible for designing pipelines and architectures for data processing.\n\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Available languages</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nEnglish\n\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">What do I do when I finish this course?</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nAfter finishing this course, you can explore additional content in your learning path or browse the catalog.\n\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">What badges can I earn?</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nUpon finishing the required items in a course, you will earn a badge of completion. Badges can be viewed on your profile and shared with your social network.\n\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Interested in taking this course with one of our authorized on-demand partners? </div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nExplore Google Cloud content on <a href=\"https://www.coursera.org/googlecloud\" target=\"_blank\"> Coursera </a> and <a href=\"https://www.pluralsight.com/authors/google-cloud\" target=\"_blank\"> Pluralsight. </a>\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Prefer learning with an instructor? </div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nView the public classroom schedule <a href=\"https://cloud.google.com/training/courses\" target=\"_blank\"> here. </a>\n</div>\n</ql-collapsible>\n<ql-collapsible>\n<div slot=\"header\">\n<div class=\"ql-subhead-2\">Can I take this course for free?</div>\n</div>\n<div class=\"ql-body-2\" slot=\"collapsible\">\nWhen you enroll into most courses, you will be able to consume course materials like videos and documents for free. If a course consists of labs, you will need to purchase an individual subscription or credits to be able consume the labs. Labs can also be unlocked by any campaigns you participate in. All required activities in a course must be completed to be awarded the completion badge.\n\n</div>\n</ql-collapsible>\n</ql-expandable-list>\n</div>\n</ql-collapsible>\n</div>\n</div>\n</div>\n</ql-drawer-content>\n</ql-drawer-container>\n</main>\n<footer class=\"application-footer\">\n<a href=\"/privacy_policy\" target=\"_blank\">Privacy</a>\n<a href=\"/terms_of_service\">Terms</a>\n<div class=\"powered-by\">\n<span aria-hidden=\"\">Powered by</span>\n<img alt=\"Powered by Qwiklabs\" src=\"https://cdn.qwiklabs.com/assets/qwiklabs_logo_grayscale-55ad6830defd63d1284618a702f470409d7044b293858d0dbdd7963d5eb16626.svg\"/>\n</div>\n</footer>\n</ql-drawer-content>\n</ql-drawer-container>\n<span class=\"hidden\" id=\"flash-sibling-before\"></span>\n<ql-snackbar></ql-snackbar>\n<script data-glue-cookie-notification-bar-category=\"2B\" src=\"https://www.gstatic.com/glue/cookienotificationbar/cookienotificationbar.min.js\"></script>\n<ql-dialog dismissallabel=\"Cancel\" headline=\"A newer version of this course is available. Your progress will carry over if you choose to upgrade. However, your completion percentage may change if the new version has added or removed any learning activities. Click the preview button to see the course changes before upgrading.\" icon=\"error\" id=\"course-session-upgrade-modal\">\n<ql-button href=\"/course_templates/53/preview\" label=\"Preview\" slot=\"action\">\nPreview\n</ql-button>\n</ql-dialog>\n<script>\n  $(function() {\n    ql.initMaterialInputs();\n    initChosen();\n    initSearch();\n    initTabs();\n    ql.list.init();\n    ql.favoriting.init();\n    ql.header.myAccount.init();\n    initTooltips();\n    ql.autocomplete.init();\n    ql.modals.init();\n    ql.toggleButtons.init();\n    ql.analytics.init();\n    ql.jumpContent.init();\n  \n  });\n</script>\n</body>\n</html>\n\n","output_type":"stream"}]},{"cell_type":"markdown","source":"#### Information from Parsed HTML Data","metadata":{}},{"cell_type":"code","source":"print(type(modules))\nprint(\"\\n\\n\")\nprint(modules)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.692375Z","iopub.execute_input":"2023-10-06T09:32:11.692706Z","iopub.status.idle":"2023-10-06T09:32:11.699702Z","shell.execute_reply.started":"2023-10-06T09:32:11.692679Z","shell.execute_reply":"2023-10-06T09:32:11.69868Z"},"trusted":true},"execution_count":8,"outputs":[{"name":"stdout","text":"<class 'str'>\n\n\n\n[{\"id\":\"59338\",\"title\":\"Introduction\",\"description\":\"\\u003cp\\u003eIn this module, we introduce the course and agenda\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386567\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379215\",\"href\":null,\"isLocked\":false,\"duration\":55000,\"title\":\"Course Introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59339\",\"title\":\"Introduction to Building Batch Data Pipelines\",\"description\":\"\\u003cp\\u003eThis module reviews different methods of data loading: EL, ELT and ETL and when to use what\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386568\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379216\",\"href\":null,\"isLocked\":false,\"duration\":69000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386569\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379217\",\"href\":null,\"isLocked\":false,\"duration\":220000,\"title\":\"EL, ELT, ETL\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386570\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379218\",\"href\":null,\"isLocked\":false,\"duration\":168000,\"title\":\"Quality considerations\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386571\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379219\",\"href\":null,\"isLocked\":false,\"duration\":180000,\"title\":\"How to carry out operations in BigQuery\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386572\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379220\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Shortcomings\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386573\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379221\",\"href\":null,\"isLocked\":false,\"duration\":428000,\"title\":\"ETL to solve data quality issues\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386574\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379222\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Introduction to Building Batch Data Pipelines\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59340\",\"title\":\"Executing Spark on Dataproc\",\"description\":\"\\u003cp\\u003eThis module shows how to run Hadoop on Dataproc, how to leverage Cloud Storage, and how to optimize your Dataproc jobs.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386575\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379223\",\"href\":null,\"isLocked\":false,\"duration\":27000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386576\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379224\",\"href\":null,\"isLocked\":false,\"duration\":286000,\"title\":\"The Hadoop ecosystem\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386577\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379225\",\"href\":null,\"isLocked\":false,\"duration\":602000,\"title\":\"Running Hadoop on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386578\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379226\",\"href\":null,\"isLocked\":false,\"duration\":379000,\"title\":\"Cloud Storage instead of HDFS\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386579\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379227\",\"href\":null,\"isLocked\":false,\"duration\":172000,\"title\":\"Optimizing Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386580\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379228\",\"href\":null,\"isLocked\":false,\"duration\":567000,\"title\":\"Optimizing Dataproc storage\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386581\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379229\",\"href\":null,\"isLocked\":false,\"duration\":312000,\"title\":\"Optimizing Dataproc templates and autoscaling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386582\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379230\",\"href\":null,\"isLocked\":false,\"duration\":186000,\"title\":\"Optimizing Dataproc monitoring\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386583\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379231\",\"href\":null,\"isLocked\":false,\"duration\":28000,\"title\":\"Lab Intro: Running Apache Spark jobs on Dataproc\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386584\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379232\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Running Apache Spark jobs on Cloud Dataproc\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386585\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379233\",\"href\":null,\"isLocked\":false,\"duration\":34000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386586\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379234\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Executing Spark on Dataproc\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59341\",\"title\":\"Serverless Data Processing with Dataflow\",\"description\":\"\\u003cp\\u003eThis module covers using Dataflow to build your data processing pipelines\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386587\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379235\",\"href\":null,\"isLocked\":false,\"duration\":53000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386588\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379236\",\"href\":null,\"isLocked\":false,\"duration\":342000,\"title\":\"Introduction to Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386589\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379237\",\"href\":null,\"isLocked\":false,\"duration\":171000,\"title\":\"Why customers value Dataflow\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386590\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379238\",\"href\":null,\"isLocked\":false,\"duration\":228000,\"title\":\"Building Dataflow pipelines in code\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386591\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379239\",\"href\":null,\"isLocked\":false,\"duration\":127000,\"title\":\"Key considerations with designing pipelines\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386592\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379240\",\"href\":null,\"isLocked\":false,\"duration\":188000,\"title\":\"Transforming data with PTransforms\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386593\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379241\",\"href\":null,\"isLocked\":false,\"duration\":10000,\"title\":\"Lab Intro: Building a Simple Dataflow Pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386594\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379242\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"A Simple Dataflow Pipeline (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379243\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: A Simple Dataflow Pipeline (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386595\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379244\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Aggregate with GroupByKey and Combine\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386596\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379245\",\"href\":null,\"isLocked\":false,\"duration\":21000,\"title\":\"Lab Intro: MapReduce in Beam\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386597\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379246\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"MapReduce in Beam (Python) 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379247\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Beam: MapReduce in Beam (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386598\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379248\",\"href\":null,\"isLocked\":false,\"duration\":283000,\"title\":\"Side inputs and windows of data\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386599\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379249\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Practicing Pipeline Side Inputs\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386600\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379250\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Python)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false},{\"id\":\"379251\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"Serverless Data Analysis with Dataflow: Side Inputs (Java)\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386601\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379252\",\"href\":null,\"isLocked\":false,\"duration\":210000,\"title\":\"Creating and re-using pipeline templates\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386602\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379253\",\"href\":null,\"isLocked\":false,\"duration\":126000,\"title\":\"Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386603\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379254\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Serverless Data Processing with Dataflow\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59342\",\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"description\":\"\\u003cp\\u003eThis module shows how to manage data pipelines with Cloud Data Fusion and Cloud Composer.\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386604\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379255\",\"href\":null,\"isLocked\":false,\"duration\":19000,\"title\":\"Module introduction\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386605\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379256\",\"href\":null,\"isLocked\":false,\"duration\":219000,\"title\":\"Introduction to Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386606\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379257\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Components of Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386607\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379258\",\"href\":null,\"isLocked\":false,\"duration\":95000,\"title\":\"Cloud Data Fusion UI\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386608\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379259\",\"href\":null,\"isLocked\":false,\"duration\":291000,\"title\":\"Build a pipeline\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386609\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379260\",\"href\":null,\"isLocked\":false,\"duration\":108000,\"title\":\"Explore data using wrangler\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386610\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379261\",\"href\":null,\"isLocked\":false,\"duration\":18000,\"title\":\"Lab Intro: Building and executing a pipeline graph in Cloud Data Fusion\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386611\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379262\",\"href\":null,\"isLocked\":false,\"duration\":9000000,\"title\":\"Building and Executing a Pipeline Graph with Data Fusion 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386612\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379263\",\"href\":null,\"isLocked\":false,\"duration\":64000,\"title\":\"Orchestrate work between Google Cloud services with Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386613\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379264\",\"href\":null,\"isLocked\":false,\"duration\":74000,\"title\":\"Apache Airflow environment\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386614\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379265\",\"href\":null,\"isLocked\":false,\"duration\":470000,\"title\":\"DAGs and Operators\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386615\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379266\",\"href\":null,\"isLocked\":false,\"duration\":335000,\"title\":\"Workflow scheduling\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386616\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379267\",\"href\":null,\"isLocked\":false,\"duration\":208000,\"title\":\"Monitoring and Logging\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386617\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379268\",\"href\":null,\"isLocked\":false,\"duration\":22000,\"title\":\"Lab Intro: An Introduction to Cloud Composer\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386618\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379269\",\"href\":null,\"isLocked\":false,\"duration\":5400000,\"title\":\"An Introduction to Cloud Composer 2.5\",\"type\":\"lab\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false},{\"id\":\"386619\",\"prompt\":null,\"isOptional\":false,\"activities\":[{\"id\":\"379270\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Manage Data Pipelines with Cloud Data Fusion and Cloud Composer\",\"type\":\"quiz\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59343\",\"title\":\"Course Summary\",\"description\":\"\\u003cp\\u003eCourse Summary\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386620\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379271\",\"href\":null,\"isLocked\":false,\"duration\":197000,\"title\":\"Course Summary\",\"type\":\"video\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false},{\"id\":\"59344\",\"title\":\"Course Resources\",\"description\":\"\\u003cp\\u003ePDF links to all modules\\u003c/p\\u003e\",\"steps\":[{\"id\":\"386621\",\"prompt\":null,\"isOptional\":true,\"activities\":[{\"id\":\"379272\",\"href\":null,\"isLocked\":false,\"duration\":0,\"title\":\"Building Batch Data Pipelines on Google Cloud\",\"type\":\"document\",\"isComplete\":false,\"inProgress\":false,\"score\":null,\"disabled\":false}],\"isComplete\":false,\"allActivitiesRequired\":false}],\"expanded\":false}]\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## 4) Data Preprocessing\n___\nBefore performing keyword extraction, it is essential to preprocess the data like text cleaning, tokenization and removing stopwords.\n\n1. **Text Cleaning**: Remove any special characters, HTML tags or irrelevant formatting from the descriptions.\n2. **Tokenization**: Split the text into individual words or tokens.\n3. **Stopword Removal**: Eliminate common stopwords from the text.","metadata":{}},{"cell_type":"markdown","source":"#### Converting the extracted `modules` data (JSON string) into a Python dictionary","metadata":{}},{"cell_type":"code","source":"json_modules = json.loads(modules)\nprint(type(json_modules))\nprint(\"\\n\\n\")\npprint(json_modules[0])","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.701044Z","iopub.execute_input":"2023-10-06T09:32:11.701337Z","iopub.status.idle":"2023-10-06T09:32:11.722667Z","shell.execute_reply.started":"2023-10-06T09:32:11.701313Z","shell.execute_reply":"2023-10-06T09:32:11.721719Z"},"trusted":true},"execution_count":9,"outputs":[{"name":"stdout","text":"<class 'list'>\n\n\n\n{'description': '<p>In this module, we introduce the course and agenda</p>',\n 'expanded': False,\n 'id': '59338',\n 'steps': [{'activities': [{'disabled': False,\n                            'duration': 55000,\n                            'href': None,\n                            'id': '379215',\n                            'inProgress': False,\n                            'isComplete': False,\n                            'isLocked': False,\n                            'score': None,\n                            'title': 'Course Introduction',\n                            'type': 'video'}],\n            'allActivitiesRequired': False,\n            'id': '386567',\n            'isComplete': False,\n            'isOptional': True,\n            'prompt': None}],\n 'title': 'Introduction'}\n","output_type":"stream"}]},{"cell_type":"markdown","source":"- `json_modules = json.loads(modules)`: This line uses the `json.loads()` function to convert the `modules` string assumed to be in JSON format into a Python data structure. In this case, it is a list of dictionaries.","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"RLMdMGqkD8qX","outputId":"dcbae736-343c-4742-aabb-bb5337360a84"}},{"cell_type":"markdown","source":"#### Converting to Pandas DataFrame","metadata":{}},{"cell_type":"code","source":"df = pd.DataFrame(json_modules)\ndf","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":390},"id":"Tm0OAbawuIus","outputId":"b725f526-ea25-412b-da7c-bf356140c699","execution":{"iopub.status.busy":"2023-10-06T09:32:11.724142Z","iopub.execute_input":"2023-10-06T09:32:11.724593Z","iopub.status.idle":"2023-10-06T09:32:11.777286Z","shell.execute_reply.started":"2023-10-06T09:32:11.724564Z","shell.execute_reply":"2023-10-06T09:32:11.77636Z"},"trusted":true},"execution_count":10,"outputs":[{"execution_count":10,"output_type":"execute_result","data":{"text/plain":"      id                                              title  \\\n0  59338                                       Introduction   \n1  59339      Introduction to Building Batch Data Pipelines   \n2  59340                        Executing Spark on Dataproc   \n3  59341           Serverless Data Processing with Dataflow   \n4  59342  Manage Data Pipelines with Cloud Data Fusion a...   \n5  59343                                     Course Summary   \n6  59344                                   Course Resources   \n\n                                         description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module shows how to run Hadoop on Data...   \n3  <p>This module covers using Dataflow to build ...   \n4  <p>This module shows how to manage data pipeli...   \n5                              <p>Course Summary</p>   \n6                    <p>PDF links to all modules</p>   \n\n                                               steps  expanded  \n0  [{'id': '386567', 'prompt': None, 'isOptional'...     False  \n1  [{'id': '386568', 'prompt': None, 'isOptional'...     False  \n2  [{'id': '386575', 'prompt': None, 'isOptional'...     False  \n3  [{'id': '386587', 'prompt': None, 'isOptional'...     False  \n4  [{'id': '386604', 'prompt': None, 'isOptional'...     False  \n5  [{'id': '386620', 'prompt': None, 'isOptional'...     False  \n6  [{'id': '386621', 'prompt': None, 'isOptional'...     False  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>title</th>\n      <th>description</th>\n      <th>steps</th>\n      <th>expanded</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>59338</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>[{'id': '386567', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>59339</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>[{'id': '386568', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>59340</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n      <td>[{'id': '386575', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>59341</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n      <td>[{'id': '386587', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>59342</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n      <td>[{'id': '386604', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>59343</td>\n      <td>Course Summary</td>\n      <td>&lt;p&gt;Course Summary&lt;/p&gt;</td>\n      <td>[{'id': '386620', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>59344</td>\n      <td>Course Resources</td>\n      <td>&lt;p&gt;PDF links to all modules&lt;/p&gt;</td>\n      <td>[{'id': '386621', 'prompt': None, 'isOptional'...</td>\n      <td>False</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"print('The size of Dataframe is: ', df.shape)\nprint('-'*100)\nprint('The Column Name, Record Count and Data Types are as follows: ')\ndf.info()\nprint('-'*100)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.778648Z","iopub.execute_input":"2023-10-06T09:32:11.779207Z","iopub.status.idle":"2023-10-06T09:32:11.793087Z","shell.execute_reply.started":"2023-10-06T09:32:11.779179Z","shell.execute_reply":"2023-10-06T09:32:11.791931Z"},"trusted":true},"execution_count":11,"outputs":[{"name":"stdout","text":"The size of Dataframe is:  (7, 5)\n----------------------------------------------------------------------------------------------------\nThe Column Name, Record Count and Data Types are as follows: \n<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 7 entries, 0 to 6\nData columns (total 5 columns):\n #   Column       Non-Null Count  Dtype \n---  ------       --------------  ----- \n 0   id           7 non-null      object\n 1   title        7 non-null      object\n 2   description  7 non-null      object\n 3   steps        7 non-null      object\n 4   expanded     7 non-null      bool  \ndtypes: bool(1), object(4)\nmemory usage: 359.0+ bytes\n----------------------------------------------------------------------------------------------------\n","output_type":"stream"}]},{"cell_type":"markdown","source":"#### Flattening JSON Data into a Pandas DataFrame","metadata":{}},{"cell_type":"code","source":"flatten_df = pd.json_normalize(json_modules, \n                               record_path=['steps', ['activities']], \n                               meta=['id','title', 'description'], \n                               meta_prefix='meta-', \n                               record_prefix='record-')\nflatten_df.head(3)","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":406},"id":"9qaZXczGLcBJ","outputId":"1b225863-3c64-4a5e-97b1-df2d7cf570d2","execution":{"iopub.status.busy":"2023-10-06T09:32:11.79455Z","iopub.execute_input":"2023-10-06T09:32:11.795562Z","iopub.status.idle":"2023-10-06T09:32:11.823353Z","shell.execute_reply.started":"2023-10-06T09:32:11.795517Z","shell.execute_reply":"2023-10-06T09:32:11.822068Z"},"trusted":true},"execution_count":12,"outputs":[{"execution_count":12,"output_type":"execute_result","data":{"text/plain":"  record-id record-href  record-isLocked  record-duration  \\\n0    379215        None            False            55000   \n1    379216        None            False            69000   \n2    379217        None            False           220000   \n\n          record-title record-type  record-isComplete  record-inProgress  \\\n0  Course Introduction       video              False              False   \n1  Module introduction       video              False              False   \n2         EL, ELT, ETL       video              False              False   \n\n  record-score  record-disabled meta-id  \\\n0         None            False   59338   \n1         None            False   59339   \n2         None            False   59339   \n\n                                      meta-title  \\\n0                                   Introduction   \n1  Introduction to Building Batch Data Pipelines   \n2  Introduction to Building Batch Data Pipelines   \n\n                                    meta-description  \n0  <p>In this module, we introduce the course and...  \n1  <p>This module reviews different methods of da...  \n2  <p>This module reviews different methods of da...  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-id</th>\n      <th>record-href</th>\n      <th>record-isLocked</th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-isComplete</th>\n      <th>record-inProgress</th>\n      <th>record-score</th>\n      <th>record-disabled</th>\n      <th>meta-id</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>379215</td>\n      <td>None</td>\n      <td>False</td>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>False</td>\n      <td>None</td>\n      <td>False</td>\n      <td>59338</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>379216</td>\n      <td>None</td>\n      <td>False</td>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>False</td>\n      <td>None</td>\n      <td>False</td>\n      <td>59339</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>379217</td>\n      <td>None</td>\n      <td>False</td>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>False</td>\n      <td>None</td>\n      <td>False</td>\n      <td>59339</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"This code is taking a JSON data structure with nested arrays, extracting specific nested records while preserving some top-level metadata and converting the structured data into a Pandas DataFrame. The resulting DataFrame will be a tabular representation of the nested JSON data, making it easier to analyze and work with in a tabular format.","metadata":{}},{"cell_type":"code","source":"print('The size of Dataframe is: ', flatten_df.shape)\nprint('-'*100)\nprint('The Column Name, Record Count and Data Types are as follows: ')\nflatten_df.info()\nprint('-'*100)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:11.82488Z","iopub.execute_input":"2023-10-06T09:32:11.825268Z","iopub.status.idle":"2023-10-06T09:32:11.846579Z","shell.execute_reply.started":"2023-10-06T09:32:11.825236Z","shell.execute_reply":"2023-10-06T09:32:11.845371Z"},"trusted":true},"execution_count":13,"outputs":[{"name":"stdout","text":"The size of Dataframe is:  (58, 13)\n----------------------------------------------------------------------------------------------------\nThe Column Name, Record Count and Data Types are as follows: \n<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 58 entries, 0 to 57\nData columns (total 13 columns):\n #   Column             Non-Null Count  Dtype \n---  ------             --------------  ----- \n 0   record-id          58 non-null     object\n 1   record-href        0 non-null      object\n 2   record-isLocked    58 non-null     bool  \n 3   record-duration    58 non-null     int64 \n 4   record-title       58 non-null     object\n 5   record-type        58 non-null     object\n 6   record-isComplete  58 non-null     bool  \n 7   record-inProgress  58 non-null     bool  \n 8   record-score       0 non-null      object\n 9   record-disabled    58 non-null     bool  \n 10  meta-id            58 non-null     object\n 11  meta-title         58 non-null     object\n 12  meta-description   58 non-null     object\ndtypes: bool(4), int64(1), object(8)\nmemory usage: 4.4+ KB\n----------------------------------------------------------------------------------------------------\n","output_type":"stream"}]},{"cell_type":"markdown","source":"#### Dropping Unwanted Columns","metadata":{"execution":{"iopub.status.busy":"2023-10-05T19:05:34.895478Z","iopub.execute_input":"2023-10-05T19:05:34.89587Z","iopub.status.idle":"2023-10-05T19:05:34.901336Z","shell.execute_reply.started":"2023-10-05T19:05:34.895841Z","shell.execute_reply":"2023-10-05T19:05:34.90012Z"}}},{"cell_type":"code","source":"flatten_df.drop(['record-href', \n                 'record-isLocked', \n                 'record-isComplete', \n                 'record-inProgress',\n                 'record-score',\n                 'meta-id',\n                 'record-id'], axis=1, inplace=True)","metadata":{"id":"g07W8PS1UHeq","execution":{"iopub.status.busy":"2023-10-06T09:32:11.849312Z","iopub.execute_input":"2023-10-06T09:32:11.849838Z","iopub.status.idle":"2023-10-06T09:32:11.863392Z","shell.execute_reply.started":"2023-10-06T09:32:11.849807Z","shell.execute_reply":"2023-10-06T09:32:11.862123Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"code","source":"print('The size of Dataframe is: ', flatten_df.shape)\nprint('-'*100)\nprint('The Column Name, Record Count and Data Types are as follows: ')\nflatten_df.info()\nprint('-'*100)","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"n0nSc1otObqh","outputId":"3cc55fa8-2b51-47ce-c332-82d7040ab8eb","execution":{"iopub.status.busy":"2023-10-06T09:32:11.864931Z","iopub.execute_input":"2023-10-06T09:32:11.866296Z","iopub.status.idle":"2023-10-06T09:32:11.888248Z","shell.execute_reply.started":"2023-10-06T09:32:11.866257Z","shell.execute_reply":"2023-10-06T09:32:11.887047Z"},"trusted":true},"execution_count":15,"outputs":[{"name":"stdout","text":"The size of Dataframe is:  (58, 6)\n----------------------------------------------------------------------------------------------------\nThe Column Name, Record Count and Data Types are as follows: \n<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 58 entries, 0 to 57\nData columns (total 6 columns):\n #   Column            Non-Null Count  Dtype \n---  ------            --------------  ----- \n 0   record-duration   58 non-null     int64 \n 1   record-title      58 non-null     object\n 2   record-type       58 non-null     object\n 3   record-disabled   58 non-null     bool  \n 4   meta-title        58 non-null     object\n 5   meta-description  58 non-null     object\ndtypes: bool(1), int64(1), object(4)\nmemory usage: 2.4+ KB\n----------------------------------------------------------------------------------------------------\n","output_type":"stream"}]},{"cell_type":"markdown","source":"#### Aggregating and Grouping the Data","metadata":{"id":"q6_dZQ7v6LVh"}},{"cell_type":"code","source":"grouped_df = flatten_df.groupby(['meta-title', 'meta-description'])\nprint(grouped_df.ngroups)","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vQMhV5uoPEYa","outputId":"b0a053e8-0c09-4aae-8c05-90b3223d6276","execution":{"iopub.status.busy":"2023-10-06T09:32:41.510217Z","iopub.execute_input":"2023-10-06T09:32:41.51066Z","iopub.status.idle":"2023-10-06T09:32:41.521244Z","shell.execute_reply.started":"2023-10-06T09:32:41.510625Z","shell.execute_reply":"2023-10-06T09:32:41.519457Z"},"trusted":true},"execution_count":16,"outputs":[{"name":"stdout","text":"7\n","output_type":"stream"}]},{"cell_type":"markdown","source":"- There are 7(Seven) unique groups(combinations) of `meta-title` and `meta-description` in the grouped DataFrame","metadata":{}},{"cell_type":"code","source":"grouped_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:43.382542Z","iopub.execute_input":"2023-10-06T09:32:43.383209Z","iopub.status.idle":"2023-10-06T09:32:43.399902Z","shell.execute_reply.started":"2023-10-06T09:32:43.383177Z","shell.execute_reply":"2023-10-06T09:32:43.398274Z"},"trusted":true},"execution_count":17,"outputs":[{"execution_count":17,"output_type":"execute_result","data":{"text/plain":"    record-duration                                   record-title  \\\n0             55000                            Course Introduction   \n1             69000                            Module introduction   \n2            220000                                   EL, ELT, ETL   \n3            168000                         Quality considerations   \n4            180000        How to carry out operations in BigQuery   \n5            208000                                   Shortcomings   \n8             27000                            Module introduction   \n9            286000                           The Hadoop ecosystem   \n10           602000                     Running Hadoop on Dataproc   \n11           379000                  Cloud Storage instead of HDFS   \n12           172000                            Optimizing Dataproc   \n20            53000                            Module introduction   \n21           342000                       Introduction to Dataflow   \n22           171000                   Why customers value Dataflow   \n23           228000            Building Dataflow pipelines in code   \n24           127000    Key considerations with designing pipelines   \n40            19000                            Module introduction   \n41           219000              Introduction to Cloud Data Fusion   \n42            74000                Components of Cloud Data Fusion   \n43            95000                           Cloud Data Fusion UI   \n44           291000                               Build a pipeline   \n56           197000                                 Course Summary   \n57                0  Building Batch Data Pipelines on Google Cloud   \n\n   record-type  record-disabled  \\\n0        video            False   \n1        video            False   \n2        video            False   \n3        video            False   \n4        video            False   \n5        video            False   \n8        video            False   \n9        video            False   \n10       video            False   \n11       video            False   \n12       video            False   \n20       video            False   \n21       video            False   \n22       video            False   \n23       video            False   \n24       video            False   \n40       video            False   \n41       video            False   \n42       video            False   \n43       video            False   \n44       video            False   \n56       video            False   \n57    document            False   \n\n                                           meta-title  \\\n0                                        Introduction   \n1       Introduction to Building Batch Data Pipelines   \n2       Introduction to Building Batch Data Pipelines   \n3       Introduction to Building Batch Data Pipelines   \n4       Introduction to Building Batch Data Pipelines   \n5       Introduction to Building Batch Data Pipelines   \n8                         Executing Spark on Dataproc   \n9                         Executing Spark on Dataproc   \n10                        Executing Spark on Dataproc   \n11                        Executing Spark on Dataproc   \n12                        Executing Spark on Dataproc   \n20           Serverless Data Processing with Dataflow   \n21           Serverless Data Processing with Dataflow   \n22           Serverless Data Processing with Dataflow   \n23           Serverless Data Processing with Dataflow   \n24           Serverless Data Processing with Dataflow   \n40  Manage Data Pipelines with Cloud Data Fusion a...   \n41  Manage Data Pipelines with Cloud Data Fusion a...   \n42  Manage Data Pipelines with Cloud Data Fusion a...   \n43  Manage Data Pipelines with Cloud Data Fusion a...   \n44  Manage Data Pipelines with Cloud Data Fusion a...   \n56                                     Course Summary   \n57                                   Course Resources   \n\n                                     meta-description  \n0   <p>In this module, we introduce the course and...  \n1   <p>This module reviews different methods of da...  \n2   <p>This module reviews different methods of da...  \n3   <p>This module reviews different methods of da...  \n4   <p>This module reviews different methods of da...  \n5   <p>This module reviews different methods of da...  \n8   <p>This module shows how to run Hadoop on Data...  \n9   <p>This module shows how to run Hadoop on Data...  \n10  <p>This module shows how to run Hadoop on Data...  \n11  <p>This module shows how to run Hadoop on Data...  \n12  <p>This module shows how to run Hadoop on Data...  \n20  <p>This module covers using Dataflow to build ...  \n21  <p>This module covers using Dataflow to build ...  \n22  <p>This module covers using Dataflow to build ...  \n23  <p>This module covers using Dataflow to build ...  \n24  <p>This module covers using Dataflow to build ...  \n40  <p>This module shows how to manage data pipeli...  \n41  <p>This module shows how to manage data pipeli...  \n42  <p>This module shows how to manage data pipeli...  \n43  <p>This module shows how to manage data pipeli...  \n44  <p>This module shows how to manage data pipeli...  \n56                              <p>Course Summary</p>  \n57                    <p>PDF links to all modules</p>  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>208000</td>\n      <td>Shortcomings</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>27000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>286000</td>\n      <td>The Hadoop ecosystem</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>602000</td>\n      <td>Running Hadoop on Dataproc</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>379000</td>\n      <td>Cloud Storage instead of HDFS</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>172000</td>\n      <td>Optimizing Dataproc</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>53000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>342000</td>\n      <td>Introduction to Dataflow</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>171000</td>\n      <td>Why customers value Dataflow</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>228000</td>\n      <td>Building Dataflow pipelines in code</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>127000</td>\n      <td>Key considerations with designing pipelines</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Serverless Data Processing with Dataflow</td>\n      <td>&lt;p&gt;This module covers using Dataflow to build ...</td>\n    </tr>\n    <tr>\n      <th>40</th>\n      <td>19000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n    </tr>\n    <tr>\n      <th>41</th>\n      <td>219000</td>\n      <td>Introduction to Cloud Data Fusion</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n    </tr>\n    <tr>\n      <th>42</th>\n      <td>74000</td>\n      <td>Components of Cloud Data Fusion</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n    </tr>\n    <tr>\n      <th>43</th>\n      <td>95000</td>\n      <td>Cloud Data Fusion UI</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n    </tr>\n    <tr>\n      <th>44</th>\n      <td>291000</td>\n      <td>Build a pipeline</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Manage Data Pipelines with Cloud Data Fusion a...</td>\n      <td>&lt;p&gt;This module shows how to manage data pipeli...</td>\n    </tr>\n    <tr>\n      <th>56</th>\n      <td>197000</td>\n      <td>Course Summary</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Course Summary</td>\n      <td>&lt;p&gt;Course Summary&lt;/p&gt;</td>\n    </tr>\n    <tr>\n      <th>57</th>\n      <td>0</td>\n      <td>Building Batch Data Pipelines on Google Cloud</td>\n      <td>document</td>\n      <td>False</td>\n      <td>Course Resources</td>\n      <td>&lt;p&gt;PDF links to all modules&lt;/p&gt;</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"#### Combining 'record-title' within Grouped Data","metadata":{}},{"cell_type":"code","source":"flatten_df['text'] = flatten_df.groupby(['meta-title', \n                                         'meta-description'])['record-title'].transform(lambda x:'. '.join(x))\nflatten_df.head(10)","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":553},"id":"vTFY1ok80pyE","outputId":"fa76ed49-3508-451c-ae7d-b1f031dbacee","execution":{"iopub.status.busy":"2023-10-06T09:32:44.687881Z","iopub.execute_input":"2023-10-06T09:32:44.688462Z","iopub.status.idle":"2023-10-06T09:32:44.707332Z","shell.execute_reply.started":"2023-10-06T09:32:44.688423Z","shell.execute_reply":"2023-10-06T09:32:44.706116Z"},"trusted":true},"execution_count":18,"outputs":[{"execution_count":18,"output_type":"execute_result","data":{"text/plain":"   record-duration                                   record-title record-type  \\\n0            55000                            Course Introduction       video   \n1            69000                            Module introduction       video   \n2           220000                                   EL, ELT, ETL       video   \n3           168000                         Quality considerations       video   \n4           180000        How to carry out operations in BigQuery       video   \n5           208000                                   Shortcomings       video   \n6           428000               ETL to solve data quality issues       video   \n7                0  Introduction to Building Batch Data Pipelines        quiz   \n8            27000                            Module introduction       video   \n9           286000                           The Hadoop ecosystem       video   \n\n   record-disabled                                     meta-title  \\\n0            False                                   Introduction   \n1            False  Introduction to Building Batch Data Pipelines   \n2            False  Introduction to Building Batch Data Pipelines   \n3            False  Introduction to Building Batch Data Pipelines   \n4            False  Introduction to Building Batch Data Pipelines   \n5            False  Introduction to Building Batch Data Pipelines   \n6            False  Introduction to Building Batch Data Pipelines   \n7            False  Introduction to Building Batch Data Pipelines   \n8            False                    Executing Spark on Dataproc   \n9            False                    Executing Spark on Dataproc   \n\n                                    meta-description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module reviews different methods of da...   \n3  <p>This module reviews different methods of da...   \n4  <p>This module reviews different methods of da...   \n5  <p>This module reviews different methods of da...   \n6  <p>This module reviews different methods of da...   \n7  <p>This module reviews different methods of da...   \n8  <p>This module shows how to run Hadoop on Data...   \n9  <p>This module shows how to run Hadoop on Data...   \n\n                                                text  \n0                                Course Introduction  \n1  Module introduction. EL, ELT, ETL. Quality con...  \n2  Module introduction. EL, ELT, ETL. Quality con...  \n3  Module introduction. EL, ELT, ETL. Quality con...  \n4  Module introduction. EL, ELT, ETL. Quality con...  \n5  Module introduction. EL, ELT, ETL. Quality con...  \n6  Module introduction. EL, ELT, ETL. Quality con...  \n7  Module introduction. EL, ELT, ETL. Quality con...  \n8  Module introduction. The Hadoop ecosystem. Run...  \n9  Module introduction. The Hadoop ecosystem. Run...  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>Course Introduction</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>208000</td>\n      <td>Shortcomings</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>428000</td>\n      <td>ETL to solve data quality issues</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>0</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>quiz</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>27000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>286000</td>\n      <td>The Hadoop ecosystem</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Executing Spark on Dataproc</td>\n      <td>&lt;p&gt;This module shows how to run Hadoop on Data...</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"This code is adding a new column named `text` to the Pandas DataFrame `flatten_df`. This new column contains concatenated values from the 'record-title' column within each group defined by the 'meta-title' and 'meta-description' columns which can be useful for various analysis and visualization tasks.","metadata":{}},{"cell_type":"markdown","source":"## 5) Keyword Extraction\n___","metadata":{}},{"cell_type":"markdown","source":"### 5.1) KeyBERT\n\n1. `model`: (str) The name of the language model to use for keyword extraction. It specifies the pre-trained model architecture, such as 'distilbert-base-nli-mean-tokens', 'roberta-base-nli-stsb-mean-tokens', 'bert-base-nli-mean-tokens', etc.\n\n2. `top_n`: (int, optional) The number of top keywords to extract. It determines how many keywords will be returned by the model. The default value is 5.\n\n3. `min_kw`: (int, optional) The minimum number of characters required for a keyword to be considered. Keywords shorter than this length are filtered out. The default value is 1.\n\n4. `stop_words`: (str or list, optional) A list of stop words to be removed from the extracted keywords. Stop words are common words like 'the', 'and', 'in', etc., that are often removed to focus on more meaningful keywords. You can specify a list of stop words or 'english' to use a predefined list of English stop words. The default is None, which means no stop words are removed.\n\n5. `use_mmr`: (bool, optional) If True, the Maximum Marginal Relevance (MMR) algorithm is applied to select a diverse set of keywords. MMR promotes diversity by selecting keywords that are not too similar to each other. The default value is False.\n\n6. `diversity`: (float, optional) The diversity parameter for MMR. It controls the trade-off between relevance and diversity. A higher value encourages more diverse keywords. It only applies if `use_mmr` is set to True. The default value is 0.7.\n\n7. `keyphrase_ngram_range`: (tuple, optional) The ngram range for keyphrase extraction. It determines the length of the keyphrases that the model will extract. For example, setting it to `(1, 1)` will extract single words as keyphrases, while `(1, 2)` will extract both single words and two-word phrases as keyphrases. The default value is `(1, 1)","metadata":{}},{"cell_type":"markdown","source":"### Sample Demo KeyBERT","metadata":{}},{"cell_type":"code","source":"sample_text =\"\"\"\n         Supervised learning is the machine learning task of\n         learning a function that maps an input to an output based\n         on example input-output pairs.[1] It infers a function\n         from labeled training data consisting of a set of\n         training examples.[2] In supervised learning, each\n         example is a pair consisting of an input object\n         (typically a vector) and a desired output value (also\n         called the supervisory signal). A supervised learning\n         algorithm analyzes the training data and produces an\n         inferred function, which can be used for mapping new\n         examples. An optimal scenario will allow for the algorithm\n         to correctly determine the class labels for unseen\n         instances. This requires the learning algorithm to\n         generalize from the training data to unseen situations\n         in a 'reasonable' way (see inductive bias).\n      \"\"\"\n\n\nfrom keybert import KeyBERT\nkw_model = KeyBERT()\nkeywords = kw_model.extract_keywords(sample_text)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:50.456862Z","iopub.execute_input":"2023-10-06T09:32:50.45805Z","iopub.status.idle":"2023-10-06T09:32:51.054178Z","shell.execute_reply.started":"2023-10-06T09:32:50.457999Z","shell.execute_reply":"2023-10-06T09:32:51.052882Z"},"trusted":true},"execution_count":19,"outputs":[]},{"cell_type":"code","source":"keywords","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:51.271876Z","iopub.execute_input":"2023-10-06T09:32:51.27231Z","iopub.status.idle":"2023-10-06T09:32:51.280555Z","shell.execute_reply.started":"2023-10-06T09:32:51.272279Z","shell.execute_reply":"2023-10-06T09:32:51.278835Z"},"trusted":true},"execution_count":20,"outputs":[{"execution_count":20,"output_type":"execute_result","data":{"text/plain":"[('supervised', 0.6523),\n ('labeled', 0.4702),\n ('learning', 0.467),\n ('training', 0.3858),\n ('labels', 0.3728)]"},"metadata":{}}]},{"cell_type":"code","source":"keywords = [x[0] for x in keywords]\nkeywords","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:51.829713Z","iopub.execute_input":"2023-10-06T09:32:51.831066Z","iopub.status.idle":"2023-10-06T09:32:51.83918Z","shell.execute_reply.started":"2023-10-06T09:32:51.831015Z","shell.execute_reply":"2023-10-06T09:32:51.838007Z"},"trusted":true},"execution_count":21,"outputs":[{"execution_count":21,"output_type":"execute_result","data":{"text/plain":"['supervised', 'labeled', 'learning', 'training', 'labels']"},"metadata":{}}]},{"cell_type":"markdown","source":"### Extracting Keywords using KeyBERT from course modules","metadata":{}},{"cell_type":"markdown","source":"#### Step-1: Define a list of parameter values to tune","metadata":{}},{"cell_type":"code","source":"ngram_range_values = [(1, 1), (1, 2), (1, 3)]\nstop_words_values = [None, \"english\"]","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:53.779066Z","iopub.execute_input":"2023-10-06T09:32:53.779455Z","iopub.status.idle":"2023-10-06T09:32:53.784811Z","shell.execute_reply.started":"2023-10-06T09:32:53.779427Z","shell.execute_reply":"2023-10-06T09:32:53.783585Z"},"trusted":true},"execution_count":22,"outputs":[]},{"cell_type":"markdown","source":"#### Step-2: Define a list of model names to tune","metadata":{}},{"cell_type":"code","source":"model_names = ['distilbert-base-nli-mean-tokens', \n               'roberta-base-nli-stsb-mean-tokens', \n               'bert-base-nli-mean-tokens']","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:55.004723Z","iopub.execute_input":"2023-10-06T09:32:55.005118Z","iopub.status.idle":"2023-10-06T09:32:55.011152Z","shell.execute_reply.started":"2023-10-06T09:32:55.005079Z","shell.execute_reply":"2023-10-06T09:32:55.009837Z"},"trusted":true},"execution_count":23,"outputs":[]},{"cell_type":"markdown","source":"The models used in the code are all part of the **Hugging Face Transformers library** and are pre-trained models for various natural language processing (NLP) tasks.These models are designed to generate meaningful sentence embeddings and can be used for various NLP tasks including keyword extraction, document similarity and more.\n\n1. `distilbert-base-nli-mean-tokens`: This is a distilled version of the **BERT (Bidirectional Encoder Representations from Transformers) model**. It has been trained on various NLP tasks and is fine-tuned to generate sentence embeddings, making it useful for tasks like keyword extraction and semantic similarity.\n\n2. `roberta-base-nli-stsb-mean-tokens`: This is a **RoBERTa model** which is another variant of the BERT architecture. It has been fine-tuned for the Natural Language Inference (NLI) task and uses sentence embeddings. The \"stsb\" in the name stands for the **Semantic Textual Similarity Benchmark**.\n\n3. `bert-base-nli-mean-tokens`: This is a **standard BERT model** that has been fine-tuned for generating sentence embeddings and is suitable for tasks involving semantic similarity and keyword extraction.","metadata":{}},{"cell_type":"markdown","source":"#### Step-3: Create a function for extracting keywords for a specific model and parameters","metadata":{}},{"cell_type":"code","source":"def get_keywords_keybert(text, model_name, ngram_range, stop_words):\n    \"\"\"\n    Extracts keywords from the given text using the KeyBERT model.\n\n    Parameters:\n    - text (str): The input text from which keywords will be extracted.\n    - model_name (str): The name of the KeyBERT model to use for keyword extraction.\n    - ngram_range (tuple): A tuple specifying the range of n-grams to consider during keyword extraction.\n                          Example: (1, 2) for unigrams and bigrams.\n    - stop_words (str or list): A list of stop words to be removed from the extracted keywords.\n                              It can also be \"english\" to use a predefined list of English stop words.\n                              If None, no stop words are removed.\n\n    Returns:\n    - list of str: A list of extracted keywords from the input text.\n    \"\"\"\n    model = KeyBERT(model_name)\n    keywords_arr = model.extract_keywords(text)\n    return [x[0] for x in keywords_arr]","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:32:58.468724Z","iopub.execute_input":"2023-10-06T09:32:58.469125Z","iopub.status.idle":"2023-10-06T09:32:58.476687Z","shell.execute_reply.started":"2023-10-06T09:32:58.469096Z","shell.execute_reply":"2023-10-06T09:32:58.47524Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"markdown","source":"#### Step-4: Iterate through different models with different parameter combinations","metadata":{}},{"cell_type":"code","source":"for model_name in model_names:\n    for ngram_range in ngram_range_values:\n        for stop_words in stop_words_values:\n            column_name = f'keywords_{model_name}_{ngram_range}_{stop_words}'\n            \n            flatten_df[column_name] = flatten_df['text'].apply(lambda x: get_keywords_keybert(x, \n                                                                                              model_name, \n                                                                                              ngram_range, \n                                                                                              stop_words))","metadata":{"execution":{"iopub.status.busy":"2023-10-06T09:33:00.799348Z","iopub.execute_input":"2023-10-06T09:33:00.799751Z","iopub.status.idle":"2023-10-06T10:04:05.474666Z","shell.execute_reply.started":"2023-10-06T09:33:00.799721Z","shell.execute_reply":"2023-10-06T10:04:05.473417Z"},"trusted":true},"execution_count":25,"outputs":[]},{"cell_type":"markdown","source":"This code is responsible for iterating through a set of predefined model names, n-gram range values, and stop words values to perform keyword extraction for different combinations of models and keyword extraction parameters. It creates new columns in a DataFrame to store the extracted keywords for each combination.\n\n- The outermost loop iterates through a list of model names (`model_names`). These model names represent different pre-trained models for keyword extraction.\n\n- The first nested loop iterates through a list of n-gram range values (`ngram_range_values`). N-gram range specifies the range of n-grams (e.g., unigrams, bigrams) to consider during keyword extraction.\n\n- The second nested loop iterates through a list of stop words values (`stop_words_values`). Stop words are common words (e.g., \"the,\" \"and,\" \"in\") that are often removed from extracted keywords to focus on more meaningful terms.\n\n- For each combination of model name, n-gram range and stop words, a unique column name is generated using string formatting. This column name will be used to store the extracted keywords in the DataFrame.\n\n- Inside the innermost loop, the `get_keywords_keybert` function is applied to each row of the DataFrame. This function takes the input text, the current model name, n-gram range and stop words as parameters. It extracts keywords using the specified model and parameters.\n\n- The extracted keywords are then stored in the corresponding column in the DataFrame.","metadata":{}},{"cell_type":"code","source":"print('The size of Dataframe is: ', flatten_df.shape)\nprint('-'*100)\nprint('The Column Name, Record Count and Data Types are as follows: ')\nflatten_df.info()\nprint('-'*100)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:06:02.805571Z","iopub.execute_input":"2023-10-06T10:06:02.806029Z","iopub.status.idle":"2023-10-06T10:06:02.822344Z","shell.execute_reply.started":"2023-10-06T10:06:02.805989Z","shell.execute_reply":"2023-10-06T10:06:02.821067Z"},"trusted":true},"execution_count":26,"outputs":[{"name":"stdout","text":"The size of Dataframe is:  (58, 25)\n----------------------------------------------------------------------------------------------------\nThe Column Name, Record Count and Data Types are as follows: \n<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 58 entries, 0 to 57\nData columns (total 25 columns):\n #   Column                                                     Non-Null Count  Dtype \n---  ------                                                     --------------  ----- \n 0   record-duration                                            58 non-null     int64 \n 1   record-title                                               58 non-null     object\n 2   record-type                                                58 non-null     object\n 3   record-disabled                                            58 non-null     bool  \n 4   meta-title                                                 58 non-null     object\n 5   meta-description                                           58 non-null     object\n 6   text                                                       58 non-null     object\n 7   keywords_distilbert-base-nli-mean-tokens_(1, 1)_None       58 non-null     object\n 8   keywords_distilbert-base-nli-mean-tokens_(1, 1)_english    58 non-null     object\n 9   keywords_distilbert-base-nli-mean-tokens_(1, 2)_None       58 non-null     object\n 10  keywords_distilbert-base-nli-mean-tokens_(1, 2)_english    58 non-null     object\n 11  keywords_distilbert-base-nli-mean-tokens_(1, 3)_None       58 non-null     object\n 12  keywords_distilbert-base-nli-mean-tokens_(1, 3)_english    58 non-null     object\n 13  keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_None     58 non-null     object\n 14  keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_english  58 non-null     object\n 15  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_None     58 non-null     object\n 16  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  58 non-null     object\n 17  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None     58 non-null     object\n 18  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  58 non-null     object\n 19  keywords_bert-base-nli-mean-tokens_(1, 1)_None             58 non-null     object\n 20  keywords_bert-base-nli-mean-tokens_(1, 1)_english          58 non-null     object\n 21  keywords_bert-base-nli-mean-tokens_(1, 2)_None             58 non-null     object\n 22  keywords_bert-base-nli-mean-tokens_(1, 2)_english          58 non-null     object\n 23  keywords_bert-base-nli-mean-tokens_(1, 3)_None             58 non-null     object\n 24  keywords_bert-base-nli-mean-tokens_(1, 3)_english          58 non-null     object\ndtypes: bool(1), int64(1), object(23)\nmemory usage: 11.1+ KB\n----------------------------------------------------------------------------------------------------\n","output_type":"stream"}]},{"cell_type":"code","source":"flatten_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:06:07.633496Z","iopub.execute_input":"2023-10-06T10:06:07.633916Z","iopub.status.idle":"2023-10-06T10:06:07.674247Z","shell.execute_reply.started":"2023-10-06T10:06:07.633876Z","shell.execute_reply":"2023-10-06T10:06:07.672928Z"},"trusted":true},"execution_count":27,"outputs":[{"execution_count":27,"output_type":"execute_result","data":{"text/plain":"   record-duration                             record-title record-type  \\\n0            55000                      Course Introduction       video   \n1            69000                      Module introduction       video   \n2           220000                             EL, ELT, ETL       video   \n3           168000                   Quality considerations       video   \n4           180000  How to carry out operations in BigQuery       video   \n\n   record-disabled                                     meta-title  \\\n0            False                                   Introduction   \n1            False  Introduction to Building Batch Data Pipelines   \n2            False  Introduction to Building Batch Data Pipelines   \n3            False  Introduction to Building Batch Data Pipelines   \n4            False  Introduction to Building Batch Data Pipelines   \n\n                                    meta-description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module reviews different methods of da...   \n3  <p>This module reviews different methods of da...   \n4  <p>This module reviews different methods of da...   \n\n                                                text  \\\n0                                Course Introduction   \n1  Module introduction. EL, ELT, ETL. Quality con...   \n2  Module introduction. EL, ELT, ETL. Quality con...   \n3  Module introduction. EL, ELT, ETL. Quality con...   \n4  Module introduction. EL, ELT, ETL. Quality con...   \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]     \n1  [pipelines, introduction, batch, building, con...     \n2  [pipelines, introduction, batch, building, con...     \n3  [pipelines, introduction, batch, building, con...     \n4  [pipelines, introduction, batch, building, con...     \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]        \n1  [pipelines, introduction, batch, building, con...        \n2  [pipelines, introduction, batch, building, con...        \n3  [pipelines, introduction, batch, building, con...        \n4  [pipelines, introduction, batch, building, con...        \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 2)_None  ...  \\\n0                             [introduction, course]    ...   \n1  [pipelines, introduction, batch, building, con...    ...   \n2  [pipelines, introduction, batch, building, con...    ...   \n3  [pipelines, introduction, batch, building, con...    ...   \n4  [pipelines, introduction, batch, building, con...    ...   \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_None  \\\n0                             [course, introduction]       \n1          [introduction, etl, module, quality, elt]       \n2          [introduction, etl, module, quality, elt]       \n3          [introduction, etl, module, quality, elt]       \n4          [introduction, etl, module, quality, elt]       \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None  \\\n0                             [course, introduction]       \n1          [introduction, etl, module, quality, elt]       \n2          [introduction, etl, module, quality, elt]       \n3          [introduction, etl, module, quality, elt]       \n4          [introduction, etl, module, quality, elt]       \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n      keywords_bert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 2)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 2)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 3)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 3)_english  \n0                             [introduction, course]  \n1  [introduction, data, pipelines, building, bigq...  \n2  [introduction, data, pipelines, building, bigq...  \n3  [introduction, data, pipelines, building, bigq...  \n4  [introduction, data, pipelines, building, bigq...  \n\n[5 rows x 25 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n      <th>text</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>...</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_english</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>Course Introduction</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>...</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 25 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"### 5.2) WordWise\n\nWordWise is a Python library designed for keyword extraction and keyphrase generation from text data. This library is particularly useful for tasks such as content summarization, document categorization and information retrieval, where identifying key terms can provide valuable insights. \nHowever, this library may not have as extensive and customizable parameter options as some other libraries like KeyBERT.\n\n1. `spacy_model` (str, optional): Specifies the spaCy model to use for text processing. Default is \"en_core_web_sm\".\n\n2. `top_k` (int, optional): Determines the number of top keywords to extract. Default is 5.\n","metadata":{}},{"cell_type":"markdown","source":"### Sample Demo WordWise","metadata":{}},{"cell_type":"code","source":"sample_text =\"\"\"\n         Supervised learning is the machine learning task of\n         learning a function that maps an input to an output based\n         on example input-output pairs.[1] It infers a function\n         from labeled training data consisting of a set of\n         training examples.[2] In supervised learning, each\n         example is a pair consisting of an input object\n         (typically a vector) and a desired output value (also\n         called the supervisory signal). A supervised learning\n         algorithm analyzes the training data and produces an\n         inferred function, which can be used for mapping new\n         examples. An optimal scenario will allow for the algorithm\n         to correctly determine the class labels for unseen\n         instances. This requires the learning algorithm to\n         generalize from the training data to unseen situations\n         in a 'reasonable' way (see inductive bias).\n      \"\"\"\n\n\nfrom wordwise import Extractor\nextractor = Extractor()\nextractor.generate(sample_text)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:06:24.324129Z","iopub.execute_input":"2023-10-06T10:06:24.325327Z","iopub.status.idle":"2023-10-06T10:06:26.31338Z","shell.execute_reply.started":"2023-10-06T10:06:24.325283Z","shell.execute_reply":"2023-10-06T10:06:26.312075Z"},"trusted":true},"execution_count":28,"outputs":[{"execution_count":28,"output_type":"execute_result","data":{"text/plain":"['supervised learning', 'learning', 'inductive bias', 'labels', 'example']"},"metadata":{}}]},{"cell_type":"markdown","source":"### Extracting Keywords using WordWise from course modules","metadata":{"execution":{"iopub.status.busy":"2023-10-06T08:25:19.31548Z","iopub.execute_input":"2023-10-06T08:25:19.315924Z","iopub.status.idle":"2023-10-06T08:25:19.321833Z","shell.execute_reply.started":"2023-10-06T08:25:19.315893Z","shell.execute_reply":"2023-10-06T08:25:19.320764Z"}}},{"cell_type":"markdown","source":"#### Creating Function to get keywords using WordWise","metadata":{}},{"cell_type":"code","source":"def get_keywords_wordwise(text):\n    \"\"\"\n    Extract keywords from the given text using the WordWise library.\n\n    Parameters:\n        text (str): The input text from which keywords are to be extracted.\n\n    Returns:\n        list: A list of extracted keywords or an empty list if the extraction fails.\n\n    Notes:\n        - This function uses the WordWise library to extract keywords from the input text.\n        - If the input text is empty or if an error occurs during keyword extraction, an\n          empty list is returned.\n        - Errors encountered during keyword extraction are printed to the console.\n\n    Example:\n        keywords = get_keywords_wordwise(\"This is a sample text for keyword extraction.\")\n        print(keywords)  # ['sample text', 'keyword extraction']\n    \"\"\"\n    if not text:  # Check if text is empty\n        print(f\"Empty text encountered: {text}\")\n        return []  # Return an empty list if text is empty\n    extractor = Extractor()\n    try:\n        return extractor.generate(text)\n    except Exception as e:\n        print(f\"Error processing text: {text}\")\n        print(f\"Error details: {str(e)}\")\n        return []","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:06:31.509651Z","iopub.execute_input":"2023-10-06T10:06:31.510076Z","iopub.status.idle":"2023-10-06T10:06:31.51731Z","shell.execute_reply.started":"2023-10-06T10:06:31.510045Z","shell.execute_reply":"2023-10-06T10:06:31.515855Z"},"trusted":true},"execution_count":29,"outputs":[]},{"cell_type":"markdown","source":"#### Applying Function to the DataFrame","metadata":{}},{"cell_type":"code","source":"flatten_df['wordwise_keywords'] = flatten_df['text'].apply(get_keywords_wordwise)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:06:38.117754Z","iopub.execute_input":"2023-10-06T10:06:38.118174Z","iopub.status.idle":"2023-10-06T10:08:27.018078Z","shell.execute_reply.started":"2023-10-06T10:06:38.118145Z","shell.execute_reply":"2023-10-06T10:08:27.017129Z"},"trusted":true},"execution_count":30,"outputs":[{"name":"stdout","text":"Error processing text: Course Introduction\nError details: list index out of range\nError processing text: Course Summary\nError details: list index out of range\nError processing text: Building Batch Data Pipelines on Google Cloud\nError details: list index out of range\n","output_type":"stream"}]},{"cell_type":"code","source":"print('The size of Dataframe is: ', flatten_df.shape)\nprint('-'*100)\nprint('The Column Name, Record Count and Data Types are as follows: ')\nflatten_df.info()\nprint('-'*100)","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:08:49.930132Z","iopub.execute_input":"2023-10-06T10:08:49.930546Z","iopub.status.idle":"2023-10-06T10:08:49.947751Z","shell.execute_reply.started":"2023-10-06T10:08:49.930516Z","shell.execute_reply":"2023-10-06T10:08:49.946374Z"},"trusted":true},"execution_count":31,"outputs":[{"name":"stdout","text":"The size of Dataframe is:  (58, 26)\n----------------------------------------------------------------------------------------------------\nThe Column Name, Record Count and Data Types are as follows: \n<class 'pandas.core.frame.DataFrame'>\nRangeIndex: 58 entries, 0 to 57\nData columns (total 26 columns):\n #   Column                                                     Non-Null Count  Dtype \n---  ------                                                     --------------  ----- \n 0   record-duration                                            58 non-null     int64 \n 1   record-title                                               58 non-null     object\n 2   record-type                                                58 non-null     object\n 3   record-disabled                                            58 non-null     bool  \n 4   meta-title                                                 58 non-null     object\n 5   meta-description                                           58 non-null     object\n 6   text                                                       58 non-null     object\n 7   keywords_distilbert-base-nli-mean-tokens_(1, 1)_None       58 non-null     object\n 8   keywords_distilbert-base-nli-mean-tokens_(1, 1)_english    58 non-null     object\n 9   keywords_distilbert-base-nli-mean-tokens_(1, 2)_None       58 non-null     object\n 10  keywords_distilbert-base-nli-mean-tokens_(1, 2)_english    58 non-null     object\n 11  keywords_distilbert-base-nli-mean-tokens_(1, 3)_None       58 non-null     object\n 12  keywords_distilbert-base-nli-mean-tokens_(1, 3)_english    58 non-null     object\n 13  keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_None     58 non-null     object\n 14  keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_english  58 non-null     object\n 15  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_None     58 non-null     object\n 16  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  58 non-null     object\n 17  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None     58 non-null     object\n 18  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  58 non-null     object\n 19  keywords_bert-base-nli-mean-tokens_(1, 1)_None             58 non-null     object\n 20  keywords_bert-base-nli-mean-tokens_(1, 1)_english          58 non-null     object\n 21  keywords_bert-base-nli-mean-tokens_(1, 2)_None             58 non-null     object\n 22  keywords_bert-base-nli-mean-tokens_(1, 2)_english          58 non-null     object\n 23  keywords_bert-base-nli-mean-tokens_(1, 3)_None             58 non-null     object\n 24  keywords_bert-base-nli-mean-tokens_(1, 3)_english          58 non-null     object\n 25  wordwise_keywords                                          58 non-null     object\ndtypes: bool(1), int64(1), object(24)\nmemory usage: 11.5+ KB\n----------------------------------------------------------------------------------------------------\n","output_type":"stream"}]},{"cell_type":"code","source":"flatten_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:08:56.846662Z","iopub.execute_input":"2023-10-06T10:08:56.847097Z","iopub.status.idle":"2023-10-06T10:08:56.891617Z","shell.execute_reply.started":"2023-10-06T10:08:56.847051Z","shell.execute_reply":"2023-10-06T10:08:56.890317Z"},"trusted":true},"execution_count":32,"outputs":[{"execution_count":32,"output_type":"execute_result","data":{"text/plain":"   record-duration                             record-title record-type  \\\n0            55000                      Course Introduction       video   \n1            69000                      Module introduction       video   \n2           220000                             EL, ELT, ETL       video   \n3           168000                   Quality considerations       video   \n4           180000  How to carry out operations in BigQuery       video   \n\n   record-disabled                                     meta-title  \\\n0            False                                   Introduction   \n1            False  Introduction to Building Batch Data Pipelines   \n2            False  Introduction to Building Batch Data Pipelines   \n3            False  Introduction to Building Batch Data Pipelines   \n4            False  Introduction to Building Batch Data Pipelines   \n\n                                    meta-description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module reviews different methods of da...   \n3  <p>This module reviews different methods of da...   \n4  <p>This module reviews different methods of da...   \n\n                                                text  \\\n0                                Course Introduction   \n1  Module introduction. EL, ELT, ETL. Quality con...   \n2  Module introduction. EL, ELT, ETL. Quality con...   \n3  Module introduction. EL, ELT, ETL. Quality con...   \n4  Module introduction. EL, ELT, ETL. Quality con...   \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]     \n1  [pipelines, introduction, batch, building, con...     \n2  [pipelines, introduction, batch, building, con...     \n3  [pipelines, introduction, batch, building, con...     \n4  [pipelines, introduction, batch, building, con...     \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]        \n1  [pipelines, introduction, batch, building, con...        \n2  [pipelines, introduction, batch, building, con...        \n3  [pipelines, introduction, batch, building, con...        \n4  [pipelines, introduction, batch, building, con...        \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 2)_None  ...  \\\n0                             [introduction, course]    ...   \n1  [pipelines, introduction, batch, building, con...    ...   \n2  [pipelines, introduction, batch, building, con...    ...   \n3  [pipelines, introduction, batch, building, con...    ...   \n4  [pipelines, introduction, batch, building, con...    ...   \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None  \\\n0                             [course, introduction]       \n1          [introduction, etl, module, quality, elt]       \n2          [introduction, etl, module, quality, elt]       \n3          [introduction, etl, module, quality, elt]       \n4          [introduction, etl, module, quality, elt]       \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n      keywords_bert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 2)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 2)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 3)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 3)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n                                   wordwise_keywords  \n0                                                 []  \n1  [quality, data, introduction, considerations, ...  \n2  [quality, data, introduction, considerations, ...  \n3  [quality, data, introduction, considerations, ...  \n4  [quality, data, introduction, considerations, ...  \n\n[5 rows x 26 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n      <th>text</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>...</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_english</th>\n      <th>wordwise_keywords</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>Course Introduction</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>...</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[]</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 26 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"#### Reaaranging and Renaming Columns","metadata":{"id":"V-B00Soz5Zuy"}},{"cell_type":"code","source":"index_df = flatten_df.set_index(['meta-title', 'meta-description','record-title'])","metadata":{"id":"yN7mh6aSWoqx","execution":{"iopub.status.busy":"2023-10-06T10:09:03.709409Z","iopub.execute_input":"2023-10-06T10:09:03.709809Z","iopub.status.idle":"2023-10-06T10:09:03.720946Z","shell.execute_reply.started":"2023-10-06T10:09:03.709779Z","shell.execute_reply":"2023-10-06T10:09:03.719825Z"},"trusted":true},"execution_count":33,"outputs":[]},{"cell_type":"code","source":"flatten_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:09:19.8325Z","iopub.execute_input":"2023-10-06T10:09:19.832913Z","iopub.status.idle":"2023-10-06T10:09:19.873251Z","shell.execute_reply.started":"2023-10-06T10:09:19.83288Z","shell.execute_reply":"2023-10-06T10:09:19.872069Z"},"trusted":true},"execution_count":35,"outputs":[{"execution_count":35,"output_type":"execute_result","data":{"text/plain":"   record-duration                             record-title record-type  \\\n0            55000                      Course Introduction       video   \n1            69000                      Module introduction       video   \n2           220000                             EL, ELT, ETL       video   \n3           168000                   Quality considerations       video   \n4           180000  How to carry out operations in BigQuery       video   \n\n   record-disabled                                     meta-title  \\\n0            False                                   Introduction   \n1            False  Introduction to Building Batch Data Pipelines   \n2            False  Introduction to Building Batch Data Pipelines   \n3            False  Introduction to Building Batch Data Pipelines   \n4            False  Introduction to Building Batch Data Pipelines   \n\n                                    meta-description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module reviews different methods of da...   \n3  <p>This module reviews different methods of da...   \n4  <p>This module reviews different methods of da...   \n\n                                                text  \\\n0                                Course Introduction   \n1  Module introduction. EL, ELT, ETL. Quality con...   \n2  Module introduction. EL, ELT, ETL. Quality con...   \n3  Module introduction. EL, ELT, ETL. Quality con...   \n4  Module introduction. EL, ELT, ETL. Quality con...   \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]     \n1  [pipelines, introduction, batch, building, con...     \n2  [pipelines, introduction, batch, building, con...     \n3  [pipelines, introduction, batch, building, con...     \n4  [pipelines, introduction, batch, building, con...     \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]        \n1  [pipelines, introduction, batch, building, con...        \n2  [pipelines, introduction, batch, building, con...        \n3  [pipelines, introduction, batch, building, con...        \n4  [pipelines, introduction, batch, building, con...        \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 2)_None  ...  \\\n0                             [introduction, course]    ...   \n1  [pipelines, introduction, batch, building, con...    ...   \n2  [pipelines, introduction, batch, building, con...    ...   \n3  [pipelines, introduction, batch, building, con...    ...   \n4  [pipelines, introduction, batch, building, con...    ...   \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None  \\\n0                             [course, introduction]       \n1          [introduction, etl, module, quality, elt]       \n2          [introduction, etl, module, quality, elt]       \n3          [introduction, etl, module, quality, elt]       \n4          [introduction, etl, module, quality, elt]       \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n      keywords_bert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 2)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 2)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 3)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 3)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n                                   wordwise_keywords  \n0                                                 []  \n1  [quality, data, introduction, considerations, ...  \n2  [quality, data, introduction, considerations, ...  \n3  [quality, data, introduction, considerations, ...  \n4  [quality, data, introduction, considerations, ...  \n\n[5 rows x 26 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n      <th>text</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>...</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_english</th>\n      <th>wordwise_keywords</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>Course Introduction</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>...</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[]</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 26 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"print(f'index names ----> {index_df.index.names}')\nprint('\\n')\nprint(f'column names ----> {index_df.columns}')","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"9EL86HJ89sW4","outputId":"64c62451-5965-4374-fee1-f4e09be687f4","execution":{"iopub.status.busy":"2023-10-06T10:09:35.431602Z","iopub.execute_input":"2023-10-06T10:09:35.43205Z","iopub.status.idle":"2023-10-06T10:09:35.439554Z","shell.execute_reply.started":"2023-10-06T10:09:35.432016Z","shell.execute_reply":"2023-10-06T10:09:35.438263Z"},"trusted":true},"execution_count":36,"outputs":[{"name":"stdout","text":"index names ----> ['meta-title', 'meta-description', 'record-title']\n\n\ncolumn names ----> Index(['record-duration', 'record-type', 'record-disabled', 'text',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 1)_None',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 1)_english',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 2)_None',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 2)_english',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 3)_None',\n       'keywords_distilbert-base-nli-mean-tokens_(1, 3)_english',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_None',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 1)_english',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_None',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None',\n       'keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english',\n       'keywords_bert-base-nli-mean-tokens_(1, 1)_None',\n       'keywords_bert-base-nli-mean-tokens_(1, 1)_english',\n       'keywords_bert-base-nli-mean-tokens_(1, 2)_None',\n       'keywords_bert-base-nli-mean-tokens_(1, 2)_english',\n       'keywords_bert-base-nli-mean-tokens_(1, 3)_None',\n       'keywords_bert-base-nli-mean-tokens_(1, 3)_english',\n       'wordwise_keywords'],\n      dtype='object')\n","output_type":"stream"}]},{"cell_type":"code","source":"index_df.index.names = ['Titles', 'Description', 'Activities']\nindex_df.rename(columns={'record-duration':'Duration', 'record-type':'Type'}, errors='raise')","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"u8cMdVGz9fHg","outputId":"1ebe58b2-c98d-4bbf-c20d-42101c944495","execution":{"iopub.status.busy":"2023-10-06T10:10:07.240249Z","iopub.execute_input":"2023-10-06T10:10:07.240753Z","iopub.status.idle":"2023-10-06T10:10:07.498774Z","shell.execute_reply.started":"2023-10-06T10:10:07.240716Z","shell.execute_reply":"2023-10-06T10:10:07.497588Z"},"trusted":true},"execution_count":37,"outputs":[{"execution_count":37,"output_type":"execute_result","data":{"text/plain":"                                                                                                                                                          Duration  \\\nTitles                                             Description                                        Activities                                                     \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                    55000   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                    69000   \n                                                                                                      EL, ELT, ETL                                          220000   \n                                                                                                      Quality considerations                                168000   \n                                                                                                      How to carry out operations in BigQuery               180000   \n                                                                                                      Shortcomings                                          208000   \n                                                                                                      ETL to solve data quality issues                      428000   \n                                                                                                      Introduction to Building Batch Data Pipelines              0   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                    27000   \n                                                                                                      The Hadoop ecosystem                                  286000   \n                                                                                                      Running Hadoop on Dataproc                            602000   \n                                                                                                      Cloud Storage instead of HDFS                         379000   \n                                                                                                      Optimizing Dataproc                                   172000   \n                                                                                                      Optimizing Dataproc storage                           567000   \n                                                                                                      Optimizing Dataproc templates and autoscaling         312000   \n                                                                                                      Optimizing Dataproc monitoring                        186000   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc       28000   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc          5400000   \n                                                                                                      Summary                                                34000   \n                                                                                                      Executing Spark on Dataproc                                0   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                    53000   \n                                                                                                      Introduction to Dataflow                              342000   \n                                                                                                      Why customers value Dataflow                          171000   \n                                                                                                      Building Dataflow pipelines in code                   228000   \n                                                                                                      Key considerations with designing pipelines           127000   \n                                                                                                      Transforming data with PTransforms                    188000   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline         10000   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5              5400000   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...   5400000   \n                                                                                                      Aggregate with GroupByKey and Combine                 335000   \n                                                                                                      Lab Intro: MapReduce in Beam                           21000   \n                                                                                                      MapReduce in Beam (Python) 2.5                       5400000   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...   5400000   \n                                                                                                      Side inputs and windows of data                       283000   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs             18000   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...   5400000   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...   5400000   \n                                                                                                      Creating and re-using pipeline templates              210000   \n                                                                                                      Summary                                               126000   \n                                                                                                      Serverless Data Processing with Dataflow                   0   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                    19000   \n                                                                                                      Introduction to Cloud Data Fusion                     219000   \n                                                                                                      Components of Cloud Data Fusion                        74000   \n                                                                                                      Cloud Data Fusion UI                                   95000   \n                                                                                                      Build a pipeline                                      291000   \n                                                                                                      Explore data using wrangler                           108000   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...     18000   \n                                                                                                      Building and Executing a Pipeline Graph with Da...   9000000   \n                                                                                                      Orchestrate work between Google Cloud services ...     64000   \n                                                                                                      Apache Airflow environment                             74000   \n                                                                                                      DAGs and Operators                                    470000   \n                                                                                                      Workflow scheduling                                   335000   \n                                                                                                      Monitoring and Logging                                208000   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer           22000   \n                                                                                                      An Introduction to Cloud Composer 2.5                5400000   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...         0   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                        197000   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              0   \n\n                                                                                                                                                              Type  \\\nTitles                                             Description                                        Activities                                                     \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                    video   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                    video   \n                                                                                                      EL, ELT, ETL                                           video   \n                                                                                                      Quality considerations                                 video   \n                                                                                                      How to carry out operations in BigQuery                video   \n                                                                                                      Shortcomings                                           video   \n                                                                                                      ETL to solve data quality issues                       video   \n                                                                                                      Introduction to Building Batch Data Pipelines           quiz   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                    video   \n                                                                                                      The Hadoop ecosystem                                   video   \n                                                                                                      Running Hadoop on Dataproc                             video   \n                                                                                                      Cloud Storage instead of HDFS                          video   \n                                                                                                      Optimizing Dataproc                                    video   \n                                                                                                      Optimizing Dataproc storage                            video   \n                                                                                                      Optimizing Dataproc templates and autoscaling          video   \n                                                                                                      Optimizing Dataproc monitoring                         video   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc       video   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc              lab   \n                                                                                                      Summary                                                video   \n                                                                                                      Executing Spark on Dataproc                             quiz   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                    video   \n                                                                                                      Introduction to Dataflow                               video   \n                                                                                                      Why customers value Dataflow                           video   \n                                                                                                      Building Dataflow pipelines in code                    video   \n                                                                                                      Key considerations with designing pipelines            video   \n                                                                                                      Transforming data with PTransforms                     video   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline         video   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                  lab   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...       lab   \n                                                                                                      Aggregate with GroupByKey and Combine                  video   \n                                                                                                      Lab Intro: MapReduce in Beam                           video   \n                                                                                                      MapReduce in Beam (Python) 2.5                           lab   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...       lab   \n                                                                                                      Side inputs and windows of data                        video   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs             video   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...       lab   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...       lab   \n                                                                                                      Creating and re-using pipeline templates               video   \n                                                                                                      Summary                                                video   \n                                                                                                      Serverless Data Processing with Dataflow                quiz   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                    video   \n                                                                                                      Introduction to Cloud Data Fusion                      video   \n                                                                                                      Components of Cloud Data Fusion                        video   \n                                                                                                      Cloud Data Fusion UI                                   video   \n                                                                                                      Build a pipeline                                       video   \n                                                                                                      Explore data using wrangler                            video   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...     video   \n                                                                                                      Building and Executing a Pipeline Graph with Da...       lab   \n                                                                                                      Orchestrate work between Google Cloud services ...     video   \n                                                                                                      Apache Airflow environment                             video   \n                                                                                                      DAGs and Operators                                     video   \n                                                                                                      Workflow scheduling                                    video   \n                                                                                                      Monitoring and Logging                                 video   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer           video   \n                                                                                                      An Introduction to Cloud Composer 2.5                    lab   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...      quiz   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                         video   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud       document   \n\n                                                                                                                                                          record-disabled  \\\nTitles                                             Description                                        Activities                                                            \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                           False   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                           False   \n                                                                                                      EL, ELT, ETL                                                  False   \n                                                                                                      Quality considerations                                        False   \n                                                                                                      How to carry out operations in BigQuery                       False   \n                                                                                                      Shortcomings                                                  False   \n                                                                                                      ETL to solve data quality issues                              False   \n                                                                                                      Introduction to Building Batch Data Pipelines                 False   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                           False   \n                                                                                                      The Hadoop ecosystem                                          False   \n                                                                                                      Running Hadoop on Dataproc                                    False   \n                                                                                                      Cloud Storage instead of HDFS                                 False   \n                                                                                                      Optimizing Dataproc                                           False   \n                                                                                                      Optimizing Dataproc storage                                   False   \n                                                                                                      Optimizing Dataproc templates and autoscaling                 False   \n                                                                                                      Optimizing Dataproc monitoring                                False   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc              False   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc                   False   \n                                                                                                      Summary                                                       False   \n                                                                                                      Executing Spark on Dataproc                                   False   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                           False   \n                                                                                                      Introduction to Dataflow                                      False   \n                                                                                                      Why customers value Dataflow                                  False   \n                                                                                                      Building Dataflow pipelines in code                           False   \n                                                                                                      Key considerations with designing pipelines                   False   \n                                                                                                      Transforming data with PTransforms                            False   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline                False   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                       False   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...            False   \n                                                                                                      Aggregate with GroupByKey and Combine                         False   \n                                                                                                      Lab Intro: MapReduce in Beam                                  False   \n                                                                                                      MapReduce in Beam (Python) 2.5                                False   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...            False   \n                                                                                                      Side inputs and windows of data                               False   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                    False   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...            False   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...            False   \n                                                                                                      Creating and re-using pipeline templates                      False   \n                                                                                                      Summary                                                       False   \n                                                                                                      Serverless Data Processing with Dataflow                      False   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           False   \n                                                                                                      Introduction to Cloud Data Fusion                             False   \n                                                                                                      Components of Cloud Data Fusion                               False   \n                                                                                                      Cloud Data Fusion UI                                          False   \n                                                                                                      Build a pipeline                                              False   \n                                                                                                      Explore data using wrangler                                   False   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            False   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            False   \n                                                                                                      Orchestrate work between Google Cloud services ...            False   \n                                                                                                      Apache Airflow environment                                    False   \n                                                                                                      DAGs and Operators                                            False   \n                                                                                                      Workflow scheduling                                           False   \n                                                                                                      Monitoring and Logging                                        False   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  False   \n                                                                                                      An Introduction to Cloud Composer 2.5                         False   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            False   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                False   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud                 False   \n\n                                                                                                                                                                                                       text  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                               Course Introduction   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      EL, ELT, ETL                                        Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      Quality considerations                              Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      How to carry out operations in BigQuery             Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      Shortcomings                                        Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      ETL to solve data quality issues                    Module introduction. EL, ELT, ETL. Quality con...   \n                                                                                                      Introduction to Building Batch Data Pipelines       Module introduction. EL, ELT, ETL. Quality con...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      The Hadoop ecosystem                                Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Running Hadoop on Dataproc                          Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Cloud Storage instead of HDFS                       Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Optimizing Dataproc                                 Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Optimizing Dataproc storage                         Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Optimizing Dataproc monitoring                      Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Summary                                             Module introduction. The Hadoop ecosystem. Run...   \n                                                                                                      Executing Spark on Dataproc                         Module introduction. The Hadoop ecosystem. Run...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 Module introduction. Introduction to Dataflow....   \n                                                                                                      Introduction to Dataflow                            Module introduction. Introduction to Dataflow....   \n                                                                                                      Why customers value Dataflow                        Module introduction. Introduction to Dataflow....   \n                                                                                                      Building Dataflow pipelines in code                 Module introduction. Introduction to Dataflow....   \n                                                                                                      Key considerations with designing pipelines         Module introduction. Introduction to Dataflow....   \n                                                                                                      Transforming data with PTransforms                  Module introduction. Introduction to Dataflow....   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      Module introduction. Introduction to Dataflow....   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             Module introduction. Introduction to Dataflow....   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  Module introduction. Introduction to Dataflow....   \n                                                                                                      Aggregate with GroupByKey and Combine               Module introduction. Introduction to Dataflow....   \n                                                                                                      Lab Intro: MapReduce in Beam                        Module introduction. Introduction to Dataflow....   \n                                                                                                      MapReduce in Beam (Python) 2.5                      Module introduction. Introduction to Dataflow....   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  Module introduction. Introduction to Dataflow....   \n                                                                                                      Side inputs and windows of data                     Module introduction. Introduction to Dataflow....   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          Module introduction. Introduction to Dataflow....   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  Module introduction. Introduction to Dataflow....   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  Module introduction. Introduction to Dataflow....   \n                                                                                                      Creating and re-using pipeline templates            Module introduction. Introduction to Dataflow....   \n                                                                                                      Summary                                             Module introduction. Introduction to Dataflow....   \n                                                                                                      Serverless Data Processing with Dataflow            Module introduction. Introduction to Dataflow....   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                 Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Introduction to Cloud Data Fusion                   Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Components of Cloud Data Fusion                     Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Cloud Data Fusion UI                                Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Build a pipeline                                    Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Explore data using wrangler                         Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...  Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Building and Executing a Pipeline Graph with Da...  Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Orchestrate work between Google Cloud services ...  Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Apache Airflow environment                          Module introduction. Introduction to Cloud Dat...   \n                                                                                                      DAGs and Operators                                  Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Workflow scheduling                                 Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Monitoring and Logging                              Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer        Module introduction. Introduction to Cloud Dat...   \n                                                                                                      An Introduction to Cloud Composer 2.5               Module introduction. Introduction to Cloud Dat...   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...  Module introduction. Introduction to Cloud Dat...   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                         Course Summary   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud           Building Batch Data Pipelines on Google Cloud   \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 1)_None  \\\nTitles                                             Description                                        Activities                                                                                                \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]     \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...     \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...     \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...     \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...     \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...     \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]     \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]     \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]     \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]     \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]     \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]     \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]     \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 1)_english  \\\nTitles                                             Description                                        Activities                                                                                                   \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]        \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...        \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...        \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...        \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...        \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...        \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]        \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]        \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]        \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]        \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]        \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]        \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]        \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 2)_None  \\\nTitles                                             Description                                        Activities                                                                                                \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]     \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...     \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...     \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...     \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...     \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...     \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]     \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]     \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]     \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]     \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]     \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]     \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]     \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 2)_english  \\\nTitles                                             Description                                        Activities                                                                                                   \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]        \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...        \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...        \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...        \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...        \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...        \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]        \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]        \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]        \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]        \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]        \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]        \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]        \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 3)_None  \\\nTitles                                             Description                                        Activities                                                                                                \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]     \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...     \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...     \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...     \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...     \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...     \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...     \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]     \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]     \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]     \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]     \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]     \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]     \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]     \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]     \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]     \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]     \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]     \n\n                                                                                                                                                         keywords_distilbert-base-nli-mean-tokens_(1, 3)_english  \\\nTitles                                             Description                                        Activities                                                                                                   \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]        \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [pipelines, introduction, batch, building, con...        \n                                                                                                      EL, ELT, ETL                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      Quality considerations                              [pipelines, introduction, batch, building, con...        \n                                                                                                      How to carry out operations in BigQuery             [pipelines, introduction, batch, building, con...        \n                                                                                                      Shortcomings                                        [pipelines, introduction, batch, building, con...        \n                                                                                                      ETL to solve data quality issues                    [pipelines, introduction, batch, building, con...        \n                                                                                                      Introduction to Building Batch Data Pipelines       [pipelines, introduction, batch, building, con...        \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      The Hadoop ecosystem                                  [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Hadoop on Dataproc                            [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Cloud Storage instead of HDFS                         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc                                   [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc storage                           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc templates and autoscaling         [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Optimizing Dataproc monitoring                        [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc      [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc           [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Summary                                               [dataproc, autoscaling, apache, lab, templates]        \n                                                                                                      Executing Spark on Dataproc                           [dataproc, autoscaling, apache, lab, templates]        \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Introduction to Dataflow                                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Why customers value Dataflow                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Building Dataflow pipelines in code                         [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Key considerations with designing pipelines                 [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Transforming data with PTransforms                          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Aggregate with GroupByKey and Combine                       [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: MapReduce in Beam                                [python, java, dataflow, lab, groupbykey]        \n                                                                                                      MapReduce in Beam (Python) 2.5                              [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Side inputs and windows of data                             [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs                  [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...          [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Creating and re-using pipeline templates                    [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Summary                                                     [python, java, dataflow, lab, groupbykey]        \n                                                                                                      Serverless Data Processing with Dataflow                    [python, java, dataflow, lab, groupbykey]        \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Introduction to Cloud Data Fusion                           [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Components of Cloud Data Fusion                             [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Cloud Data Fusion UI                                        [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Build a pipeline                                            [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Explore data using wrangler                                 [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: Building and executing a pipeline gr...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Building and Executing a Pipeline Graph with Da...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Orchestrate work between Google Cloud services ...          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Apache Airflow environment                                  [google, lab, orchestrate, fusion, graph]        \n                                                                                                      DAGs and Operators                                          [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Workflow scheduling                                         [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Monitoring and Logging                                      [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                [google, lab, orchestrate, fusion, graph]        \n                                                                                                      An Introduction to Cloud Composer 2.5                       [google, lab, orchestrate, fusion, graph]        \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...          [google, lab, orchestrate, fusion, graph]        \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]        \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, pipelines, building, cloud, data]        \n\n                                                                                                                                                          ...  \\\nTitles                                             Description                                        Activities                                          ...   \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                 ...   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 ...   \n                                                                                                      EL, ELT, ETL                                        ...   \n                                                                                                      Quality considerations                              ...   \n                                                                                                      How to carry out operations in BigQuery             ...   \n                                                                                                      Shortcomings                                        ...   \n                                                                                                      ETL to solve data quality issues                    ...   \n                                                                                                      Introduction to Building Batch Data Pipelines       ...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 ...   \n                                                                                                      The Hadoop ecosystem                                ...   \n                                                                                                      Running Hadoop on Dataproc                          ...   \n                                                                                                      Cloud Storage instead of HDFS                       ...   \n                                                                                                      Optimizing Dataproc                                 ...   \n                                                                                                      Optimizing Dataproc storage                         ...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       ...   \n                                                                                                      Optimizing Dataproc monitoring                      ...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    ...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         ...   \n                                                                                                      Summary                                             ...   \n                                                                                                      Executing Spark on Dataproc                         ...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 ...   \n                                                                                                      Introduction to Dataflow                            ...   \n                                                                                                      Why customers value Dataflow                        ...   \n                                                                                                      Building Dataflow pipelines in code                 ...   \n                                                                                                      Key considerations with designing pipelines         ...   \n                                                                                                      Transforming data with PTransforms                  ...   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      ...   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             ...   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  ...   \n                                                                                                      Aggregate with GroupByKey and Combine               ...   \n                                                                                                      Lab Intro: MapReduce in Beam                        ...   \n                                                                                                      MapReduce in Beam (Python) 2.5                      ...   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  ...   \n                                                                                                      Side inputs and windows of data                     ...   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          ...   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  ...   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  ...   \n                                                                                                      Creating and re-using pipeline templates            ...   \n                                                                                                      Summary                                             ...   \n                                                                                                      Serverless Data Processing with Dataflow            ...   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                 ...   \n                                                                                                      Introduction to Cloud Data Fusion                   ...   \n                                                                                                      Components of Cloud Data Fusion                     ...   \n                                                                                                      Cloud Data Fusion UI                                ...   \n                                                                                                      Build a pipeline                                    ...   \n                                                                                                      Explore data using wrangler                         ...   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...  ...   \n                                                                                                      Building and Executing a Pipeline Graph with Da...  ...   \n                                                                                                      Orchestrate work between Google Cloud services ...  ...   \n                                                                                                      Apache Airflow environment                          ...   \n                                                                                                      DAGs and Operators                                  ...   \n                                                                                                      Workflow scheduling                                 ...   \n                                                                                                      Monitoring and Logging                              ...   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer        ...   \n                                                                                                      An Introduction to Cloud Composer 2.5               ...   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...  ...   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                      ...   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud       ...   \n\n                                                                                                                                                         keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  \\\nTitles                                             Description                                        Activities                                                                                                     \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [course, introduction]          \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                         [introduction, etl, module, quality, elt]          \n                                                                                                      EL, ELT, ETL                                                [introduction, etl, module, quality, elt]          \n                                                                                                      Quality considerations                                      [introduction, etl, module, quality, elt]          \n                                                                                                      How to carry out operations in BigQuery                     [introduction, etl, module, quality, elt]          \n                                                                                                      Shortcomings                                                [introduction, etl, module, quality, elt]          \n                                                                                                      ETL to solve data quality issues                            [introduction, etl, module, quality, elt]          \n                                                                                                      Introduction to Building Batch Data Pipelines               [introduction, etl, module, quality, elt]          \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [apache, spark, introduction, executing, optim...          \n                                                                                                      The Hadoop ecosystem                                [apache, spark, introduction, executing, optim...          \n                                                                                                      Running Hadoop on Dataproc                          [apache, spark, introduction, executing, optim...          \n                                                                                                      Cloud Storage instead of HDFS                       [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc                                 [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc storage                         [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc templates and autoscaling       [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc monitoring                      [apache, spark, introduction, executing, optim...          \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [apache, spark, introduction, executing, optim...          \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [apache, spark, introduction, executing, optim...          \n                                                                                                      Summary                                             [apache, spark, introduction, executing, optim...          \n                                                                                                      Executing Spark on Dataproc                         [apache, spark, introduction, executing, optim...          \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Introduction to Dataflow                            [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Why customers value Dataflow                        [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Building Dataflow pipelines in code                 [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Key considerations with designing pipelines         [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Transforming data with PTransforms                  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      [dataflow, ptransforms, creating, transforming...          \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Aggregate with GroupByKey and Combine               [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: MapReduce in Beam                        [dataflow, ptransforms, creating, transforming...          \n                                                                                                      MapReduce in Beam (Python) 2.5                      [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Side inputs and windows of data                     [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Creating and re-using pipeline templates            [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Summary                                             [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Processing with Dataflow            [dataflow, ptransforms, creating, transforming...          \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                 [introduction, module, executing, cloud, sched...          \n                                                                                                      Introduction to Cloud Data Fusion                   [introduction, module, executing, cloud, sched...          \n                                                                                                      Components of Cloud Data Fusion                     [introduction, module, executing, cloud, sched...          \n                                                                                                      Cloud Data Fusion UI                                [introduction, module, executing, cloud, sched...          \n                                                                                                      Build a pipeline                                    [introduction, module, executing, cloud, sched...          \n                                                                                                      Explore data using wrangler                         [introduction, module, executing, cloud, sched...          \n                                                                                                      Lab Intro: Building and executing a pipeline gr...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Building and Executing a Pipeline Graph with Da...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Orchestrate work between Google Cloud services ...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Apache Airflow environment                          [introduction, module, executing, cloud, sched...          \n                                                                                                      DAGs and Operators                                  [introduction, module, executing, cloud, sched...          \n                                                                                                      Workflow scheduling                                 [introduction, module, executing, cloud, sched...          \n                                                                                                      Monitoring and Logging                              [introduction, module, executing, cloud, sched...          \n                                                                                                      Lab Intro: An Introduction to Cloud Composer        [introduction, module, executing, cloud, sched...          \n                                                                                                      An Introduction to Cloud Composer 2.5               [introduction, module, executing, cloud, sched...          \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...  [introduction, module, executing, cloud, sched...          \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [course, summary]          \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [cloud, google, data, pipelines, building]          \n\n                                                                                                                                                         keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None  \\\nTitles                                             Description                                        Activities                                                                                                  \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [course, introduction]       \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                         [introduction, etl, module, quality, elt]       \n                                                                                                      EL, ELT, ETL                                                [introduction, etl, module, quality, elt]       \n                                                                                                      Quality considerations                                      [introduction, etl, module, quality, elt]       \n                                                                                                      How to carry out operations in BigQuery                     [introduction, etl, module, quality, elt]       \n                                                                                                      Shortcomings                                                [introduction, etl, module, quality, elt]       \n                                                                                                      ETL to solve data quality issues                            [introduction, etl, module, quality, elt]       \n                                                                                                      Introduction to Building Batch Data Pipelines               [introduction, etl, module, quality, elt]       \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [apache, spark, introduction, executing, optim...       \n                                                                                                      The Hadoop ecosystem                                [apache, spark, introduction, executing, optim...       \n                                                                                                      Running Hadoop on Dataproc                          [apache, spark, introduction, executing, optim...       \n                                                                                                      Cloud Storage instead of HDFS                       [apache, spark, introduction, executing, optim...       \n                                                                                                      Optimizing Dataproc                                 [apache, spark, introduction, executing, optim...       \n                                                                                                      Optimizing Dataproc storage                         [apache, spark, introduction, executing, optim...       \n                                                                                                      Optimizing Dataproc templates and autoscaling       [apache, spark, introduction, executing, optim...       \n                                                                                                      Optimizing Dataproc monitoring                      [apache, spark, introduction, executing, optim...       \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [apache, spark, introduction, executing, optim...       \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [apache, spark, introduction, executing, optim...       \n                                                                                                      Summary                                             [apache, spark, introduction, executing, optim...       \n                                                                                                      Executing Spark on Dataproc                         [apache, spark, introduction, executing, optim...       \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Introduction to Dataflow                            [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Why customers value Dataflow                        [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Building Dataflow pipelines in code                 [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Key considerations with designing pipelines         [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Transforming data with PTransforms                  [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      [dataflow, ptransforms, creating, transforming...       \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Aggregate with GroupByKey and Combine               [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Lab Intro: MapReduce in Beam                        [dataflow, ptransforms, creating, transforming...       \n                                                                                                      MapReduce in Beam (Python) 2.5                      [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Side inputs and windows of data                     [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Creating and re-using pipeline templates            [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Summary                                             [dataflow, ptransforms, creating, transforming...       \n                                                                                                      Serverless Data Processing with Dataflow            [dataflow, ptransforms, creating, transforming...       \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                 [introduction, module, executing, cloud, sched...       \n                                                                                                      Introduction to Cloud Data Fusion                   [introduction, module, executing, cloud, sched...       \n                                                                                                      Components of Cloud Data Fusion                     [introduction, module, executing, cloud, sched...       \n                                                                                                      Cloud Data Fusion UI                                [introduction, module, executing, cloud, sched...       \n                                                                                                      Build a pipeline                                    [introduction, module, executing, cloud, sched...       \n                                                                                                      Explore data using wrangler                         [introduction, module, executing, cloud, sched...       \n                                                                                                      Lab Intro: Building and executing a pipeline gr...  [introduction, module, executing, cloud, sched...       \n                                                                                                      Building and Executing a Pipeline Graph with Da...  [introduction, module, executing, cloud, sched...       \n                                                                                                      Orchestrate work between Google Cloud services ...  [introduction, module, executing, cloud, sched...       \n                                                                                                      Apache Airflow environment                          [introduction, module, executing, cloud, sched...       \n                                                                                                      DAGs and Operators                                  [introduction, module, executing, cloud, sched...       \n                                                                                                      Workflow scheduling                                 [introduction, module, executing, cloud, sched...       \n                                                                                                      Monitoring and Logging                              [introduction, module, executing, cloud, sched...       \n                                                                                                      Lab Intro: An Introduction to Cloud Composer        [introduction, module, executing, cloud, sched...       \n                                                                                                      An Introduction to Cloud Composer 2.5               [introduction, module, executing, cloud, sched...       \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...  [introduction, module, executing, cloud, sched...       \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [course, summary]       \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [cloud, google, data, pipelines, building]       \n\n                                                                                                                                                         keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  \\\nTitles                                             Description                                        Activities                                                                                                     \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [course, introduction]          \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                         [introduction, etl, module, quality, elt]          \n                                                                                                      EL, ELT, ETL                                                [introduction, etl, module, quality, elt]          \n                                                                                                      Quality considerations                                      [introduction, etl, module, quality, elt]          \n                                                                                                      How to carry out operations in BigQuery                     [introduction, etl, module, quality, elt]          \n                                                                                                      Shortcomings                                                [introduction, etl, module, quality, elt]          \n                                                                                                      ETL to solve data quality issues                            [introduction, etl, module, quality, elt]          \n                                                                                                      Introduction to Building Batch Data Pipelines               [introduction, etl, module, quality, elt]          \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [apache, spark, introduction, executing, optim...          \n                                                                                                      The Hadoop ecosystem                                [apache, spark, introduction, executing, optim...          \n                                                                                                      Running Hadoop on Dataproc                          [apache, spark, introduction, executing, optim...          \n                                                                                                      Cloud Storage instead of HDFS                       [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc                                 [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc storage                         [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc templates and autoscaling       [apache, spark, introduction, executing, optim...          \n                                                                                                      Optimizing Dataproc monitoring                      [apache, spark, introduction, executing, optim...          \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [apache, spark, introduction, executing, optim...          \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [apache, spark, introduction, executing, optim...          \n                                                                                                      Summary                                             [apache, spark, introduction, executing, optim...          \n                                                                                                      Executing Spark on Dataproc                         [apache, spark, introduction, executing, optim...          \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Introduction to Dataflow                            [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Why customers value Dataflow                        [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Building Dataflow pipelines in code                 [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Key considerations with designing pipelines         [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Transforming data with PTransforms                  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      [dataflow, ptransforms, creating, transforming...          \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Aggregate with GroupByKey and Combine               [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: MapReduce in Beam                        [dataflow, ptransforms, creating, transforming...          \n                                                                                                      MapReduce in Beam (Python) 2.5                      [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Side inputs and windows of data                     [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Creating and re-using pipeline templates            [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Summary                                             [dataflow, ptransforms, creating, transforming...          \n                                                                                                      Serverless Data Processing with Dataflow            [dataflow, ptransforms, creating, transforming...          \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                 [introduction, module, executing, cloud, sched...          \n                                                                                                      Introduction to Cloud Data Fusion                   [introduction, module, executing, cloud, sched...          \n                                                                                                      Components of Cloud Data Fusion                     [introduction, module, executing, cloud, sched...          \n                                                                                                      Cloud Data Fusion UI                                [introduction, module, executing, cloud, sched...          \n                                                                                                      Build a pipeline                                    [introduction, module, executing, cloud, sched...          \n                                                                                                      Explore data using wrangler                         [introduction, module, executing, cloud, sched...          \n                                                                                                      Lab Intro: Building and executing a pipeline gr...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Building and Executing a Pipeline Graph with Da...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Orchestrate work between Google Cloud services ...  [introduction, module, executing, cloud, sched...          \n                                                                                                      Apache Airflow environment                          [introduction, module, executing, cloud, sched...          \n                                                                                                      DAGs and Operators                                  [introduction, module, executing, cloud, sched...          \n                                                                                                      Workflow scheduling                                 [introduction, module, executing, cloud, sched...          \n                                                                                                      Monitoring and Logging                              [introduction, module, executing, cloud, sched...          \n                                                                                                      Lab Intro: An Introduction to Cloud Composer        [introduction, module, executing, cloud, sched...          \n                                                                                                      An Introduction to Cloud Composer 2.5               [introduction, module, executing, cloud, sched...          \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...  [introduction, module, executing, cloud, sched...          \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [course, summary]          \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [cloud, google, data, pipelines, building]          \n\n                                                                                                                                                             keywords_bert-base-nli-mean-tokens_(1, 1)_None  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                          keywords_bert-base-nli-mean-tokens_(1, 1)_english  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                             keywords_bert-base-nli-mean-tokens_(1, 2)_None  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                          keywords_bert-base-nli-mean-tokens_(1, 2)_english  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                             keywords_bert-base-nli-mean-tokens_(1, 3)_None  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                          keywords_bert-base-nli-mean-tokens_(1, 3)_english  \\\nTitles                                             Description                                        Activities                                                                                              \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                            [introduction, course]   \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [introduction, data, pipelines, building, bigq...   \n                                                                                                      EL, ELT, ETL                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      Quality considerations                              [introduction, data, pipelines, building, bigq...   \n                                                                                                      How to carry out operations in BigQuery             [introduction, data, pipelines, building, bigq...   \n                                                                                                      Shortcomings                                        [introduction, data, pipelines, building, bigq...   \n                                                                                                      ETL to solve data quality issues                    [introduction, data, pipelines, building, bigq...   \n                                                                                                      Introduction to Building Batch Data Pipelines       [introduction, data, pipelines, building, bigq...   \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      The Hadoop ecosystem                                [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Hadoop on Dataproc                          [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Cloud Storage instead of HDFS                       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc                                 [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc storage                         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc templates and autoscaling       [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Optimizing Dataproc monitoring                      [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Summary                                             [dataproc, templates, autoscaling, lab, optimi...   \n                                                                                                      Executing Spark on Dataproc                         [dataproc, templates, autoscaling, lab, optimi...   \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Introduction to Dataflow                                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Why customers value Dataflow                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Building Dataflow pipelines in code                     [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Key considerations with designing pipelines             [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Transforming data with PTransforms                      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Aggregate with GroupByKey and Combine                   [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: MapReduce in Beam                            [java, templates, dataflow, code, groupbykey]   \n                                                                                                      MapReduce in Beam (Python) 2.5                          [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Side inputs and windows of data                         [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs              [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...      [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Creating and re-using pipeline templates                [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Summary                                                 [java, templates, dataflow, code, groupbykey]   \n                                                                                                      Serverless Data Processing with Dataflow                [java, templates, dataflow, code, groupbykey]   \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Introduction to Cloud Data Fusion                             [google, workflow, logging, lab, graph]   \n                                                                                                      Components of Cloud Data Fusion                               [google, workflow, logging, lab, graph]   \n                                                                                                      Cloud Data Fusion UI                                          [google, workflow, logging, lab, graph]   \n                                                                                                      Build a pipeline                                              [google, workflow, logging, lab, graph]   \n                                                                                                      Explore data using wrangler                                   [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: Building and executing a pipeline gr...            [google, workflow, logging, lab, graph]   \n                                                                                                      Building and Executing a Pipeline Graph with Da...            [google, workflow, logging, lab, graph]   \n                                                                                                      Orchestrate work between Google Cloud services ...            [google, workflow, logging, lab, graph]   \n                                                                                                      Apache Airflow environment                                    [google, workflow, logging, lab, graph]   \n                                                                                                      DAGs and Operators                                            [google, workflow, logging, lab, graph]   \n                                                                                                      Workflow scheduling                                           [google, workflow, logging, lab, graph]   \n                                                                                                      Monitoring and Logging                                        [google, workflow, logging, lab, graph]   \n                                                                                                      Lab Intro: An Introduction to Cloud Composer                  [google, workflow, logging, lab, graph]   \n                                                                                                      An Introduction to Cloud Composer 2.5                         [google, workflow, logging, lab, graph]   \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...            [google, workflow, logging, lab, graph]   \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                      [summary, course]   \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud              [google, data, pipelines, batch, building]   \n\n                                                                                                                                                                                          wordwise_keywords  \nTitles                                             Description                                        Activities                                                                                             \nIntroduction                                       <p>In this module, we introduce the course and ... Course Introduction                                                                                []  \nIntroduction to Building Batch Data Pipelines      <p>This module reviews different methods of dat... Module introduction                                 [quality, data, introduction, considerations, ...  \n                                                                                                      EL, ELT, ETL                                        [quality, data, introduction, considerations, ...  \n                                                                                                      Quality considerations                              [quality, data, introduction, considerations, ...  \n                                                                                                      How to carry out operations in BigQuery             [quality, data, introduction, considerations, ...  \n                                                                                                      Shortcomings                                        [quality, data, introduction, considerations, ...  \n                                                                                                      ETL to solve data quality issues                    [quality, data, introduction, considerations, ...  \n                                                                                                      Introduction to Building Batch Data Pipelines       [quality, data, introduction, considerations, ...  \nExecuting Spark on Dataproc                        <p>This module shows how to run Hadoop on Datap... Module introduction                                 [introduction, templates, storage, jobs, monit...  \n                                                                                                      The Hadoop ecosystem                                [introduction, templates, storage, jobs, monit...  \n                                                                                                      Running Hadoop on Dataproc                          [introduction, templates, storage, jobs, monit...  \n                                                                                                      Cloud Storage instead of HDFS                       [introduction, templates, storage, jobs, monit...  \n                                                                                                      Optimizing Dataproc                                 [introduction, templates, storage, jobs, monit...  \n                                                                                                      Optimizing Dataproc storage                         [introduction, templates, storage, jobs, monit...  \n                                                                                                      Optimizing Dataproc templates and autoscaling       [introduction, templates, storage, jobs, monit...  \n                                                                                                      Optimizing Dataproc monitoring                      [introduction, templates, storage, jobs, monit...  \n                                                                                                      Lab Intro: Running Apache Spark jobs on Dataproc    [introduction, templates, storage, jobs, monit...  \n                                                                                                      Running Apache Spark jobs on Cloud Dataproc         [introduction, templates, storage, jobs, monit...  \n                                                                                                      Summary                                             [introduction, templates, storage, jobs, monit...  \n                                                                                                      Executing Spark on Dataproc                         [introduction, templates, storage, jobs, monit...  \nServerless Data Processing with Dataflow           <p>This module covers using Dataflow to build y... Module introduction                                 [pipelines, data, introduction, templates, pip...  \n                                                                                                      Introduction to Dataflow                            [pipelines, data, introduction, templates, pip...  \n                                                                                                      Why customers value Dataflow                        [pipelines, data, introduction, templates, pip...  \n                                                                                                      Building Dataflow pipelines in code                 [pipelines, data, introduction, templates, pip...  \n                                                                                                      Key considerations with designing pipelines         [pipelines, data, introduction, templates, pip...  \n                                                                                                      Transforming data with PTransforms                  [pipelines, data, introduction, templates, pip...  \n                                                                                                      Lab Intro: Building a Simple Dataflow Pipeline      [pipelines, data, introduction, templates, pip...  \n                                                                                                      A Simple Dataflow Pipeline (Python) 2.5             [pipelines, data, introduction, templates, pip...  \n                                                                                                      Serverless Data Analysis with Dataflow: A Simpl...  [pipelines, data, introduction, templates, pip...  \n                                                                                                      Aggregate with GroupByKey and Combine               [pipelines, data, introduction, templates, pip...  \n                                                                                                      Lab Intro: MapReduce in Beam                        [pipelines, data, introduction, templates, pip...  \n                                                                                                      MapReduce in Beam (Python) 2.5                      [pipelines, data, introduction, templates, pip...  \n                                                                                                      Serverless Data Analysis with Beam: MapReduce i...  [pipelines, data, introduction, templates, pip...  \n                                                                                                      Side inputs and windows of data                     [pipelines, data, introduction, templates, pip...  \n                                                                                                      Lab Intro: Practicing Pipeline Side Inputs          [pipelines, data, introduction, templates, pip...  \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [pipelines, data, introduction, templates, pip...  \n                                                                                                      Serverless Data Analysis with Dataflow: Side In...  [pipelines, data, introduction, templates, pip...  \n                                                                                                      Creating and re-using pipeline templates            [pipelines, data, introduction, templates, pip...  \n                                                                                                      Summary                                             [pipelines, data, introduction, templates, pip...  \n                                                                                                      Serverless Data Processing with Dataflow            [pipelines, data, introduction, templates, pip...  \nManage Data Pipelines with Cloud Data Fusion an... <p>This module shows how to manage data pipelin... Module introduction                                    [introduction, data, services, wrangler, work]  \n                                                                                                      Introduction to Cloud Data Fusion                      [introduction, data, services, wrangler, work]  \n                                                                                                      Components of Cloud Data Fusion                        [introduction, data, services, wrangler, work]  \n                                                                                                      Cloud Data Fusion UI                                   [introduction, data, services, wrangler, work]  \n                                                                                                      Build a pipeline                                       [introduction, data, services, wrangler, work]  \n                                                                                                      Explore data using wrangler                            [introduction, data, services, wrangler, work]  \n                                                                                                      Lab Intro: Building and executing a pipeline gr...     [introduction, data, services, wrangler, work]  \n                                                                                                      Building and Executing a Pipeline Graph with Da...     [introduction, data, services, wrangler, work]  \n                                                                                                      Orchestrate work between Google Cloud services ...     [introduction, data, services, wrangler, work]  \n                                                                                                      Apache Airflow environment                             [introduction, data, services, wrangler, work]  \n                                                                                                      DAGs and Operators                                     [introduction, data, services, wrangler, work]  \n                                                                                                      Workflow scheduling                                    [introduction, data, services, wrangler, work]  \n                                                                                                      Monitoring and Logging                                 [introduction, data, services, wrangler, work]  \n                                                                                                      Lab Intro: An Introduction to Cloud Composer           [introduction, data, services, wrangler, work]  \n                                                                                                      An Introduction to Cloud Composer 2.5                  [introduction, data, services, wrangler, work]  \n                                                                                                      Manage Data Pipelines with Cloud Data Fusion an...     [introduction, data, services, wrangler, work]  \nCourse Summary                                     <p>Course Summary</p>                              Course Summary                                                                                     []  \nCourse Resources                                   <p>PDF links to all modules</p>                    Building Batch Data Pipelines on Google Cloud                                                      []  \n\n[58 rows x 23 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th></th>\n      <th></th>\n      <th>Duration</th>\n      <th>Type</th>\n      <th>record-disabled</th>\n      <th>text</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 3)_english</th>\n      <th>...</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_english</th>\n      <th>wordwise_keywords</th>\n    </tr>\n    <tr>\n      <th>Titles</th>\n      <th>Description</th>\n      <th>Activities</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>Introduction</th>\n      <th>&lt;p&gt;In this module, we introduce the course and agenda&lt;/p&gt;</th>\n      <th>Course Introduction</th>\n      <td>55000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Course Introduction</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>...</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[]</td>\n    </tr>\n    <tr>\n      <th rowspan=\"7\" valign=\"top\">Introduction to Building Batch Data Pipelines</th>\n      <th rowspan=\"7\" valign=\"top\">&lt;p&gt;This module reviews different methods of data loading: EL, ELT and ETL and when to use what&lt;/p&gt;</th>\n      <th>Module introduction</th>\n      <td>69000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>EL, ELT, ETL</th>\n      <td>220000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>Quality considerations</th>\n      <td>168000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>How to carry out operations in BigQuery</th>\n      <td>180000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>Shortcomings</th>\n      <td>208000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>ETL to solve data quality issues</th>\n      <td>428000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>Introduction to Building Batch Data Pipelines</th>\n      <td>0</td>\n      <td>quiz</td>\n      <td>False</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th rowspan=\"12\" valign=\"top\">Executing Spark on Dataproc</th>\n      <th rowspan=\"12\" valign=\"top\">&lt;p&gt;This module shows how to run Hadoop on Dataproc, how to leverage Cloud Storage, and how to optimize your Dataproc jobs.&lt;/p&gt;</th>\n      <th>Module introduction</th>\n      <td>27000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>The Hadoop ecosystem</th>\n      <td>286000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Running Hadoop on Dataproc</th>\n      <td>602000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Cloud Storage instead of HDFS</th>\n      <td>379000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Optimizing Dataproc</th>\n      <td>172000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Optimizing Dataproc storage</th>\n      <td>567000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Optimizing Dataproc templates and autoscaling</th>\n      <td>312000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Optimizing Dataproc monitoring</th>\n      <td>186000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: Running Apache Spark jobs on Dataproc</th>\n      <td>28000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Running Apache Spark jobs on Cloud Dataproc</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Summary</th>\n      <td>34000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th>Executing Spark on Dataproc</th>\n      <td>0</td>\n      <td>quiz</td>\n      <td>False</td>\n      <td>Module introduction. The Hadoop ecosystem. Run...</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>[dataproc, autoscaling, apache, lab, templates]</td>\n      <td>...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[apache, spark, introduction, executing, optim...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[dataproc, templates, autoscaling, lab, optimi...</td>\n      <td>[introduction, templates, storage, jobs, monit...</td>\n    </tr>\n    <tr>\n      <th rowspan=\"20\" valign=\"top\">Serverless Data Processing with Dataflow</th>\n      <th rowspan=\"20\" valign=\"top\">&lt;p&gt;This module covers using Dataflow to build your data processing pipelines&lt;/p&gt;</th>\n      <th>Module introduction</th>\n      <td>53000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Introduction to Dataflow</th>\n      <td>342000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Why customers value Dataflow</th>\n      <td>171000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Building Dataflow pipelines in code</th>\n      <td>228000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Key considerations with designing pipelines</th>\n      <td>127000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Transforming data with PTransforms</th>\n      <td>188000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: Building a Simple Dataflow Pipeline</th>\n      <td>10000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>A Simple Dataflow Pipeline (Python) 2.5</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Serverless Data Analysis with Dataflow: A Simple Dataflow Pipeline (Java)</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Aggregate with GroupByKey and Combine</th>\n      <td>335000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: MapReduce in Beam</th>\n      <td>21000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>MapReduce in Beam (Python) 2.5</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Serverless Data Analysis with Beam: MapReduce in Beam (Java)</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Side inputs and windows of data</th>\n      <td>283000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: Practicing Pipeline Side Inputs</th>\n      <td>18000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Serverless Data Analysis with Dataflow: Side Inputs (Python)</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Serverless Data Analysis with Dataflow: Side Inputs (Java)</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Creating and re-using pipeline templates</th>\n      <td>210000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Summary</th>\n      <td>126000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th>Serverless Data Processing with Dataflow</th>\n      <td>0</td>\n      <td>quiz</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Dataflow....</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>[python, java, dataflow, lab, groupbykey]</td>\n      <td>...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[dataflow, ptransforms, creating, transforming...</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[java, templates, dataflow, code, groupbykey]</td>\n      <td>[pipelines, data, introduction, templates, pip...</td>\n    </tr>\n    <tr>\n      <th rowspan=\"16\" valign=\"top\">Manage Data Pipelines with Cloud Data Fusion and Cloud Composer</th>\n      <th rowspan=\"16\" valign=\"top\">&lt;p&gt;This module shows how to manage data pipelines with Cloud Data Fusion and Cloud Composer.&lt;/p&gt;</th>\n      <th>Module introduction</th>\n      <td>19000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Introduction to Cloud Data Fusion</th>\n      <td>219000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Components of Cloud Data Fusion</th>\n      <td>74000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Cloud Data Fusion UI</th>\n      <td>95000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Build a pipeline</th>\n      <td>291000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Explore data using wrangler</th>\n      <td>108000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: Building and executing a pipeline graph in Cloud Data Fusion</th>\n      <td>18000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Building and Executing a Pipeline Graph with Data Fusion 2.5</th>\n      <td>9000000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Orchestrate work between Google Cloud services with Cloud Composer</th>\n      <td>64000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Apache Airflow environment</th>\n      <td>74000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>DAGs and Operators</th>\n      <td>470000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Workflow scheduling</th>\n      <td>335000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Monitoring and Logging</th>\n      <td>208000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Lab Intro: An Introduction to Cloud Composer</th>\n      <td>22000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>An Introduction to Cloud Composer 2.5</th>\n      <td>5400000</td>\n      <td>lab</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Manage Data Pipelines with Cloud Data Fusion and Cloud Composer</th>\n      <td>0</td>\n      <td>quiz</td>\n      <td>False</td>\n      <td>Module introduction. Introduction to Cloud Dat...</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>[google, lab, orchestrate, fusion, graph]</td>\n      <td>...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[introduction, module, executing, cloud, sched...</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[google, workflow, logging, lab, graph]</td>\n      <td>[introduction, data, services, wrangler, work]</td>\n    </tr>\n    <tr>\n      <th>Course Summary</th>\n      <th>&lt;p&gt;Course Summary&lt;/p&gt;</th>\n      <th>Course Summary</th>\n      <td>197000</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Course Summary</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>...</td>\n      <td>[course, summary]</td>\n      <td>[course, summary]</td>\n      <td>[course, summary]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[summary, course]</td>\n      <td>[]</td>\n    </tr>\n    <tr>\n      <th>Course Resources</th>\n      <th>&lt;p&gt;PDF links to all modules&lt;/p&gt;</th>\n      <th>Building Batch Data Pipelines on Google Cloud</th>\n      <td>0</td>\n      <td>document</td>\n      <td>False</td>\n      <td>Building Batch Data Pipelines on Google Cloud</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>[google, pipelines, building, cloud, data]</td>\n      <td>...</td>\n      <td>[cloud, google, data, pipelines, building]</td>\n      <td>[cloud, google, data, pipelines, building]</td>\n      <td>[cloud, google, data, pipelines, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[google, data, pipelines, batch, building]</td>\n      <td>[]</td>\n    </tr>\n  </tbody>\n</table>\n<p>58 rows × 23 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"code","source":"flatten_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-06T10:11:09.243119Z","iopub.execute_input":"2023-10-06T10:11:09.243543Z","iopub.status.idle":"2023-10-06T10:11:09.285705Z","shell.execute_reply.started":"2023-10-06T10:11:09.243513Z","shell.execute_reply":"2023-10-06T10:11:09.28441Z"},"trusted":true},"execution_count":39,"outputs":[{"execution_count":39,"output_type":"execute_result","data":{"text/plain":"   record-duration                             record-title record-type  \\\n0            55000                      Course Introduction       video   \n1            69000                      Module introduction       video   \n2           220000                             EL, ELT, ETL       video   \n3           168000                   Quality considerations       video   \n4           180000  How to carry out operations in BigQuery       video   \n\n   record-disabled                                     meta-title  \\\n0            False                                   Introduction   \n1            False  Introduction to Building Batch Data Pipelines   \n2            False  Introduction to Building Batch Data Pipelines   \n3            False  Introduction to Building Batch Data Pipelines   \n4            False  Introduction to Building Batch Data Pipelines   \n\n                                    meta-description  \\\n0  <p>In this module, we introduce the course and...   \n1  <p>This module reviews different methods of da...   \n2  <p>This module reviews different methods of da...   \n3  <p>This module reviews different methods of da...   \n4  <p>This module reviews different methods of da...   \n\n                                                text  \\\n0                                Course Introduction   \n1  Module introduction. EL, ELT, ETL. Quality con...   \n2  Module introduction. EL, ELT, ETL. Quality con...   \n3  Module introduction. EL, ELT, ETL. Quality con...   \n4  Module introduction. EL, ELT, ETL. Quality con...   \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]     \n1  [pipelines, introduction, batch, building, con...     \n2  [pipelines, introduction, batch, building, con...     \n3  [pipelines, introduction, batch, building, con...     \n4  [pipelines, introduction, batch, building, con...     \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]        \n1  [pipelines, introduction, batch, building, con...        \n2  [pipelines, introduction, batch, building, con...        \n3  [pipelines, introduction, batch, building, con...        \n4  [pipelines, introduction, batch, building, con...        \n\n  keywords_distilbert-base-nli-mean-tokens_(1, 2)_None  ...  \\\n0                             [introduction, course]    ...   \n1  [pipelines, introduction, batch, building, con...    ...   \n2  [pipelines, introduction, batch, building, con...    ...   \n3  [pipelines, introduction, batch, building, con...    ...   \n4  [pipelines, introduction, batch, building, con...    ...   \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None  \\\n0                             [course, introduction]       \n1          [introduction, etl, module, quality, elt]       \n2          [introduction, etl, module, quality, elt]       \n3          [introduction, etl, module, quality, elt]       \n4          [introduction, etl, module, quality, elt]       \n\n  keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english  \\\n0                             [course, introduction]          \n1          [introduction, etl, module, quality, elt]          \n2          [introduction, etl, module, quality, elt]          \n3          [introduction, etl, module, quality, elt]          \n4          [introduction, etl, module, quality, elt]          \n\n      keywords_bert-base-nli-mean-tokens_(1, 1)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 1)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 2)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 2)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n      keywords_bert-base-nli-mean-tokens_(1, 3)_None  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n   keywords_bert-base-nli-mean-tokens_(1, 3)_english  \\\n0                             [introduction, course]   \n1  [introduction, data, pipelines, building, bigq...   \n2  [introduction, data, pipelines, building, bigq...   \n3  [introduction, data, pipelines, building, bigq...   \n4  [introduction, data, pipelines, building, bigq...   \n\n                                   wordwise_keywords  \n0                                                 []  \n1  [quality, data, introduction, considerations, ...  \n2  [quality, data, introduction, considerations, ...  \n3  [quality, data, introduction, considerations, ...  \n4  [quality, data, introduction, considerations, ...  \n\n[5 rows x 26 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>record-duration</th>\n      <th>record-title</th>\n      <th>record-type</th>\n      <th>record-disabled</th>\n      <th>meta-title</th>\n      <th>meta-description</th>\n      <th>text</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_distilbert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>...</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 2)_english</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_None</th>\n      <th>keywords_roberta-base-nli-stsb-mean-tokens_(1, 3)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 1)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 2)_english</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_None</th>\n      <th>keywords_bert-base-nli-mean-tokens_(1, 3)_english</th>\n      <th>wordwise_keywords</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>55000</td>\n      <td>Course Introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction</td>\n      <td>&lt;p&gt;In this module, we introduce the course and...</td>\n      <td>Course Introduction</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>...</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[course, introduction]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[introduction, course]</td>\n      <td>[]</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>69000</td>\n      <td>Module introduction</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>220000</td>\n      <td>EL, ELT, ETL</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>168000</td>\n      <td>Quality considerations</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>180000</td>\n      <td>How to carry out operations in BigQuery</td>\n      <td>video</td>\n      <td>False</td>\n      <td>Introduction to Building Batch Data Pipelines</td>\n      <td>&lt;p&gt;This module reviews different methods of da...</td>\n      <td>Module introduction. EL, ELT, ETL. Quality con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>[pipelines, introduction, batch, building, con...</td>\n      <td>...</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, etl, module, quality, elt]</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[introduction, data, pipelines, building, bigq...</td>\n      <td>[quality, data, introduction, considerations, ...</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 26 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"## 6) Exporting Keyword Extracted Output as Excel File\n---","metadata":{"id":"z06KSACU542g"}},{"cell_type":"code","source":"with pd.ExcelWriter('Keyword_Extraction_Comparison.xlsx', engine='xlsxwriter') as writer:\n    index_df.to_excel(writer, sheet_name=\"Keyword_Extraction_Comparison\")","metadata":{"id":"K5l9C8RiTw6O","execution":{"iopub.status.busy":"2023-10-06T10:13:39.765774Z","iopub.execute_input":"2023-10-06T10:13:39.766266Z","iopub.status.idle":"2023-10-06T10:13:39.817745Z","shell.execute_reply.started":"2023-10-06T10:13:39.766229Z","shell.execute_reply":"2023-10-06T10:13:39.816533Z"},"trusted":true},"execution_count":40,"outputs":[]},{"cell_type":"markdown","source":"**`Note:`** Information, when saved as a CSV (Comma Separated Values) file, may lose certain formatting or structural details, depending on the complexity of the data. CSV files are primarily used for storing tabular data and may not preserve specialized data types, hierarchical structures, or formatting elements found in the original data.","metadata":{}},{"cell_type":"markdown","source":"**No Significant change found in Keyword Extraction with different model and parameter tunning**","metadata":{}}]}