Adding CLI based Website Summarizer

CLI_websitesummarizer/README.md

@@ -0,0 +1,24 @@
+# Website Summarizer CLI
+
+A **command-line interface (CLI) tool** that generates a beautiful, structured summary of any website using **Google's Generative AI** (Gemini). Summaries are saved in Markdown format for easy reading and sharing.
+
+---
+
+## Features
+
+- Fetches text content from any URL.
+- Generates a structured summary in Markdown using **Google Gemini AI**.
+- Displays the summary in the terminal with a typewriter effect.
+- Saves the summary as a `.md` file in a `summaries/` folder.
+- Easy-to-use interactive CLI.
+
+---
+
+## Demo
+
+```bash
+$ python main.py
+Hello I am your chatbot. To exit write Quit
+Enter URL or quit to exit: https://example.com
+<---Summary--->
+✅ Summary saved to 'summaries/example_com_summary.md'

CLI_websitesummarizer/summaries/www_freecodecamp_org_summary.md

@@ -0,0 +1,25 @@
+Generate a beautiful summary of the given site in markdown with proper heading in structured format. for the site www_freecodecamp_org
+# 7 React Projects to Build in 2024
+
+This article outlines seven React project ideas suitable for developers looking to enhance their skills in 2024. Ranging from simple to complex, these projects aim to provide practical experience and build confidence in React development. Each project includes suggested tech stacks and a brief overview of the implementation steps.
+
+## Project Ideas:
+
+1.  **ChatGPT AI App:** Utilize the ChatGPT API to build applications like text summarizers, translation apps, or code explainer tools. A specific example mentioned is "Draw a UI," where a user draws a UI mockup, sends it to ChatGPT, and receives the generated HTML code in return. The suggested stack is Next.js and the `tldraw` npm package, along with the `openai` npm package for API interaction.
+2.  **Personal Website:** A great starting point for React beginners to familiarize themselves with JSX and CSS. Build basic components, add images, links to social media, and showcase future projects. Recommendations include Next.js for statically rendered pages (SEO-friendly), `next/image` for image optimization, and ContentLayer for managing blog posts as Markdown or MDX files.
+3.  **Chat App:** A dynamic web application mimicking familiar chat applications like WhatsApp. The core components include a message area, input for new messages, and a list of contacts. Vite is recommended for project creation, and Supabase can power the backend, providing real-time chat functionality and authentication without server-side code. Supabase Storage can be used to add image and video support.
+4.  **E-Commerce App:** A platform for selling physical or digital products. Start with a basic storefront featuring product images, descriptions, and a "buy" button. Next.js is suggested with Stripe integration for handling purchases. A simple database can manage inventory, updating stock levels as products are added or purchased.
+5.  **Online Marketplace:** An extension of the e-commerce app, adding more products and features like customer reviews. This project introduces the challenge of implementing a shopping cart. The recommended stack is Next.js and Stripe, along with the `use-shopping-cart` package for managing the cart functionality. Supabase or a third-party service like Trustpilot can be used for handling reviews.
+6.  **SaaS App (Software as a Service):** Creating a subscription-based service based on an existing app or a new creation. This could be a paid version of the AI app or chat app. Stripe or Paddle is recommended for handling subscriptions and taxes. Stripe Checkout is suggested for allowing customers to manage and cancel their subscriptions.
+7.  **Real-World App Clone:** The most ambitious project involves cloning an app that the developer uses and enjoys. This requires a deep understanding of the original app's functionality and user interface. A YouTube clone is suggested, requiring the replication of features like menus, notifications, video upload/viewing, comments, and likes. Recommendations include Supabase or PlanetScale (MySQL) for the database, Next.js for the framework, TailwindCSS and Radix UI for the user interface, and Mux for video streaming. Supabase Storage can be used for storing images and media.
+
+## React Bootcamp:
+
+For developers looking for guidance and resources to build these projects, the React Bootcamp is mentioned. It includes:
+
+*   200+ in-depth videos
+*   100+ hands-on React challenges
+*   5+ impressive portfolio projects
+*   10+ essential React cheat sheets
+*   A complete Next.js bootcamp
+*   A complete series of animated videos

CLI_websitesummarizer/websitesummarizer.py

@@ -0,0 +1,68 @@
+import time
+import sys
+from dotenv import load_dotenv
+import os
+import google.generativeai as genai
+from IPython.display import Markdown, display,clear_output
+import requests
+from bs4 import BeautifulSoup
+from urllib.parse import urljoin, urlparse
+
+url=""
+msg=""
+# Step 1: Fetch the webpage
+def fetchingsoup(url1):
+     url=url1
+     headers = {"User-Agent": "Mozilla/5.0"}
+     response = requests.get(url, headers=headers)
+     soup = BeautifulSoup(response.text, 'html.parser')
+     return soup,url
+
+def textfetching(url1):
+    soup,url=fetchingsoup(url1)
+    msg=soup.get_text(separator=' ',strip=True)
+    return msg,url
+load_dotenv()  # This loads the .env file
+api_key = os.getenv("GEMINI_API_KEY")
+if not api_key:
+    raise ValueError("GEMINI_API_KEY not found in .env")
+
+genai.configure(api_key=api_key)
+model=genai.GenerativeModel('gemini-2.0-flash')
+
+sysprompt="Generate a beautiful summary of the given site in markdown with proper heading in structured format."
+def summarygen(msg):
+    try:
+        response1=model.generate_content(sysprompt+" "+msg)
+        return response1.text
+    except Exception as e:
+        return f"Error:{str(e)}"
+
+os.makedirs("summaries", exist_ok=True)
+
+def writer(text,delay=0.02):
+    displayed_text = ""
+    for char in text:
+        print(char, end='', flush=True)
+        time.sleep(delay)
+    print()
+
+
+print("Hello I am your chatbot. To exit write Quit")
+while True:
+    url1=input("Enter URL or quit to exit")
+    if(url1.lower()=="quit"):
+        print("thankyou for visiting")
+        break
+    msg, url=textfetching(url1)
+    summary = summarygen(msg)
+    writer(summary)
+    domain = urlparse(url).netloc.replace('.', '_')
+    filename = f"summaries/{domain}_summary.md"
+    with open(filename, "w", encoding="utf-8") as f:
+        f.write(sysprompt+" for the site "+domain+"\n"+summary)
+    print("✅ Summary saved to 'summary.md'")
+
+
+
+