ScrapQuest is your go-to solution for effortlessly extracting data from websites. It's a simple yet powerful web scraper built with Next.js, TypeScript, and Puppeteer with Browserless. With ScrapQuest, you can quickly gather valuable information from any website without hassle.
ScrapQuest.mp4
Frontend Deployed here :- scrap-quest.vercel.app/
Note
Due to some limitations, website functionality is limited on vercel. So setup project locally for complete functionality
- Next.js: A React framework for building server-side rendered and static web applications.
- Puppeteer: A Node.js library for controlling headless Chrome or Chromium, used here for web scraping.
- Browserless: A service that provides headless browser instances for Puppeteer.
- TypeScript: A statically typed superset of JavaScript that adds type definitions for improved code quality and developer productivity.
Follow these steps to set up ScrapQuest locally:
- Node.js (version 18) installed on your machine. You can download it here.
-
Fork the Repository
-
Clone your Fork from GitHub
git clone https://github.com/<your-user-name>/ScrapQuest.git
-
cd ScrapQuest -
docker-compose up
-
Fork the Repository
-
Clone your Fork from GitHub
git clone https://github.com/<your-user-name>/ScrapQuest.git
-
cd ScrapQuest -
npm install
-
npm run dev
- open localhost:3000 on the browser
We welcome contributions from the community to improve ScrapQuest. Whether it's bug fixes, feature enhancements, or documentation improvements, your contributions are valuable.
To submit your contributions, follow these steps:
-
Fork the Repository: Click the "Fork" button at the top right corner of the repository to create your own copy.
-
Clone Your Fork: Clone your forked repository to your local machine using the following command:
git clone https://github.com/Your-Username/ScrapQuest.git -
Create a Branch: Create a new branch for your changes:
git checkout -b my-feature -
Make Changes: Make your desired changes to the codebase.
-
Commit Changes: Commit your changes with a descriptive commit message:
git commit -m "Add new feature" -
Push Changes: Push your changes to your forked repository:
git push origin my-feature -
Submit a Pull Request: Go to your forked repository on GitHub and submit a pull request. Be sure to provide a detailed description of your changes and why they are necessary.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. And we appreciate your interest in contributing to ScrapQuest! Let's make it better together.
| Event Logo | Event Name | Event Description |
|---|---|---|
 |
GirlScript Summer of Code 2024 | GirlScript Summer of Code is a three-month-long Open Source Program conducted every summer by GirlScript Foundation. It is an initiative to bring more beginners to Open-Source Software Development. |
This project is licensed under the MIT License.
for this project!