Skip to content

Automatically fine-tunes the ML model with previously learned data.

License

Notifications You must be signed in to change notification settings

AnamolZ/retainAI

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RetainAI

Automatically Fine-Tunes Machine Learning Models with Previously Learned Data -- I'm just bored with this project so if you want to clear up the things that this project will offer you, can complete this by lunching this, in testing phase this project has sucessfully completed, but for lunching this project im not doing so if you want then you can.


Project Overview

RetainAI is a dynamic system designed to continuously scrape stock price data at specific intervals and leverage that data to retrain a machine learning model. What sets this project apart is its ability to retain previously learned knowledge while incorporating new data. This allows the model to evolve, continuously improving its accuracy and making more informed predictions over time.


Key Features

  • Continuous Data Scraping: The system scrapes stock price data at regular intervals (specific hours in daily basis) to gather the most up-to-date information.
  • Model Retention: Unlike traditional training, the model doesn't forget its past learnings. It preserves previous knowledge while adapting to new data, allowing for more accurate and evolving predictions.
  • Ongoing Model Training: After each data scrape, the model is retrained using both new and old data, keeping it up-to-date with the latest market trends while maintaining its prior knowledge base.
  • Accurate Price Predictions: Over time, as the model evolves, it becomes better at predicting stock prices with higher precision.

How It Works

  1. Data Scraping: The system scrapes stock price data from reliable sources at regular intervals.
  2. Training the Model: After collecting new data, the model is retrained using both the new data and the data it has previously learned from, ensuring it retains historical patterns.
  3. Model Evolution: The model refines its predictions over time as more data is added. This continuous learning process makes it increasingly accurate in predicting stock prices.
  4. Predictive Accuracy: The model evolves to reflect the most current market trends, ensuring the predictions remain relevant and precise.

Why RetainAI?

Traditional machine learning models often train on a fixed dataset, which can result in outdated predictions as new data becomes available. RetainAI solves this problem by ensuring that the model retains all previously learned data, making it more robust and capable of adapting to changes over time. This is especially useful for dynamic environments like stock markets, where predictions need to be constantly updated based on the latest information.


Technologies Used

  • Web Scraping: Scrapy, BeautifulSoup, or similar tools for extracting stock price data.
  • Machine Learning: Models like LSTM (Long Short-Term Memory) or other time-series forecasting models.
  • Model Training: Keras/TensorFlow for training the machine learning model.
  • Scheduler: APScheduler or cron jobs to run the scraping and retraining at scheduled intervals.

Project Status

  • Completed with some minor bugs to be resolved.
  • Main focus was on deepening understanding of concepts like web scraping, data preprocessing, machine learning, and continuous model improvement.

Learnings & Takeaways

This project allowed me to gain hands-on experience in a number of areas:

  • Machine Learning: Implementing continuous training and retention of data to improve model predictions over time.
  • Web Scraping: Collecting stock market data and feeding it into the training pipeline.
  • Prediction Systems: Building a system that can evolve and scale over time without forgetting previously learned knowledge.

Contributions

Feel free to contribute to this project! Open an issue or submit a pull request if you want to improve the system, fix bugs, or add new features.


License

This project is licensed under the MIT License - see the LICENSE file for details.


About

Automatically fine-tunes the ML model with previously learned data.

Topics

Resources

License

Stars

Watchers

Forks

Languages