spark-streaming

Statistics of coin’s volume in real time on Binance in the last 1 hour. Use Kafka to get data from Binance API, then Spark Streaming reads data stream.

flask kafka spark chartjs spark-streaming binance-api

Updated Apr 17, 2023
HTML

gaurav-aiml / ecommerce-data-pipeline

Star

As a Data Engineer for a fictional E-commerce startup, this project addresses the task of analyzing the web server logs to find the number of product pages visited and the number of items in the cart.

airflow real-time kafka spark-streaming data-pipeline data-engineer

Updated Mar 28, 2021
HTML

rutvik5 / ncsu-csc591-adbi

Star

Python projects related to the course- Algorithms for Data Guided Business Intelligence

kafka spark sentiment-analysis adwords community-detection spark-streaming logistic-regression recommender-system data-wrangling bayesian-data-analysis doc2vec-word2vec stochastic-gradient-descent bipartite-graphs graph-embedding

Updated Mar 16, 2020
HTML

mtpatter / bilao

Sponsor

Star

Jupyter notebooks for filtering Kafka data with Spark Streaming.

docker kafka spark avro jupyter-notebook spark-streaming

Updated Nov 15, 2021
HTML

EdoWhite / RealTime-CreditCard-Transactions-Analysis

Star

End-to-end real-time credit card transactions application. Made with Kafka, Spark, Bootstrap, ECharts, RxJS.

docker kafka spark docker-compose spark-streaming echarts spark-sql boostrap5

Updated Jun 18, 2022
HTML

hosnaa / Apache-Spark-Streaming-Analysis

Star

Analysis for a streaming daily retail data using Spark structured streaming and querying this data to get insights

query scala spark analytics bigdata spark-streaming scala3 databricks spark-sql big-data-analytics spark-structured-streaming

Updated Jan 12, 2022
HTML

sivansasidharan / sivansasidharan.github.io

Star

My Technical Blog Repo

machine-learning spark hadoop bigdata spark-streaming

Updated Dec 13, 2022
HTML

lesnofla / mbit-m02-dc02-spark

Star

MBIT Big Data 2019-2020 Apache Spark Case Study (M-02 DC-02)

spark jupyter-notebook spark-streaming

Updated Feb 4, 2020
HTML

dhruv007patel / Automated-Twitter-Response-Tool

Star

This project uses machine learning to categorize and prioritize airline user tweets based on content and sentiment. The goal is to reduce airlines' workload and provide personalized, empathetic responses to users. By training a sentiment analysis model, airlines can better understand customers' needs and improve their overall service on Twitter.

kafka sentiment-analysis lstm spark-streaming lda nlp-machine-learning bilstm roberta nrc-emotion-lexicon

Updated Apr 11, 2022
HTML

savacano28 / big-data-training

Star

Just a few proyects with py, r, spark, storm, and others...

python r spark hive storm spark-streaming pig haddop

Updated Apr 20, 2021
HTML

georgeha / kafka_spark_startup

Star

scripts for starting up a kafka+spark cluster on a slurm manager

kafka spark-streaming

Updated Jan 14, 2018
HTML

dgallitelli / IOTBigDataProcessing-TPT

Star

Official repo for DK908 - IOT Big Data Processing

python java iot scala big-data spark spark-streaming hadoop-mapreduce telecom-paristech

Updated Nov 6, 2017
HTML

AVSBharadwaj / Big-Data-Pipeline-

Star

A Data pipeline and a analytics dashboard that tracks the number of data updates in interval of 1 mins. The result is shown as a graph on the web browser. The input data is first supplied to Kafka, from which it gets streamlined to Spark streaming where some processing happens. Then the final data is updated on the browser.

nodejs kafka spark zookeeper spark-streaming