Dukaan Prototype - A Production Grade Dukaan MVP

Django+PostgreSQL+Redis+Nginx+Gunicorn+Docker Compose 🔥 🔥 🔥

Quick Introduction:

This project is a production grade MVP of Dukaan(DIY platform to create your own E-Commerce store).
In this project I've used Django for the backend, PostgreSQL for production database, Introduced caching(Redis) mechanism
for increasing the throughput of the server and used event+time driven caching invalidation mechanism for invalidating cache.
Used Gunicorn for the production server, Nginx for revrese proxy and for serving static files.
Then I have Dockerize my project into 4 containers. i.e Web(Django+Gunicorn), db(PostgreSQL), redis_db(Redis), nginx(Nginx).
And finally I've used Docker Compose for running multiple containers as a single service.

Infrastructre Diagram of Dukaan Prototype:

Live Demo

I've deployed this project in AWS EC2 instance(ubuntu server, 1 gig ram)
Go to http://ec2-13-233-115-87.ap-south-1.compute.amazonaws.com/ and login via below credential(I've already populated the DB for the demo)

username:
demo
password:
demo1234

Web App walkthrough

(Note: While creating the project my aim was to focus enterily in the Backend part and to implement state of the art Backend Infra.
So, I've created a very simple frontend by using just html and css. I'm able to do this by leveraging DTL and using my engineering jugadu mind)

The web app is having minimal functionality of Dukaan. i.e Vendors can add their products and manage them using a dashboard.
Then there is a unique link for each vendor which they can send to their customers. By using this unique link, customers can place the order
by entering the neccesary details. Each placed order then are displayed in the order's dashboard of vendor.
Below are the snapshot of the web app.

Signup Page (http://127.0.0.1:8000 #eg url for showing route) 👇

Login Page ( http://127.0.0.1:8000/login/ #eg url for showing route) 👇

Vendor's "Your Product Dashboard" (http://127.0.0.1:8000/your_product/ #eg url for showing route) 👇

Vendor's Add NeW Product Form ( http://127.0.0.1:8000/new_product/ #eg url for showing route) 👇

Unique Link To Buy Product From vendors (http://127.0.0.1:8000/order/demo #eg url for showing route) 👇

Vendor's Your Orders Dashboard (http://127.0.0.1:8000/your_order/ #eg url for showing route) 👇

Optimization using caching mechanism

Suppose there's a sale in a particular day in a vendor's dukaan shop. In this day the website is going to face high amount of traffic and this is gonna cause heavy load in the backend.
When customers visits the vendor's unique link for buying products.
Then for every visit, django is creating same response again and again. Because of this the throughput of the
server is gonna be decrease drastically. Let's test throughput speed by bursting 100 requests in the vendor's unique link(I have already populated the db with dummy data)

For completing 100 requesting it's taking around 3.38 sec. Pretty slow right?

For copinng with this problem I've implemented caching mechanism(redis). Which create a view level cache for each vendors
order view(this view is mapped to vendor's unique link).
Now, let's again test throughput speed by bursting 100 requests in the vendor's unique link but this time having cache enabled .

Insane, this time it took only 0.95 sec to process 100 requests. Which is roughly 72% increase in throughput 🔥 🔥 🔥

But what will happen when the vendor adds a new product or make the product unavailable? The customers will still get the cached page.
For this problem we've to invalidate cache. Cache Invalidation is a crucial part in any caching mechanism system.
I've implemented two cache Invalidation technique in this project.

1. Event based cahce invalidation.
2. Time Based cache Invalidation.

Let's see it in action and monitor what is happening underhood using redis-cli monitor command.
When the customer first visits the unique url http://127.0.0.1:8000/order/demo #eg unique url then django will accept the request do some query in the Product model and then generate the order page
simultaneously it cahced the response in the Redis Memory and then send the response back to the customer. After that any subsequent request to the unique url will not hit the db rather it'll get the requested page from the cache which is stored
in the Redis memory.

1. When the customer first the unique url http://127.0.0.1:8000/order/demo then this request hit the db do the costly query and generate the order page.
   and this page is then stored in the Redis Memory.
   You can see the $SETX redis command is executed for storing the cache. 👇
   
2. When we reload the page you can see that this time django is not hitting the db and hence not doing the costly query but rather than it's getting the requested page from Redis Memory.
   You can see the $GET redis command is executed for sending the requested page. 👇
   
3. Whenever the vendor is doing any modification in the Product model we're inavalidating the stored cache.
   i) Event Based Cache Invalidation: Whenever the vendor is adding "New Product" or making the available product
   available/unvaliable by using toogle "Yes" or "No" buttons, we're Invalidating the cache.
   ii) Time Based Cache Invalidation: All the stored cached will automatically gets invalidated after 15 mins. When we're making one of the product unavailable the cache are getting Invalidated. You can see the $DEL redis command is executed for Invalidating the cache. 👇

How to run this Project

To run this project you need to install Docker and Docker Compose.
Checkout the documentation if you don't have
How to install Docker
How to install Docker Compose
After that open terminal and clone the repo

$ git clone https://github.com/abhinavsp0730/Dukaan-Prototype
$ cd Dukaan-Prototype

Update the file permissions for making entrypoint.sh executable

$ chmod +x app/entrypoint.sh

Then simply run

# running the docker compose script to buid & spin up docker compose service. 
$ docker-compose -f docker-compose.prod.yml up -d --build 

Then run

# for doing the django migration 
$ docker-compose -f docker-compose.prod.yml exec web python manage.py migrate --noinput

# copying the static files to right dir so that Nginx can serve them
$ docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear

Now visit,
http://localhost:1337/
Yay, your production grade dukaan-protoproject has succefully up and is running 🎉 🎉 🎉
You can verify if it's working properly by runining and then looking the logs
$ docker-compose -f docker-compose.prod.yml logs -f
You can spin down the docker compose service by runing
$ docker-compose -f docker-compose.prod.yml down -v

While I was creating the project I didn't find the example of the exact tech stack in the internet.
Also, there are lots of other thing happening in this project which I didn't cover in the readme.md intentionally.
So, if you want a tutorial blog for this project then send me mail at abhinavsp0730@gmail.com
or say a hello to me in twitter @NeurlAP

Closing Note:

I made this project for getting an internship at Dukaan in the position of django backend intern.
I'll update it below wheter I'm selected or not

Result

I got the internship and then got promoted to
full time Backend Engineer at Dukaan.