-
Notifications
You must be signed in to change notification settings - Fork 13
/
change-log.txt
174 lines (135 loc) · 12.6 KB
/
change-log.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
This file describes the following:
A. The log of changes I made to the repo.
B. The results of my efforts.
C. A summary of what else can be done to enable the API to scale to 1M requests per minute.
========================================================================================================================
A. First here is the log of the changes I made to this repo:
1. Add change-log.txt to the repo.
// Added MongoDB to the project.
2. Add mongodb driver to the package.
3. Added db.js file in the database folder to set up and connect to the mongodb.
4. Added README.txt file to the database folder describing how to set up the database using docker.
5. Updated POST /api/usages endpoint to save the data to the 'Usages' collection of the 'busyapi' mongodb database.
6. Added a GET /api/usages/count endpoint to return the number of documents in the 'Usages' collection.
7. Removed unneeded temporary storage for Usages.
// Set the project up for Docker containerization
8. Added a 'docker' folder.
9. Added a README.txt file in the docker folder that explains how to: build the docker image and spin up 4 instances of the API container.
// Set up the project for simple load balancing using nginx.
10. Added an 'nginx' folder.
11. Added a README.txt file in the nginx folder that explains how to start the nginx server which will load balance the busyapi API requests
and dish them out to 4 busyapi API docker containers.
// Added info on how to test the load of the system using Bombardier
12. Added 'bombardier' folder.
13. Added README.txt file in the bombardier folder that explains how to use bombardier to load test the system locally.
14. Removed morgan logger and any other potential things that would write to the console because writing to the console is a blocking operation.
15. Set NODE_ENV to 'production'
16. Rewrite the usage endpoint so that it will save the Usage records in memory for 5 seconds and then bulk write the saved up usage records
So every 5 seconds a bulk write of the usage records that exist in memory will be written to the database.
This definitely increases performance. But it might not be feasible to do this in a real world scenario. It depends on what else
is done with the Usage records and whether or not they need to be immediately saved to the database.
17. Reduced the number of busyapi Docker containers to 3. The reason is I have 4 CPU cores and I wanted to give the mongodb a core.
So 3 cores for the 3 docker API containers and 1 core for the mongodb docker container. This did improve performance.
18. Updated documentation to reflect the rewrite to do a bulk database write instead of 1 request = 1 database write
========================================================================================================================================
B. Here is the results of my efforts.
So I did my best to make this scale to a massive amount of requests using a single computer with 4 CPU cores. My laptop
also doesn't have a Solid State Drive so yeah. Oh and I did all testing / programming inside of a linux virtual machine on
my Windows 10 laptop.
What I did was set up a mongoDb to store the Usage data because keeping the data in memory wasn't going to scale.
See database/README.txt for more detail
Then I turned the API into a docker image that could be spun up as many times as we need to in order to scale us to 1M requests
per minute. Then I spun up 3 containers because I have 4 CPU cores (3 CPU cores for 3 API containers and 1 core for the mongoDb server).
See docker/README.txt for more info.
Then I set up a simple nginx server so we could have one API url to hit; namely 'localhost:3000'. And the nginx server will load
balance the requests between all 3 of the docker API containers. See nginx/README.txt for more info.
And so with all of that I now essentially have scaled up 3 API servers.
Then in order to test the load of the system to see how close I got to our 1M requests per minute, I used Bombardier: a HTTP
benchmarking tool. See bombardier/README.txt for more info.
The final test I ran had bombardier make 250,000 POST requests to localhost:3000/api/usages. And the result was that the server
fulfilled roughly 4,979 requests per second which means roughly 298,740 requests per minute. Here is the actual output of the linux terminal:
anthony@anthony-VirtualBox:~$ bombardier -t 10s -c 1000 -n 250000 http://localhost:3000/api/usages -m PO0,"timestamp":"Tue Nov 01 2016 09:11:51 GMT-0500 (CDT)","medication":"Albuterol"}'
Bombarding http://localhost:3000/api/usages with 250000 request(s) using 1000 connection(s)
250000 / 250000 [===================================================================================================] 100.00% 50s
Done!
Statistics Avg Stdev Max
Reqs/sec 4979.02 1840.25 18098.36
Latency 199.32ms 291.86ms 4.19s
HTTP codes:
1xx - 0, 2xx - 250000, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 2.38MB/s
So did I make it there (1M requests per minute) with my single 4 core laptop? Well no I didn't. But this should demonstrate to you that I am capable of
scaling and I have a way of testing whether my code can handle large amounts of requests. According to the data I would only need
roughly 4 equivalent set ups of my laptop to scale to 1M requests per minute. So no my laptop alone can't handle 1M requests per minute but in the
next section I will discuss how we can scale this API using the pattern I demonstrated here on 1 physical PC.
====================================================================================================================================
C. Here is a discussion of how to use what I have started to fully scale this API up.
In the original assessment email there was some questions that should be included in the write up.
So the point of this section is to describe what can be done to fully scale this API up which is really
what the questions are about. So this section will take the form of a Q & A where the questions come from
the email.
Question 1: Suggested changes to the software architecture/stack that may achieve the goal
Answer:
I just want to take a moment to discuss a decision I made. I originally started out so that each POST api/usages request
would result in a single database insert to save that usage right away. But then I figured performance would increase if I
saved up a bunch of the Usage objects that need to be saved and then save a bunch of them in a bulk operation. So I made
the changes to save up the Usage objects for 5 seconds and then do an .insertMany to insert all of the Usage objects saved
in memory. So every 5 seconds a .insertMany is made to the database instead of .insertOne each request. This more than doubled
the amount of requests per second that were handled. Now this might not always be possible. It really depends on how the Usage
objects are used in the rest of the system. If the Usage objects need to be saved right away as they come in, then it wouldn't
be possible to wait 5 seconds to save them. So it depends on the scenario if this technique can be used or not.
Another thing is that I noticed we do technically have some html that can be served up. I am viewing this more as
an API than something that servers webpages. But one thing we will want to make sure is that the serving of webpages
happens in its own server. This will minimize the number of web page requests and allow the server to focus on
API requests.
Another thing that can be experimented with is creating a node cluster and then containerizing the node cluster. Then we can
spin up Docker instances of the node cluster. Then we could hook up the nginx server (or whatever other load balancer we use)
to hit the node clusters. So for example, we could set up a node cluster that contains 4 instances of the node API server. Then
we could containerize that node cluster. And then we could spin up 3 (or however many we want) of the clusters. If we have 3
clusters with 4 API instances each, then we would effectively have 12 API servers.
One moe thing that can be done is to ensure we are following the Express.js production guide. It includes a bunch
of insight on what should be done before releasing the express.js server into a production environment. I did not
have enough time to go through this guide and perform its recommendations. But one of them is setting NODE_ENV to
production which I did do. http://expressjs.com/en/advanced/best-practice-performance.html
Question 2: Suggested changes to the physical architecture/hardware that may achieve the goal
Answer:
First is that the API needs to be containerized. Which I did using docker. So I made the API into a docker image.
This docker image can then be started up as many times, on as many servers as we need or want to achieve our goal.
In order to scale this up to 1M requests per minute, I would recommend engaging cloud infrastructure. So I can talk
about how that would look using Amazon Web Services. First we would have a server base image so that we could spin up
as many linux servers as we would like. AWS calls this an EC2 base image. Then amazon can start up EC2 instances using
that base image. Then amazon can spin up our API docker image on as many EC2 instances as we configure. Then we hook
it up to Amazon's Elastic Load Balancer (ELB) which will then distribute the load of requests to the various server
instances that we have set up. And then on top of that we can hook it up to Amazon's auto scaling feature so that
we will only have as many server instances / docker API containers as we need for the given moment. Some times of
day have higher traffic than others. And so setting up auto scaling triggers allows us to make sure we rise to the
challenge but not have too many resources always allocated. So in the end Amazon determines how many docker API
containers are running and Amazon determines how many linux server instances are also running. Fully scalable,
unlimited potential, able to handle 1M requests per minute and more!
That section was just talking about scaling the API server code and not talking about the mongo database. So the
second thing that will enable us to scale to 1M requests per minute and beyond is making sure our database can scale
up as well. So I chose mongo db for the job because it is inherently able to work on a distributed system. Meaning
mongo db supports more than one mongo server working together for the same database. So another suggestion would be
to utilize a mongoDb service that allows you to have multiple mongoDb servers hookers up in a distributed system.
This is made possible through sharding and replica sets.
Another thing that would have made my PC perform better would have been to have a Solid State Drive.
Question 3. A working build of the code capable of achieving the goal
Answer:
I believe I do have a working build of the code capable of achieving the goal of 1M requests per minute.
I do have the code. The code needs to be turned into a docker image (see docker/README.txt) but that image is
capable of doing what you want. You just have to spin up enough linux servers and API containers using that
image. Given my data I believe 4 linux servers running 3 API containers each would be able to do it. Probably
only 3 is even needed because my data is based off of my laptop which isn't dedicated to purely running
the API server anyways (it is running Windows 10 and Ubuntu Linux, etc...).
Question 4. Methods of measuring whether or not the goal has been achieved
Answer:
I used a basic benchmarking tool that would give me some quick insight into the request load that my code
and tech stack is capable of. I used Bombardier for that (see bombardier/README.txt for more info).
However a more sophisticated method would be needed in order to really load test the scaled up API set up.
Especially for 1M requests per minute. I would recommend using some cloud testing tools which allows you to test this
exact thing. You would spin up your servers and API docker containers and then set up the testing tool to
hit the server via the specifications you gave it. Configure it for 1M requests per minute using x devices,
etc... And then observe the testing tool's report. Also you can observe the resource charts of the servers for the
duration of the tests. And don't forget to look at the Mongo Db reports of usage and resources as well.
All of this will give you an idea of if the system is handling 1M requests per minute.