-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathAppSensor_Update_18th_July
179 lines (149 loc) · 25.2 KB
/
AppSensor_Update_18th_July
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
Tasks Accomplished (July 14th to July 18th 2017):
1. Installed scikit-learn version 0.18.2
2. Found sample Apache web server logs here:
https://www.honeynet.org/node/456
http://log-sharing.dreamhosters.com
Description: The site contains various free shareable log samples from various systems, security and network devices,
applications, etc. The logs are collected from real systems, some contain evidence of compromise
and other malicious activity. Wherever possible, the logs are NOT sanitized, anonymized or modified in any
way (just as they came from the logging system)
License / permission to use: public; use for whatever you want.
Downloaded and extracted a few Apache Access logs from the tar.gz files found here.
3. For data creation/mocking capability:
Found 'Faker', a Python library for generating fake apache web logs.
https://github.com/kiritbasu/Fake-Apache-Log-Generator
Installed the requirements required for Faker using pip install:
fake-factory==0.7.2
numpy==1.11.2
Faker==0.7.3
pytz==2016.7
tzlocal==1.3.0
I was initially getting an error with respect to the faker installation on Mac, on Ubuntu, it worked.
This script is now running, it is possible to generate logs after running the python script:
python apache-fake-log-gen.py -n 100 -o LOG (this generates 100 log lines to a file)
This is the output I'm getting on generating 100 log lines with faker:
(Please find the acces_log file attached with the repo as well).
****************************************************************************
221.91.121.253 - - [18/Jul/2017:20:01:24 +0530] "PUT /search/tag/list HTTP/1.0" 200 5022 "http://www.white.com/app/search/terms/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_7; rv:1.9.3.20) Gecko/2015-06-10 13:03:34 Firefox/3.6.20"
4.72.194.74 - - [18/Jul/2017:20:03:55 +0530] "POST /wp-admin HTTP/1.0" 200 4983 "http://www.johnston.com/" "Opera/9.53.(Windows 98; Win 9x 4.90; en-US) Presto/2.9.170 Version/10.00"
222.79.189.106 - - [18/Jul/2017:20:04:56 +0530] "GET /posts/posts/explore HTTP/1.0" 200 5017 "http://chan-soto.com/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/5311 (KHTML, like Gecko) Chrome/13.0.847.0 Safari/5311"
57.163.21.102 - - [18/Jul/2017:20:06:32 +0530] "GET /explore HTTP/1.0" 200 5047 "http://colon.com/" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_6_4) AppleWebKit/5342 (KHTML, like Gecko) Chrome/15.0.846.0 Safari/5342"
156.218.66.7 - - [18/Jul/2017:20:09:35 +0530] "GET /wp-admin HTTP/1.0" 200 4900 "http://www.molina.org/index/" "Mozilla/5.0 (Windows 98; Win 9x 4.90; sl-SI; rv:1.9.1.20) Gecko/2015-06-19 01:34:58 Firefox/3.6.18"
104.162.213.159 - - [18/Jul/2017:20:12:26 +0530] "PUT /wp-content HTTP/1.0" 200 4991 "http://www.lopez.info/tags/blog/search/home/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2011-05-16 05:08:53 Firefox/14.0"
57.128.128.174 - - [18/Jul/2017:20:14:05 +0530] "GET /app/main/posts HTTP/1.0" 200 5000 "http://www.castro.info/index.html" "Mozilla/5.0 (Windows 98; en-US; rv:1.9.2.20) Gecko/2014-03-28 22:09:35 Firefox/15.0"
252.200.13.36 - - [18/Jul/2017:20:16:31 +0530] "GET /wp-content HTTP/1.0" 200 5008 "http://www.wilson.info/index/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_5; rv:1.9.3.20) Gecko/2010-08-25 05:56:31 Firefox/3.6.12"
175.215.53.92 - - [18/Jul/2017:20:19:29 +0530] "DELETE /apps/cart.jsp?appID=3882 HTTP/1.0" 200 5007 "http://www.maynard-black.net/faq/" "Mozilla/5.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90; Trident/3.1)"
68.24.19.43 - - [18/Jul/2017:20:22:06 +0530] "GET /explore HTTP/1.0" 200 5010 "http://allen-johnson.biz/author/" "Mozilla/5.0 (iPod; U; CPU iPhone OS 3_3 like Mac OS X; sl-SI) AppleWebKit/532.8.7 (KHTML, like Gecko) Version/4.0.5 Mobile/8B115 Safari/6532.8.7"
10.7.92.92 - - [18/Jul/2017:20:23:46 +0530] "GET /search/tag/list HTTP/1.0" 404 4982 "http://www.ward.com/posts/search/" "Opera/9.24.(X11; Linux x86_64; en-US) Presto/2.9.169 Version/11.00"
64.10.159.243 - - [18/Jul/2017:20:26:38 +0530] "GET /wp-admin HTTP/1.0" 200 4966 "http://www.vincent.com/app/categories/privacy/" "Opera/8.38.(X11; Linux i686; it-IT) Presto/2.9.167 Version/10.00"
70.225.67.91 - - [18/Jul/2017:20:30:07 +0530] "GET /list HTTP/1.0" 200 4980 "http://jones.biz/list/tags/main/login.htm" "Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5310 (KHTML, like Gecko) Chrome/14.0.803.0 Safari/5310"
56.173.4.73 - - [18/Jul/2017:20:31:40 +0530] "GET /wp-admin HTTP/1.0" 200 5031 "http://oconnor.biz/categories/tags/index.asp" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_7_7; rv:1.9.6.20) Gecko/2014-11-29 17:49:13 Firefox/3.6.3"
204.120.96.121 - - [18/Jul/2017:20:33:27 +0530] "GET /apps/cart.jsp?appID=1291 HTTP/1.0" 200 5035 "http://www.hart.com/main/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_1) AppleWebKit/5340 (KHTML, like Gecko) Chrome/14.0.844.0 Safari/5340"
4.149.11.175 - - [18/Jul/2017:20:35:11 +0530] "PUT /list HTTP/1.0" 500 4940 "http://hunter-monroe.net/tags/list/main/terms.php" "Mozilla/5.0 (X11; Linux i686; rv:1.9.5.20) Gecko/2013-07-25 19:16:56 Firefox/3.6.8"
232.54.10.167 - - [18/Jul/2017:20:38:34 +0530] "PUT /list HTTP/1.0" 200 5004 "http://www.melendez-ford.org/search/explore/index/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5342 (KHTML, like Gecko) Chrome/15.0.884.0 Safari/5342"
143.202.147.216 - - [18/Jul/2017:20:41:14 +0530] "GET /list HTTP/1.0" 200 4941 "http://willis-spence.org/main/category/posts/index.htm" "Mozilla/5.0 (X11; Linux i686; rv:1.9.6.20) Gecko/2013-11-21 10:37:55 Firefox/3.8"
64.61.101.133 - - [18/Jul/2017:20:44:36 +0530] "DELETE /posts/posts/explore HTTP/1.0" 200 5049 "http://garcia-pierce.info/homepage/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5320 (KHTML, like Gecko) Chrome/13.0.853.0 Safari/5320"
245.145.30.76 - - [18/Jul/2017:20:46:30 +0530] "GET /app/main/posts HTTP/1.0" 200 5034 "http://www.garrison.com/tag/author/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2017-04-27 18:18:28 Firefox/3.6.12"
219.98.245.21 - - [18/Jul/2017:20:50:32 +0530] "GET /posts/posts/explore HTTP/1.0" 200 5011 "http://www.moore-morris.com/blog/list/explore/home/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_1) AppleWebKit/5322 (KHTML, like Gecko) Chrome/13.0.864.0 Safari/5322"
147.193.109.74 - - [18/Jul/2017:20:54:40 +0530] "GET /list HTTP/1.0" 500 4936 "http://harrison.info/" "Mozilla/5.0 (Windows 95; it-IT; rv:1.9.0.20) Gecko/2015-05-17 13:46:55 Firefox/3.6.13"
206.47.193.143 - - [18/Jul/2017:20:59:40 +0530] "DELETE /apps/cart.jsp?appID=9170 HTTP/1.0" 200 4956 "http://www.gonzalez.net/homepage/" "Opera/9.43.(X11; Linux i686; it-IT) Presto/2.9.188 Version/11.00"
105.206.237.43 - - [18/Jul/2017:21:04:05 +0530] "GET /app/main/posts HTTP/1.0" 200 5008 "http://www.singleton.com/app/faq.asp" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_7_5; rv:1.9.2.20) Gecko/2013-03-13 16:32:54 Firefox/3.6.3"
149.10.128.68 - - [18/Jul/2017:21:06:36 +0530] "PUT /wp-admin HTTP/1.0" 200 5006 "http://www.thomas.com/blog/search/privacy.htm" "Mozilla/5.0 (iPod; U; CPU iPhone OS 3_0 like Mac OS X; en-US) AppleWebKit/532.40.5 (KHTML, like Gecko) Version/4.0.5 Mobile/8B118 Safari/6532.40.5"
38.74.185.13 - - [18/Jul/2017:21:09:28 +0530] "GET /app/main/posts HTTP/1.0" 200 5026 "http://waller.com/categories/register.asp" "Mozilla/5.0 (Windows NT 6.1; it-IT; rv:1.9.1.20) Gecko/2010-01-27 09:16:03 Firefox/3.8"
254.26.185.163 - - [18/Jul/2017:21:11:18 +0530] "GET /wp-content HTTP/1.0" 200 5059 "http://www.bennett-ford.net/tags/post.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_1; rv:1.9.4.20) Gecko/2010-02-16 04:21:49 Firefox/3.8"
88.132.10.34 - - [18/Jul/2017:21:16:14 +0530] "PUT /explore HTTP/1.0" 200 4909 "http://wallace.info/" "Mozilla/5.0 (Windows NT 6.1; it-IT; rv:1.9.2.20) Gecko/2014-01-27 09:41:33 Firefox/12.0"
225.167.14.37 - - [18/Jul/2017:21:16:54 +0530] "DELETE /wp-admin HTTP/1.0" 200 4979 "http://miller-ross.com/author/" "Mozilla/5.0 (Windows NT 4.0) AppleWebKit/5351 (KHTML, like Gecko) Chrome/14.0.860.0 Safari/5351"
199.211.176.96 - - [18/Jul/2017:21:18:12 +0530] "GET /app/main/posts HTTP/1.0" 200 4944 "http://white.com/categories/explore/category/search.html" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_4) AppleWebKit/5311 (KHTML, like Gecko) Chrome/14.0.884.0 Safari/5311"
152.128.127.163 - - [18/Jul/2017:21:22:12 +0530] "GET /wp-content HTTP/1.0" 200 5031 "http://smith-jenkins.com/author/" "Mozilla/5.0 (Windows; U; Windows NT 6.0) AppleWebKit/535.24.3 (KHTML, like Gecko) Version/4.0.5 Safari/535.24.3"
48.184.117.28 - - [18/Jul/2017:21:23:32 +0530] "POST /search/tag/list HTTP/1.0" 200 5015 "http://lopez-hodges.com/categories/main.php" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5352 (KHTML, like Gecko) Chrome/14.0.862.0 Safari/5352"
33.251.195.24 - - [18/Jul/2017:21:26:51 +0530] "PUT /wp-admin HTTP/1.0" 200 5023 "http://www.sims.info/homepage.jsp" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_8_3; rv:1.9.2.20) Gecko/2016-08-06 21:00:10 Firefox/3.8"
187.60.228.70 - - [18/Jul/2017:21:29:48 +0530] "GET /apps/cart.jsp?appID=3064 HTTP/1.0" 200 5109 "http://www.mccullough-gonzalez.biz/search/" "Mozilla/5.0 (Windows NT 4.0; it-IT; rv:1.9.1.20) Gecko/2015-10-02 12:45:42 Firefox/5.0"
193.253.33.179 - - [18/Jul/2017:21:30:21 +0530] "GET /list HTTP/1.0" 200 4978 "http://www.anderson.com/category/category.asp" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_6; rv:1.9.6.20) Gecko/2016-04-07 04:26:07 Firefox/3.8"
209.247.156.8 - - [18/Jul/2017:21:34:34 +0530] "PUT /apps/cart.jsp?appID=1255 HTTP/1.0" 200 5041 "http://goodwin.com/privacy.asp" "Mozilla/5.0 (Windows NT 5.1; sl-SI; rv:1.9.2.20) Gecko/2011-03-06 10:24:26 Firefox/3.8"
64.78.232.249 - - [18/Jul/2017:21:35:58 +0530] "GET /app/main/posts HTTP/1.0" 200 5002 "http://www.armstrong.com/main.php" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_6_4; rv:1.9.5.20) Gecko/2011-11-14 05:20:55 Firefox/3.8"
114.228.71.172 - - [18/Jul/2017:21:40:54 +0530] "GET /wp-admin HTTP/1.0" 200 5014 "http://spears-black.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.5.20) Gecko/2014-07-15 14:46:38 Firefox/12.0"
242.220.174.203 - - [18/Jul/2017:21:44:03 +0530] "POST /wp-admin HTTP/1.0" 200 4974 "http://www.hopkins.net/about.php" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2013-07-30 05:15:02 Firefox/3.6.6"
236.178.231.58 - - [18/Jul/2017:21:45:04 +0530] "GET /explore HTTP/1.0" 200 5059 "http://www.thompson-lyons.com/main/author.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_8_0; rv:1.9.2.20) Gecko/2014-05-31 21:39:08 Firefox/8.0"
166.114.216.13 - - [18/Jul/2017:21:46:18 +0530] "GET /app/main/posts HTTP/1.0" 200 5022 "http://vasquez-moss.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0; rv:1.9.3.20) Gecko/2013-03-27 19:19:33 Firefox/3.6.12"
152.186.234.78 - - [18/Jul/2017:21:51:16 +0530] "GET /explore HTTP/1.0" 200 4991 "http://woods.info/categories/homepage.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_8_7; rv:1.9.6.20) Gecko/2010-09-12 22:20:20 Firefox/3.6.7"
180.165.192.54 - - [18/Jul/2017:21:55:37 +0530] "GET /search/tag/list HTTP/1.0" 200 5044 "http://harper.com/privacy/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5312 (KHTML, like Gecko) Chrome/14.0.866.0 Safari/5312"
99.139.235.138 - - [18/Jul/2017:21:57:53 +0530] "GET /wp-admin HTTP/1.0" 301 4992 "http://morales-wood.biz/login.html" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_1; rv:1.9.4.20) Gecko/2016-02-20 20:18:40 Firefox/3.8"
19.38.97.140 - - [18/Jul/2017:21:58:57 +0530] "DELETE /explore HTTP/1.0" 301 5019 "http://martinez.com/terms/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_5) AppleWebKit/5341 (KHTML, like Gecko) Chrome/14.0.896.0 Safari/5341"
133.251.76.4 - - [18/Jul/2017:22:00:04 +0530] "PUT /wp-admin HTTP/1.0" 200 4893 "http://www.harmon-burke.biz/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5331 (KHTML, like Gecko) Chrome/15.0.814.0 Safari/5331"
46.30.10.176 - - [18/Jul/2017:22:02:18 +0530] "GET /search/tag/list HTTP/1.0" 200 5122 "http://watson.com/homepage/" "Opera/8.87.(X11; Linux x86_64; en-US) Presto/2.9.168 Version/10.00"
234.41.152.241 - - [18/Jul/2017:22:04:18 +0530] "GET /apps/cart.jsp?appID=8354 HTTP/1.0" 200 5006 "http://www.smith.com/blog/list/about.html" "Mozilla/5.0 (iPod; U; CPU iPhone OS 3_2 like Mac OS X; sl-SI) AppleWebKit/531.42.3 (KHTML, like Gecko) Version/4.0.5 Mobile/8B115 Safari/6531.42.3"
133.129.96.104 - - [18/Jul/2017:22:05:31 +0530] "GET /app/main/posts HTTP/1.0" 200 4975 "http://www.dixon.biz/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/5340 (KHTML, like Gecko) Chrome/15.0.832.0 Safari/5340"
217.201.149.60 - - [18/Jul/2017:22:10:00 +0530] "POST /list HTTP/1.0" 200 4974 "http://www.smith.biz/category/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.5.20) Gecko/2015-06-20 19:11:27 Firefox/3.6.2"
108.71.244.41 - - [18/Jul/2017:22:11:36 +0530] "GET /app/main/posts HTTP/1.0" 200 4990 "http://www.cisneros.info/terms/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/5332 (KHTML, like Gecko) Chrome/15.0.899.0 Safari/5332"
159.109.244.109 - - [18/Jul/2017:22:13:13 +0530] "GET /apps/cart.jsp?appID=8701 HTTP/1.0" 200 5091 "http://www.sullivan.com/privacy/" "Mozilla/5.0 (Windows 95) AppleWebKit/5360 (KHTML, like Gecko) Chrome/13.0.814.0 Safari/5360"
46.128.176.198 - - [18/Jul/2017:22:17:11 +0530] "GET /app/main/posts HTTP/1.0" 404 5067 "http://spencer.biz/login.html" "Mozilla/5.0 (Windows NT 4.0; sl-SI; rv:1.9.2.20) Gecko/2010-08-02 15:45:14 Firefox/3.6.19"
205.35.244.47 - - [18/Jul/2017:22:21:11 +0530] "DELETE /search/tag/list HTTP/1.0" 200 4966 "http://www.taylor.com/app/posts/faq/" "Mozilla/5.0 (X11; Linux i686; rv:1.9.6.20) Gecko/2016-10-27 18:04:53 Firefox/3.8"
89.61.14.14 - - [18/Jul/2017:22:25:17 +0530] "GET /list HTTP/1.0" 200 4975 "http://www.hoover.com/faq.jsp" "Mozilla/5.0 (Windows 98; Win 9x 4.90; en-US; rv:1.9.1.20) Gecko/2010-10-26 19:17:43 Firefox/3.8"
12.158.97.68 - - [18/Jul/2017:22:27:59 +0530] "GET /explore HTTP/1.0" 200 4987 "http://rivera-parks.biz/category/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2017-02-09 21:31:40 Firefox/8.0"
95.64.136.106 - - [18/Jul/2017:22:29:32 +0530] "PUT /explore HTTP/1.0" 404 4994 "http://jackson.com/privacy.html" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_6_6; rv:1.9.3.20) Gecko/2015-01-09 19:57:07 Firefox/15.0"
154.88.169.149 - - [18/Jul/2017:22:32:16 +0530] "GET /search/tag/list HTTP/1.0" 200 5025 "http://www.delacruz-james.biz/app/tag/wp-content/index.htm" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_6_0) AppleWebKit/5340 (KHTML, like Gecko) Chrome/13.0.809.0 Safari/5340"
58.45.116.0 - - [18/Jul/2017:22:35:38 +0530] "GET /wp-admin HTTP/1.0" 200 5059 "http://silva-pennington.info/main/terms/" "Mozilla/5.0 (Windows NT 5.0; it-IT; rv:1.9.1.20) Gecko/2013-05-03 19:03:20 Firefox/3.8"
3.227.254.232 - - [18/Jul/2017:22:36:47 +0530] "GET /apps/cart.jsp?appID=6948 HTTP/1.0" 200 5004 "http://www.sullivan-reilly.com/index/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5320 (KHTML, like Gecko) Chrome/15.0.809.0 Safari/5320"
179.129.253.139 - - [18/Jul/2017:22:40:34 +0530] "GET /search/tag/list HTTP/1.0" 200 4910 "http://www.gregory.biz/app/list/posts/login/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5351 (KHTML, like Gecko) Chrome/13.0.898.0 Safari/5351"
92.126.156.127 - - [18/Jul/2017:22:41:34 +0530] "GET /posts/posts/explore HTTP/1.0" 200 5004 "http://www.watson-martinez.biz/about.htm" "Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5321 (KHTML, like Gecko) Chrome/14.0.807.0 Safari/5321"
87.92.181.231 - - [18/Jul/2017:22:44:16 +0530] "GET /explore HTTP/1.0" 200 5013 "http://jones-valentine.com/author/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_8_6; rv:1.9.2.20) Gecko/2011-10-31 09:59:47 Firefox/3.8"
209.93.130.175 - - [18/Jul/2017:22:48:39 +0530] "GET /explore HTTP/1.0" 200 5012 "http://vincent.info/terms/" "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_0 like Mac OS X; sl-SI) AppleWebKit/532.8.6 (KHTML, like Gecko) Version/4.0.5 Mobile/8B118 Safari/6532.8.6"
146.198.151.232 - - [18/Jul/2017:22:53:34 +0530] "GET /wp-admin HTTP/1.0" 200 5049 "http://george-barr.net/search.html" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.5.20) Gecko/2016-11-09 03:56:46 Firefox/3.6.4"
230.216.113.11 - - [18/Jul/2017:22:54:58 +0530] "GET /wp-content HTTP/1.0" 200 4997 "http://cummings-mcclure.org/" "Mozilla/5.0 (Windows NT 6.1; it-IT; rv:1.9.1.20) Gecko/2016-11-11 22:41:51 Firefox/3.8"
105.184.119.133 - - [18/Jul/2017:22:58:38 +0530] "GET /apps/cart.jsp?appID=6828 HTTP/1.0" 301 5055 "http://garcia-gonzalez.org/search.htm" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_1; rv:1.9.5.20) Gecko/2016-10-28 03:25:43 Firefox/3.6.13"
252.171.105.218 - - [18/Jul/2017:23:02:05 +0530] "GET /apps/cart.jsp?appID=6437 HTTP/1.0" 301 4990 "http://richardson.com/explore/app/category/home.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5362 (KHTML, like Gecko) Chrome/15.0.848.0 Safari/5362"
204.83.149.11 - - [18/Jul/2017:23:05:58 +0530] "PUT /posts/posts/explore HTTP/1.0" 200 5004 "http://www.taylor.com/tag/app/category/" "Mozilla/5.0 (Windows NT 5.0; en-US; rv:1.9.2.20) Gecko/2017-02-01 01:42:33 Firefox/3.8"
176.252.184.157 - - [18/Jul/2017:23:09:41 +0530] "GET /list HTTP/1.0" 301 4956 "http://jones.com/main/search/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_7) AppleWebKit/5312 (KHTML, like Gecko) Chrome/15.0.834.0 Safari/5312"
202.56.150.82 - - [18/Jul/2017:23:13:40 +0530] "DELETE /wp-admin HTTP/1.0" 200 4964 "http://silva.com/index/" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_6_2; rv:1.9.4.20) Gecko/2015-10-19 20:29:09 Firefox/3.8"
26.145.219.19 - - [18/Jul/2017:23:17:37 +0530] "GET /wp-admin HTTP/1.0" 200 4960 "http://www.vaughn-smith.net/search/search/about/" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_6_5) AppleWebKit/5330 (KHTML, like Gecko) Chrome/13.0.834.0 Safari/5330"
196.253.150.99 - - [18/Jul/2017:23:19:33 +0530] "GET /search/tag/list HTTP/1.0" 200 5005 "http://www.martinez-hayes.com/tag/category.htm" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_6_0 rv:4.0; it-IT) AppleWebKit/535.20.3 (KHTML, like Gecko) Version/4.0 Safari/535.20.3"
76.107.118.255 - - [18/Jul/2017:23:20:52 +0530] "GET /app/main/posts HTTP/1.0" 200 4970 "http://smith.com/terms/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.5.20) Gecko/2017-04-25 13:25:13 Firefox/6.0"
249.110.125.234 - - [18/Jul/2017:23:21:43 +0530] "GET /wp-admin HTTP/1.0" 200 4985 "http://www.blake-barajas.net/register.htm" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5311 (KHTML, like Gecko) Chrome/15.0.886.0 Safari/5311"
201.210.21.64 - - [18/Jul/2017:23:23:31 +0530] "GET /list HTTP/1.0" 200 5004 "http://www.bennett.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5360 (KHTML, like Gecko) Chrome/14.0.892.0 Safari/5360"
241.121.150.252 - - [18/Jul/2017:23:27:08 +0530] "GET /search/tag/list HTTP/1.0" 200 5020 "http://www.wilson.com/index.php" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_6_5; rv:1.9.3.20) Gecko/2011-03-29 18:00:54 Firefox/3.6.17"
252.122.94.165 - - [18/Jul/2017:23:29:23 +0530] "GET /list HTTP/1.0" 200 5026 "http://nelson.net/wp-content/category/" "Mozilla/5.0 (Windows; U; Windows NT 6.0) AppleWebKit/535.50.5 (KHTML, like Gecko) Version/5.1 Safari/535.50.5"
222.33.13.74 - - [18/Jul/2017:23:30:03 +0530] "PUT /posts/posts/explore HTTP/1.0" 200 5029 "http://www.mcintosh.info/wp-content/explore/tag/faq/" "Mozilla/5.0 (Windows CE; en-US; rv:1.9.2.20) Gecko/2015-02-26 22:14:02 Firefox/13.0"
2.56.112.197 - - [18/Jul/2017:23:34:10 +0530] "PUT /explore HTTP/1.0" 200 5048 "http://chavez-davis.com/category.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_9; rv:1.9.3.20) Gecko/2012-10-19 08:30:14 Firefox/3.6.2"
250.102.149.170 - - [18/Jul/2017:23:38:23 +0530] "PUT /wp-content HTTP/1.0" 200 4967 "http://www.valencia-eaton.com/categories/main/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 5.2; Trident/4.0)"
169.74.184.252 - - [18/Jul/2017:23:42:19 +0530] "GET /wp-content HTTP/1.0" 200 4941 "http://jones.com/" "Mozilla/5.0 (Windows NT 6.1; sl-SI; rv:1.9.2.20) Gecko/2010-10-08 06:05:09 Firefox/3.8"
90.207.157.106 - - [18/Jul/2017:23:46:04 +0530] "POST /search/tag/list HTTP/1.0" 200 5052 "http://wells.net/privacy.php" "Opera/9.36.(Windows NT 5.0; it-IT) Presto/2.9.162 Version/11.00"
226.252.197.148 - - [18/Jul/2017:23:50:14 +0530] "GET /wp-content HTTP/1.0" 200 5082 "http://www.peterson.com/register.html" "Mozilla/5.0 (Windows NT 5.1; sl-SI; rv:1.9.1.20) Gecko/2016-10-19 22:37:18 Firefox/3.8"
214.71.140.50 - - [18/Jul/2017:23:52:29 +0530] "GET /apps/cart.jsp?appID=8223 HTTP/1.0" 200 4995 "http://www.burke.org/privacy.htm" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2013-06-30 19:12:50 Firefox/3.6.4"
251.51.97.54 - - [18/Jul/2017:23:54:12 +0530] "GET /list HTTP/1.0" 301 5033 "http://www.wright.org/blog/homepage.php" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2011-07-29 15:11:34 Firefox/3.8"
208.248.12.252 - - [18/Jul/2017:23:55:15 +0530] "GET /posts/posts/explore HTTP/1.0" 200 4989 "http://www.martinez.com/main/" "Mozilla/5.0 (X11; Linux i686; rv:1.9.5.20) Gecko/2016-08-28 06:09:08 Firefox/3.8"
6.157.3.128 - - [18/Jul/2017:23:56:23 +0530] "GET /explore HTTP/1.0" 200 4939 "http://www.wilson-erickson.info/privacy/" "Mozilla/5.0 (X11; Linux i686; rv:1.9.6.20) Gecko/2016-02-22 22:40:22 Firefox/3.6.20"
238.115.117.15 - - [19/Jul/2017:00:01:21 +0530] "GET /explore HTTP/1.0" 404 4894 "http://roberson-cox.com/search.html" "Mozilla/5.0 (Windows NT 5.2; sl-SI; rv:1.9.1.20) Gecko/2015-03-11 23:56:55 Firefox/3.8"
115.66.61.147 - - [19/Jul/2017:00:04:25 +0530] "GET /wp-content HTTP/1.0" 200 5029 "http://www.scott.biz/about/" "Mozilla/5.0 (Windows NT 4.0) AppleWebKit/5332 (KHTML, like Gecko) Chrome/15.0.887.0 Safari/5332"
160.6.145.244 - - [19/Jul/2017:00:06:53 +0530] "PUT /wp-admin HTTP/1.0" 200 4901 "http://dickerson-willis.net/blog/wp-content/author.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/5352 (KHTML, like Gecko) Chrome/14.0.897.0 Safari/5352"
104.250.109.89 - - [19/Jul/2017:00:10:51 +0530] "PUT /explore HTTP/1.0" 200 5034 "http://livingston.biz/wp-content/wp-content/tags/register.html" "Mozilla/5.0 (Windows; U; Windows NT 4.0) AppleWebKit/535.48.1 (KHTML, like Gecko) Version/5.0.5 Safari/535.48.1"
168.137.153.143 - - [19/Jul/2017:00:11:45 +0530] "GET /apps/cart.jsp?appID=5510 HTTP/1.0" 200 4956 "http://www.stevenson.org/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_7; rv:1.9.6.20) Gecko/2014-11-12 13:08:50 Firefox/3.6.2"
3.54.5.45 - - [19/Jul/2017:00:16:06 +0530] "GET /explore HTTP/1.0" 200 4931 "http://www.bailey.info/wp-content/register/" "Mozilla/5.0 (Windows NT 4.0; en-US; rv:1.9.2.20) Gecko/2017-06-07 09:18:04 Firefox/3.6.3"
61.22.242.42 - - [19/Jul/2017:00:16:43 +0530] "DELETE /wp-admin HTTP/1.0" 200 5050 "http://www.alvarez-cook.com/" "Mozilla/5.0 (Windows NT 5.1; it-IT; rv:1.9.1.20) Gecko/2010-03-20 02:09:10 Firefox/3.6.12"
26.95.26.193 - - [19/Jul/2017:00:17:43 +0530] "GET /wp-admin HTTP/1.0" 200 4932 "http://sanchez.com/tag/posts/post/" "Mozilla/5.0 (Macintosh; PPC Mac OS X 10_5_7; rv:1.9.4.20) Gecko/2017-02-06 10:59:01 Firefox/13.0"
226.84.155.235 - - [19/Jul/2017:00:20:13 +0530] "GET /wp-content HTTP/1.0" 301 4980 "http://www.ali-morales.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.5.20) Gecko/2010-09-08 09:09:09 Firefox/3.6.20"
241.77.237.14 - - [19/Jul/2017:00:22:55 +0530] "GET /list HTTP/1.0" 200 4917 "http://hernandez-knight.biz/login.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; rv:1.9.6.20) Gecko/2012-03-22 08:19:00 Firefox/3.6.8"
233.27.12.177 - - [19/Jul/2017:00:24:01 +0530] "GET /app/main/posts HTTP/1.0" 200 5082 "http://www.phillips.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:1.9.7.20) Gecko/2017-01-23 01:56:56 Firefox/3.6.13"
86.111.223.191 - - [19/Jul/2017:00:26:58 +0530] "GET /app/main/posts HTTP/1.0" 200 5029 "http://allen.com/privacy.php" "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3 like Mac OS X; en-US) AppleWebKit/535.13.3 (KHTML, like Gecko) Version/4.0.5 Mobile/8B114 Safari/6535.13.3"
**************************************************************************************
4. For the invalid authentication example, the status that the server would be returning is '401' for
invalid authentication. Found a couple of example sample logs.
x.x.x.90 - - [13/Sep/2006:07:01:53 -0700] "PROPFIND /svn/[xxxx]/Extranet/branches/SOW-101 HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:07:01:51 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:07:00:53 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/2.5 HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:07:00:53 -0700] "PROPFIND /svn/[xxxx]/Extranet/branches/SOW-101 HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:07:00:21 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:06:59:53 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/2.5 HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:06:59:50 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:06:58:52 -0700] "PROPFIND /svn/[xxxx]/[xxxx]/trunk HTTP/1.1" 401 587
x.x.x.90 - - [13/Sep/2006:06:58:52 -0700] "PROPFIND /svn/[xxxx]/Extranet/branches/SOW-101 HTTP/1.1" 401 587
Here, the same client is making requests which are being denied due to invalid credentials over a period of 3 minutes.
Source here:
http://ossec-docs.readthedocs.io/en/latest/log_samples/apache/apache.html
Tasks to be completed:
1. Run scikit-learn algorithms on the above sample data for invalid auth by converting it to JSON first and using features
accordingly.
Tasks completed:
1. Got the Faker running so that we have mocking capability ready. We can log as many lines as we want to a log file or
.gz file
2. Found Open and Ready to use public data for logs.
3. Installed Scikit-Learn