-
Notifications
You must be signed in to change notification settings - Fork 0
/
notes.txt
128 lines (99 loc) · 4.54 KB
/
notes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
Proxy server
https://github.com/joyent/node/wiki/modules
https://github.com/nodejitsu/node-http-proxy
https://github.com/pkrumins/nodejs-proxy
iframes:
http://www.html5rocks.com/en/tutorials/security/sandboxed-iframes/
http requests:
http://stackoverflow.com/questions/9577611/http-get-request-in-node-js-express
https://github.com/tomas/needle
Parsers:
html:
https://github.com/MatthewMueller/cheerio
html events:
http://molily.de/weblog/domcontentloaded
https://developer.mozilla.org/en-US/docs/DOM/Mozilla_event_reference/DOMContentLoaded
http://thanpol.as/javascript/you-dont-need-dom-ready/
iframes:
http://stackoverflow.com/questions/205087/jquery-ready-in-a-dynamically-inserted-iframe
css:
http://stackoverflow.com/questions/10963997/css-parser-for-javascript
https://npmjs.org/package/css/
https://github.com/NV/CSSOM & http://dev.w3.org/csswg/cssom/
https://github.com/nzakas/parser-lib
http:
http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
http://www.mattlunn.me.uk/blog/2012/05/handling-a-http-redirect-in-node-js/
accept headers:
http://stackoverflow.com/a/7310784/689223
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
cookies:
https://github.com/jed/cookies
http://stackoverflow.com/questions/3737062/managing-sessions-in-nodejs
https:
http://docs.nodejitsu.com/articles/HTTP/servers/how-to-create-a-HTTPS-server
http://stackoverflow.com/a/14272874/689223 http://stackoverflow.com/a/8357673/689223
possible ways to redirect to a new page:
html:
a href
css: content("URL") inside of an anchor
JS:
window.location & document.location https://developer.mozilla.org/en-US/docs/DOM/window.location
http redirects
possible ways to refer to content:
css:
url()
relative paths from the css location
relative path from the domain of the css
this is problematic if the domain is different than the page one, since referer won't help and neither will a cookie solution
only way is to parse
full href
html:
*[style="url()"]
*[src] (eg:script)
*[href] (eg:link)
JS:
editing DOM https://developer.mozilla.org/en-US/docs/DOM/MutationObserver#MutationObserverInit
editing CS (no clue how to observe this)
AJAX http://en.wikipedia.org/wiki/XMLHttpRequest
Fallbacks:
1. catch unresolved relative path request and with the help of referer, proxy them correctly.
2. IFRAME: when the iframe goes to a link outside of the proxy, redirect it to the proxy
parsing css:
url can take urls without quotes, but they must not have parenthesis
if urls have quotes they must not have the same type of quote inside of them, eg: url("adsa/?="dasd"&ds") is invalid, but url('adsa/?="dasd"&ds') is perfectly valid
semicolons are perfectly legal in urls http://stackoverflow.com/questions/1178024/can-a-url-contain-a-semi-colon
minify css first to make parsing faster and to reduce the number of assumptions (eg. circumvent white space and comments)
https://github.com/GoalSmashers/clean-css
why not use a css parser?
overkill, we only want to modify the URLS inside of url(URLS) and most css parsers don't parse the value of properties, bringing pretty much nothing new compared to a regex
Regex:
http://blog.stevenlevithan.com/archives/javascript-regex-lookbehind
graphs:
http://stackoverflow.com/questions/7034/graph-visualization-code-in-javascript
RoadMap:
1.Deal with scalability issues (huge datasets)
2.security
make sure document operations are properly authorized first (eg. clients don't have permission to read/write other runs)
bugs:
1. https to google failt with "..EVP_PKEY_get1_RSA:expecting an rsa key.."
node.js bug: https://github.com/joyent/node/issues/4771
2. when parsing with cheerio End of html document is missing
Fix: remove content length from response header
3. when parsing with cheerio some websites are broken, eg: imdb movies
4. iOS6 does not work.
the iFrame sometimes works sometimes doesn't.
CLEAR PROBLEM: sharejs was halting the server due to the long polling bug
Solution: Use websockets, see BUG 5.
5. WebSocket and SockJS not working
Tried: changing to port 80
Did nothing.
Fix: https://github.com/sockjs/sockjs-node/issues/78
Also fixed websockets and Sockjs
6. CSS transition not working on iOS but working on chrome
FIX: -webkit-transition nees the property to apply the transition or the all inclusive property "all"
eg: -webkit-transition: 1s; won't work, "all" is necessary, like this: -webkit-transition: all 1s;
Uneded bugs:
a. Couldn't make node work on port 80
Did not work: Stopping Apache
Running node as root