-
-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use MessagePack for the Unix socket API #75
Merged
Merged
Changes from 22 commits
Commits
Show all changes
183 commits
Select commit
Hold shift + click to select a range
941dfb2
Implement HTTP server into FTL, listening on port 4747 by default
DL6ER 38c706d
Merge branch 'development' into new/API
DL6ER 3b99d82
Return error 404 if a browser is asking for favicon.ico
DL6ER 03fc5be
Interpret HEAD request
DL6ER e159b56
We don't allow caching the responds of FTL
DL6ER 19c54b0
Replace sprintf() + swrite() by the new ssend() routine. This enhance…
DL6ER 35a1882
Add HTTP server tests
DL6ER a4fc933
Use silent mode if curl based tests to suppress progress meter display
DL6ER d7d57a8
Show last two lines of FTL logs when waiting for socket to become ready
DL6ER f84b461
Add a small delay when starting the threads
DL6ER 0ef2256
Test for both ports 4711 and 4747 before starting the tests
DL6ER b2688bf
Add type variable to all request subroutines which can later be used …
DL6ER 1d29630
Implement API JSON output for summary
AzureMarker 7617293
Only show API request HTTP data when debugging
AzureMarker 5f59aff
Add Content-Length to 404 API response
AzureMarker e20fe2a
Output JSON data w/o using any extra JSON module
DL6ER b15f537
Clean socket routines, move detection of what the client is asking fo…
DL6ER 1481176
Added overTime statistics to API, http://pi.hole:4747/stats/overTime …
DL6ER 824b9e9
Improve/add more comments to the code
DL6ER 4643f97
Add /stats/top_domains and /stats/top_ads and parameters therein
DL6ER 2b439cf
Remove cJSON files. Note that it can be re-added cleanly into the cod…
DL6ER 776bf43
Added /stats/top_clients
DL6ER 7af31af
Add `/stats/forward_dest`
AzureMarker 0e7f484
Add `/stats/query_types`
AzureMarker 249b9e9
Add alias path for forward destinations
AzureMarker ea0a7cc
Make memory error in getQueryTypes not fatal
AzureMarker 9cff885
Two improvements: 1. Analyze only first line of HTTP request (before …
DL6ER f2f9601
Greatly simplify code by not sending Content-Length (optional accordi…
DL6ER 48d355b
Add "/history"
DL6ER b5c500f
Fix all queries time filtering and SOCKET output determination
DL6ER 2179f5d
Don't put integer values in quotes
DL6ER b868c56
Add "/dns/recent_blocked" (can take argument ?limit=123)
DL6ER 2aa2ff1
Added "/overTime/forward_dest", renamed "/stats/overTime" -> "/overTi…
DL6ER 9d891d5
Added "/overTime/query_types"
DL6ER 70e3b5d
Move "/overTime/*" -> "/stats/overTime/*"
DL6ER 38ea0a7
Use thread locks for API responses
DL6ER 942250e
Also send "dns_queries_today" in Top Clients request
DL6ER 30a3f7e
Also send "dns_queries_today" in Top Domains / Ads requests
DL6ER 59fe873
Return what is aksed for (top_domains)
DL6ER 5df3233
Send two additional counter values along with the top lists
DL6ER ecc2ef3
Do some formatting
AzureMarker 446a1b9
Add GET /dns/whitelist and /dns/blacklist
AzureMarker cce43dc
It is no severe issue if the lists (e.g. blacklist) are not there - s…
DL6ER 1905f9c
Return data with key "blacklist" or "whitelist" instead of "data". Us…
DL6ER 0e3f8b5
Use getline() instead of the deprecated fgets() to avoid havng to use…
DL6ER 7cef472
Imporve triming of newline character when reading in the lists.
DL6ER 364ad81
Merge branch 'development' into new/API
DL6ER c8f349d
Stop at "&" after domain/client filtering strings
DL6ER cb9d1cd
Add "/dns/status"
DL6ER a06ecba
Make sure to send API headers even if there's no data
AzureMarker 693e69b
Send API headers every time & simplify getList
AzureMarker 6cab551
Merge branch 'development' into new/API
DL6ER c29ce3b
Add first API test
DL6ER 3f330dd
Make silent
DL6ER f8072a0
Minor correction to API tests
DL6ER ec47c81
Adjusted tests
DL6ER 17a7823
Merge branch 'development' into new/API
DL6ER 9893ef6
Adjust tests with new black.list interpretation
DL6ER 5df0395
Change key name of forwards over time JSON
AzureMarker 0dfb855
Merge branch 'development' into new/API
DL6ER f566fd3
Fix tests
DL6ER bbb3df2
Add a missing space, our tests require 100% accuracy
DL6ER cfea5cf
Only give relevant total in top domain JSON output
AzureMarker 542e26c
Merge remote-tracking branch 'origin/new/API' into new/API
AzureMarker 5693b72
Fix top domains and ads tests
AzureMarker e0f462e
The newline?
AzureMarker 08391f4
Rename /stats/overTime/graphs to graph
AzureMarker 52d3364
Rename /history to /stats/history
AzureMarker 7c88707
Rename /dns/recent_blocked to /stats/recent_blocked
AzureMarker ccbf0ee
Add "GET /stats/dashboard"
DL6ER 32c2a41
Don't send header multiple times
DL6ER c9de13b
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker 0a6f5a5
Sanitize domains when outputting `/stats/history`
AzureMarker 345ac16
Revert "Sanitize domains when outputting `/stats/history`"
AzureMarker 4762b8a
Reorganize to split up API functions
AzureMarker f64251e
Merge branch 'development' into new/API
DL6ER 551351b
We don't show "pi.hole" anymore in the results. Adjust tests (also fo…
DL6ER ebe0631
Add dummy POST handling for whitelist and blacklist
AzureMarker 9b98101
Fix JSON "not_found" message
AzureMarker 9ec7bd8
Implement simple JSON parsing for addList
AzureMarker 2f8e540
Check if domain is invalid
AzureMarker 3a5752d
Actually add the domain to the list
AzureMarker a743931
Improve API response method to use enums
AzureMarker 87dc82c
Add extra character for null terminator
AzureMarker 1893aa5
Add whitelist and blacklist deletion API endpoints
AzureMarker 06be1c4
Move debug API request logging to see all requests
AzureMarker 729d5d4
Respond to CORS preflight (OPTIONS)
AzureMarker fb574f7
Perform better request endpoint checking
AzureMarker 4842c16
Fix broken domain verification if the domain includes numbers
AzureMarker 834a959
Add Wildlist support
AzureMarker e5c4d19
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker e2f2f0d
Fix tests
AzureMarker 3331c42
Move getPayload to api.c
AzureMarker 604ad7e
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker 3573f54
Fix getVersion after merge
AzureMarker 894bd39
Fix newly merged in API functions
AzureMarker 1f5ca04
Add status to test summary data
AzureMarker 4e90a6e
Add quotation marks around new summary status
AzureMarker 2212e23
Fix domains_being_blocked output
AzureMarker 5005f28
Implement simple password-protection to the HTTP API
AzureMarker ae81270
Fix invalid for loop error
AzureMarker cd2fd11
Make client IPs globally available in FTL (array of strings clientip[…
DL6ER f15b9ed
Make sure the API handler ignores clients when over the limit
AzureMarker 636183b
Validate domain earlier in removeList
AzureMarker c9e40a4
Free client IP data after API handler runs and close rejected sockets
AzureMarker 6050aa5
Add IP address into the authentication check
AzureMarker 8adb3d3
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker f62f9f0
Disable HTTP tests as authorization is not gained in the tests yet
AzureMarker 3d78cc3
Fix tests
AzureMarker 595ab25
Remove HTTP API specific code so it can be replaced with a new protocol
AzureMarker 620bb44
Remove duplicate dnsmasq config file location
AzureMarker 97f6ac5
Remove extra parameter from bind_to_telnet_port
AzureMarker c0048e9
Remove Json libraries
AzureMarker e527985
Implement getStats using MessagePack
AzureMarker 439227a
Don't declare variables in for loops
AzureMarker b5c9c6e
Fix socket test
AzureMarker 616b10a
Add swrite function
AzureMarker 203cc94
Add istelnet array
DL6ER f0793be
Use istelnet[] in getStats()
DL6ER e54b391
Add socket implementation for getOverTime
AzureMarker c0022a9
Use istelnet for getOverTime
AzureMarker adfcacc
Use istelnet for getAllQueries
AzureMarker c6369dd
Remove old HTTP argument parsing from getAllQueries
AzureMarker 4eaf794
Tighten parameters on pack_unsigned_char
AzureMarker 4386138
Use uint8_t instead of unsigned char in msgpack.c
AzureMarker 348a2ee
Add socket implementation for getAllQueries
AzureMarker 7d3dfb6
Don't send fixarray for getAllQueries
AzureMarker e07786e
Remove dead code in getAllQueries
AzureMarker 05f8604
Add socket implementation to getTopDomains and use istelnet
AzureMarker 00f8877
Add socket implementation to getTopClients and use istelnet
AzureMarker 33dd350
Add socket implementation to getForwardDestinations and use istelnet
AzureMarker 3f1ffaf
Add socket implementation to getQueryTypes and use istelnet
AzureMarker 555a0cb
Add socket implementation to getRecentBlocked and use istelnet
AzureMarker 55e18ea
Remove extra formatting changes
AzureMarker 81200a2
Add socket implementation to getMemoryUsage and use istelnet
AzureMarker 89ea996
Fix pack_uint64
AzureMarker 7d0f680
Add socket implementation to getForwardDestinationsOverTime and istelnet
AzureMarker 0814339
Add socket implementation to getClientID and use istelnet
AzureMarker 779a824
Add socket implementation to getQueryTypesOverTime and use istelnet
AzureMarker bd63981
Remove the unused `first` boolean variables (used by Json serialization)
AzureMarker 2671a6f
Add socket implementation of getVersion and use istelnet
AzureMarker 64a1408
Add socket implementation of getDBstats and use istelnet
AzureMarker f90f6c8
Use ssend in getClientsOverTime
AzureMarker dde1602
Add socket implementation for getClientsOverTime and use istelnet
AzureMarker 1482742
Use ssend in getClientNames
AzureMarker 43716b4
Add socket implementation to getClientNames and use istelnet
AzureMarker da7c354
Add socket implementation to getUnknownQueries and use istelnet
AzureMarker 30ae391
Let socket-test take in an optional command
AzureMarker 4ed20a4
Don't show a warning if write() returns 0 (this isn't an error)
AzureMarker 37360bb
Send forward destinations names and IP addresses alongside overTime data
AzureMarker f02c4e8
Reimplement output for getList
AzureMarker b9780dc
Reimplement output for getPiholeStatus
AzureMarker 7d87722
Remove getList
AzureMarker 3158f17
Remove getPiholeStatus
AzureMarker cf2aff7
Remove addList
AzureMarker 071f924
Remove the rest of the DNS commands and unused helpers
AzureMarker 1983d3d
Rename api_stats.c to api.c
AzureMarker 700a21b
Update the Makefile with new API file structure
AzureMarker 353e0cb
Move command() up to the top of the file so it isn't as hidden
AzureMarker 0013edb
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker 31c422e
Fix checkClientLimit
AzureMarker 9b8b232
Fix incorrect merging
AzureMarker c88af99
Add `*.` to domains in getTopDomains if it's for audit and a wildcard
AzureMarker d3d7098
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker cf0cdfe
Reword api.c license header description
AzureMarker 7c57b86
Reword api.h license description header
AzureMarker 1fd7ade
Add back detailed ssend error message
AzureMarker 232507a
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker 0aee819
Implement some of the review comments
DL6ER df3b049
Remove clientips[]
DL6ER 996689f
Remove sleep from main()
DL6ER 9f5ae62
Fix indentation in msgpack.c
AzureMarker 590f1f9
Merge remote-tracking branch 'origin/development' into new/API
AzureMarker 173efff
Don't use buffers in getStats
AzureMarker 76aff45
Clarify getOverTime comment
AzureMarker ad99f97
Fix under-allocation in getTopDomains
AzureMarker 4e8027b
Use float instead of double in getForwardDestinationsOverTime
AzureMarker be5815f
Use float instead of double in getQueryTypesOverTime
AzureMarker 9007af7
Fix under-allocation in getVersion
AzureMarker 42bf9a4
Fix status output in getStats
AzureMarker b6b7b57
Only allow up to 2GB strings to be sent
AzureMarker 9bc0562
Use a static buffer for getallqueries string arguments
AzureMarker 5341b18
Handle possible pack_fixstr or pack_str32 error
AzureMarker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -66,19 +66,31 @@ int main (int argc, char* argv[]) { | |
// the system without the need for another thread to join with the terminated thread | ||
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); | ||
|
||
// Start log analyzing thread | ||
pthread_t piholelogthread; | ||
if(pthread_create( &piholelogthread, &attr, pihole_log_thread, NULL ) != 0) | ||
{ | ||
logg("Unable to open Pi-hole log processing thread. Exiting..."); | ||
killed = 1; | ||
} | ||
sleepms(100); | ||
|
||
// Start SOCKET thread | ||
pthread_t socket_listenthread; | ||
if(pthread_create( &socket_listenthread, &attr, socket_listenting_thread, NULL ) != 0) | ||
if(pthread_create( &socket_listenthread, &attr, socket_listening_thread, NULL ) != 0) | ||
{ | ||
logg("Unable to open socket listening thread. Exiting..."); | ||
killed = 1; | ||
} | ||
sleepms(100); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This one can be removed as well, I think. |
||
|
||
// Start API thread | ||
pthread_t api_listenthread; | ||
if(pthread_create( &api_listenthread, &attr, api_listening_thread, NULL ) != 0) | ||
{ | ||
logg("Unable to open API listening thread. Exiting..."); | ||
killed = 1; | ||
} | ||
|
||
while(!killed) | ||
{ | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this sleep here? Why 100ms? You should probably be using a lock.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this delay is needed at all.