-
-
Notifications
You must be signed in to change notification settings - Fork 25
/
CHANGES
1288 lines (961 loc) · 44.6 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
2024-08-30 Jerry Lundström
Release 2.15.2
This releases fixes 3 issues detected by code analysis tools:
- File not closed and memory not freed during error while loading known
TLD file
- https://github.com/DNS-OARC/dsc/security/code-scanning/22
label buffer should be static
- https://github.com/DNS-OARC/dsc/security/code-scanning/20
unsigned difference expression
855f037 CodeQL
b00cb11 Stack
c4d3634 Sonar
26c3b9e Badges, fixes
b4a9171 Workflow
2024-04-23 Jerry Lundström
Release 2.15.1
This release fixes client subnet indexer which overwrote the mask
options during initialization so the conf option `client_v4_mask` and
`client_v6_mask` was never used.
Other changes:
- Update documentation
- Update builtin known TLDs based on PSL
- Update copyright year
d577a97 Copyright
f71edff Known TLDs
dedafdd Client mask
8ef947c Doc
2023-08-09 Jerry Lundström
Release 2.15.0
This release fixes DNS parsing w.r.t. EDNS, implements better loop
detection during name decompression and adds a lot of EDNS indexers
and filters.
Previously the DNS parser expected the additional records to come
straight after the question section. Meaning that if the DNS packet
had any answer or authority records, they would be parsed as additional
records for the OPT record and EDNS information.
Following new indexers has been added:
- edns_cookie
- edns_cookie_len
- edns_cookie_client
- edns_cookie_server
- edns_ecs
- edns_ecs_family
- edns_ecs_source_prefix
- edns_ecs_scope_prefix
- edns_ecs_address
- edns_ecs_subnet
- edns_ede
- edns_ede_code
- edns_ede_textlen
- edns_ede_text
- edns_nsid
- edns_nsid_len
- edns_nsid_data
- edns_nsid_text
Following new filters has been added:
- edns0-only
- edns0-cookie-only
- edns0-nsid-only
- edns0-ede-only
- edns0-ecs-only
See man-page dsc.conf(5) for more information.
Other fixes/additions:
- Only parse entire DNS message if EDNS indexers are used
- `dns_protocol`: Implement proper loop detection during decompression
- `xmalloc`: Check return of `amalloc()` before using `memset()`/`memcpy()` because it's undefined behavior on null pointers
8259f30 EDNS filters
41f3b9a strtohex, nsid text
a666c04 EDNS(0) Client Subnet
b5164fe EDNS
7cabfd9 EDNS0 parsing fixes and additional EDNS0 indexers.
46b1797 memcpy/memset fixes
8fd7b7a EDNS parsing
cee2bf7 EDNS0 parsing, multi RR test
a2c00c9 DNS compression loop detection
9875a3e RR parsing
2023-06-15 Jerry Lundström
Release 2.14.1
Fixed a bug in TLD handling when using `tld_list`, it did not reset
where it was in the QNAME when nothing was found and could therefor
wrongly indicate something as a TLD.
Also fixed a typo in the `dsc.conf` man-page.
976589d GCOV
c3afee4 TLD list, doc typo
2023-04-03 Jerry Lundström
Release 2.14.0
This release adds new configure option to control the file access to
the output files, support for newer DNSTAP, improved DNSTAP message
handling and updated Public Suffix List.
- Fix #279: Add new conf options to control output file access:
- `output_user`: set output file user ownership
- `output_group`: set output file group ownership
- `output_mod`: set output file mode bits
- `dnstap`: Move DNSTAP essential attributes checks inside each type and customize them for that specific type
- Update dnswire dependencies to v0.4.0
- `encryption_index`: Add support for new DNSTAP DNS-over-QUIC socket protocol
- Update builtin Public Suffix List (PSL)
abfe245 DNSTAP
da06317 Output file access
af01a48 DOQ transport, PSL update
2023-02-10 Jerry Lundström
Release 2.13.2
Updated pcap-thread to v4.0.1:
Fixed issue with `pcap_dispatch()` during non-threaded timed runs by
checking packet timestamp and use `pcap_breakloop()` if the run
should end.
Based on reports, it looks like `pcap_dispatch()` won't stop
processing if load is high enough even if documentation says "only
one bufferful of packets is read at a time".
Many thanks to Klaus Darilion @klaus3000 (NIC.AT) for the report
and helping to track down the issue and test fixes!
e7d92fe Fix COPR
7ecf217 pcap-thread
2022-04-21 Jerry Lundström
Release 2.13.1
This patch release is mainly for build and packages where MaxMind DB
library is preferred over the legacy GeoIP library.
MaxMind has announced that the databases for GeoIP will be EOL May 2022
and recommends switching to GeoIP2 databases.
Also updated DSC's description, removing references to the now
discontinued Presenter and pointing to dsc-datatool instead.
d891e2c Package, description
c23406c Optional GeoIP
26dd506 GeoIP
2022-01-28 Jerry Lundström
Release 2.13.0
This release fixes a huge performance issue with hashing IPv6
addresses, adds support for new DNSTAP messages types and protocols,
and adds two new indexers.
Thanks to a patch sent in by Ken Renard (@kdrenard) a rather huge
performance issue related to hashing IPv6 addresses has been solved.
Old code used a very incorrect assumption about addresses in general
and while same way was used for IPv4, it didn't hit as hard as it did
for IPv6.
New code uses hashing functions on both address types and to quote
the GitHub issue (by Ken):
-"This performs about 5% better than what I did (51 sec versus 54 sec)
for 5GB pcap file with nearly 50/50 split of IPv4 and IPv6 (3.7M/3.5M
v4/v6 queries).
Old inXaddr_hash() has been running for 75 minutes and is about 20%
done. I say this is a winner!"
Many thanks to Ken for pointing this out and supplying a patch!
DSC now depends on dnswire v0.3.0 which includes new DNSTAP messages
types and protocols that was recently added to DNSTAP's Protobuf
definition.
The new `UPDATE_QUERY` and `UPDATE_RESPONSE` messages types are
now supported and are interpret as `AUTH_QUERY` and `AUTH_RESPONSE`.
The new socket protocols for DOT, DOH and DNSCrypt are also supported
and are interpret as TCP for indexers such as `ip_proto` and
`transport`. To get stats on the encryption itself you can use the
new indexer `encryption`.
Two new indexers have been added:
- `label_count`: Number of labels in the QNAME
- `encryption`: Indicates whether the DNS message was carried over an
encrypted connection or not, and if so over which. For example
"unencrypted", "dot" (DNS-over-TLS), "doh" (DNS-over-HTTPS).
Other changes:
- `inX_addr`: Rework structure, separate IPv4 and IPv6 addresses
- Fix some DNSTAP tests
- `transport_index`: Fix typo in code documentation
37df703 DNSTAP update, encryption indexer
d27171f Label count indexer
6932247 Adding labellen indexer which counts the number of labels in a DNS message
68cc9c7 New IP hashing
2022-01-13 Jerry Lundström
Release 2.12.0
This release adds a new conf option `tld_list` to control what DSC
considers are TLDs, and a script to convert the Public Suffix List to
this format (see `man dsc-psl-convert` for more information).
For example, using this option will allow DSC to gather statistics on
domains like `co.uk` and `net.au` that would otherwise be counted as
`uk` and `au`.
The release also updates the man-pages, clarifying how to use multiple
`interface` and other similar options. And removes the deprecated cron
upload scripts.
e779a87 Remove upload scripts
2880f93 PSL TLD list
ea04022 Update Copyright and known TLDs
5cbc7a4 Output format
b7e6c35 Doc
e66dae4 dh_auto_test
6a3e817 debhelper
89d033f Bye Travis
fa1c179 Mattermost
2020-10-20 Jerry Lundström
Release 2.11.2
This release fixes a bug in `asn_indexer` that didn't enabled the usage
of MaxMindDB after successful initiation. Other changes include a typo
fix in `configure` and a lot of coverage tests.
395b11a Travis, configure
ffea9ed Tests
8b0bebd Tests
09f8174 Config tests
d1514d4 Coverage
66b018c Coverage, ASN indexer
2020-08-18 Jerry Lundström
Release 2.11.1
This release fixes a 17-year old code cut&paste mistake in the
classification indexer, until now it's been classifying funny query
types based on the query class. This fix was sent in by Jim Hague
(Sinodun), thanks Jim!
Other changes are based on code analysis reports and setup for code
coverage.
8d4763c Correct funny-qtype classification.
a1dd55e getline
29bd143 Coverage
685e504 SonarCloud
f759515 Badges
2020-06-01 Jerry Lundström
Release 2.11.0
This release updates the built in known TLDs table and adds the optional
configuration option `knowntlds_file` to, instead of using the built in
table, load the data from a file.
If compiled with only MaxMindDB support then ASN and Country indexer
would complain (and exit) that no database has been specified.
This release changes the behavior to match that of GeoIP support,
making it possible to run without specifying a database.
Other changes:
- Fix compile warnings
- COPR packaging fixes
- `country_indexer`: Fixed typos in log messages (was copied from ASN)
- Fix issues and false-positives reported by newer version of scan-build
Commits:
e937d1 COPR
1382370 country, asn
423a813 scanbuild
2571b97 Compile warnings
4f69447 Known TLDs
2020-05-07 Jerry Lundström
Release 2.10.0
This release adds new configuration options to `dnstap_unixsock` to
control ownership and permissions for the DNSTAP socket file.
Other fixes:
- Unlink the DNSTAP socket file if an error during initialization occur
- Do hard exit in forks to not run `atexit()` (which will unlink the
DNSTAP socket file)
Commits:
9d1d49a fork
733b286 DNSTAP socket
2020-04-02 Jerry Lundström
Release 2.9.1
This release fixes a few bugs, removes a lot of the debug messages
about DNSTAP and removes GeoIP from openSUSE/SLE packages as it has
been deprecated on those platforms.
Changes:
- `daemon`: Fix bug with listening for SIGINT when in foreground mode
- `dnstap`:
- Fix #217: Unlink UNIX socket on exit if successfully initiated
- Fix startup bug, `exit()` if unable to initialize
- Fix #220:
- Remove/hide a lot of debug messages and the printing of the DNSTAP message
- Clarify a lot of the info and error messages
- Prefix all DNSTAP related messages with `DNSTAP: `
- Fix compile warnings and include headers when GeoIP is missing
- `asn_indexer`: Fix bug, said unknown IPv4 when it was IPv6
Commits:
08bad5b DNSTAP debug
1232264 LGTM
589ea7a GeoIP, asn indexer
4fea0d2 sigint, DNSTAP UNIX socket, DNSTAP init
2020-03-20 Jerry Lundström
Release 2.9.0
This release adds support for receiving DNS messages over DNSTAP along
with documentation updates and eliminated compiler warnings.
To enable DNSTAP support, install dependencies (check `README.md`) and
run configure with `--enable-dnstap`.
New configuration options:
- `dnstap_file`: specify input from DNSTAP file
- `dnstap_unixsock`: specify DNSTAP input from UNIX socket
- `dnstap_tcp`: specify DNSTAP input from TCP connections (dsc listens)
- `dnstap_udp`: specify DNSTAP input from UDP connections (dsc listens)
- `dnstap_network`: specify network information in place of missing DNSTAP attributes
Other changes:
- Add documentation about extra configure options that might be needed for FreeBSD/OpenBSD
- Fix compile warnings on FreeBSD 11.2
- Fix compile warning `snprintf()` truncation
- Packaging updates
Commits:
60e6950 DNSTAP
af0417b README
1f1b489 COPR, spec
435e136 Package
3f24feb FreeBSD 11 compatibility
563b986 Funding
2019-04-23 Jerry Lundström
Release 2.8.1
Added all missing config options for the response time indexer:
- `response_time_mode`
- `response_time_bucket_size`
- `response_time_max_queries`
- `response_time_full_mode`
- `response_time_max_seconds`
- `response_time_max_sec_mode`
Commits:
36f0280 Response time config
2019-02-11 Jerry Lundström
Release 2.8.0
This release brings an new indexer `response_time` (funded by NIC.AT!),
support for MaxMind DB (GeoIP2) and an option to set the DNS port.
The new indexer `response_time` can track queries and report the time
it took to receive the response in buckets of microseconds or in
logarithmic scales (see `response_time_mode`). It will also report
timeouts, missing queries (received a response but have never seen the
query), dropped queries (due to memory limitations) and internal errors.
Here is an example output of log10 mode:
<array name="response_time" dimensions="2" start_time="1478727151"
stop_time="1478727180">
<dimension number="1" type="All"/>
<dimension number="2" type="ResponseTime"/>
<data>
<All val="ALL">
<ResponseTime val="100000-1000000" count="77"/>
<ResponseTime val="10000-100000" count="42"/>
<ResponseTime val="1000-10000" count="3"/>
<ResponseTime val="missing_queries" count="1"/>
</All>
</data>
</array>
New configuration options:
- `asn_indexer_backend`: Control what backend to use for the ASN indexer
- `country_indexer_backend`: Control what backend to use for the
country indexer
- `maxminddb_asn`: Specify database for ASN lookups using MaxMind DB
- `maxminddb_country`: Specify database for country lookups using
MaxMind DB
- `dns_port`: Control the DNS port
- `response_time_mode`: Set the output mode of the response time indexer
- `response_time_bucket_size`: The size of bucket (microseconds)
- Following options exists to control internal aspects of `response_time`
indexer, see man-page for more information:
- `response_time_max_queries`
- `response_time_full_mode`
- `response_time_max_seconds`
- `response_time_max_sec_mode`
Fixes:
- Add LGTM and fix alerts
- Update `pcap_layers` with fixes for `scan-build` warnings
- Fix port in debug output of DNS message, was showing server port
on responses
Commits:
f38a655 License
48cd44e Man-page, interface any, response time
8b9345f LGTM Alert
e57a013 DNS port
38aa018 Response time statistics
7a60d53 Cleanup
5c45ce2 Copyright
0dc8a3c MaxMind DB (GeoIP2)
473387b LGTM, README, packages, scan-build
2018-08-14 Jerry Lundström
Release 2.7.0
Add support for Linux "cooked" capture encapsulation (`DLT_LINUX_SLL`).
Fixes:
- `grok_question()`: Remove usage of `strcpy()`
- `pcap_tcp_handler()`: Use `snprintf()`
- `printable_dnsname()`: Use `snprintf()`
- Fix CID 104450, 186871
Commits:
41d59ac man-page HTML
476d6ed pcap_layers, CID
747131b Configure options
43c9ad0 DLT_LINUX_SLL
8a48667 Support the linux cooked sll frame
bd4a94f Fix CID 104450
2017-08-21 Jerry Lundström
Release 2.6.1
Compatibility fixes for FreeBSD 11.1+ which is now packing `struct ip`.
Commits:
c0cd375 Handle compile warnings and FreeBSD's packing of structs
c528ccb Code formatting and moved external code to own directory
2017-07-11 Jerry Lundström
Release 2.6.0
Two new DNS filters and configuration for client subnet netmask has been
added thanks to pull request submission from Manabu Sonoda (@mimuret), see
`man 5 dsc.conf` for more details.
New DNS filters:
- `servfail-only`: Count only SERVFAIL responses
- `authentic-data-only`: Count only DNS messages with the AD bit is set
New configuration:
- `client_v4_mask`: Set the IPv4 MASK for client_subnet INDEXERS
- `client_v6_mask`: Set the IPv6 MASK for client_subnet INDEXERS
Fixes:
- Set `_DEFAULT_SOURCE`, was giving compile warnings on some platforms
- Update `pcap-thread` to v2.1.3 for compatibility fixes
- Fix bug where extra `"` would be OK in configuration
- Eat all white-space between tokens in configuration
- Minor documentation corrections
Commits:
8a20421 Config parse quote/whitespace bug
4eb91d8 PR review and corrections
1dcdbc1 add supports statistics for DNSSEC validation resolver - SERVFAIL
DNS message filter - AD bit DNS message filter - set custom mask
for ClientSubnet
7c4ce7e Update pcap-thread to v2.1.3
f5d152c Corrected date
04f137d Prepare SPEC for OSB/COPR
402c242 Config header is generated by autotools
2017-03-29 Jerry Lundström
Release 2.5.1
Various compatibility issues and a possible runtime bug, related to
pcap-thread, fixed.
Commits:
5ed03e3 Compat for OS X
8605759 Fix compiler warnings
5fbad26 Update pcap-thread to v2.1.2
47ed110 Update pcap-thread to v2.1.1
2017-03-02 Jerry Lundström
Release 2.5.0
Resolved memory leaks within the IP fragment reassembly code that was
reported by Klaus Darilion (NIC.AT) and added config option to control
some parts of the fragment handling.
Fixes:
- Add `pcap_layers_clear_fragments()` to remove old fragments after
`MAX_FRAG_IDLE` (60 seconds)
- Use correct alloc/free functions for dataset hash
- Fix spacing in dsc.conf(5) man-page
New config option:
- `drop_ip_fragments` will disable IP fragmentation reassembling and
drop any IP packet that is a fragment (even the first)
Commits:
eaee6c0 Drop IP fragments
3ebb687 Issue #146: Fix leak in fragment handling
9a5e377 Use correct alloc/free
35f663c Fix #107: add const
2017-01-27 Jerry Lundström
Release 2.4.0
Since there have been a few major issues with the threaded capturing code
it is now default disabled and have to be enabled with a configure option
to use: `./configure --enable-threads ...`
A lot of work has been done to ensure stability and correct capturing,
as of now `dsc` is continuously running on the testing platforms with
simulated traffic and tests are performance every 5-15 minutes:
https://dev.dns-oarc.net/jenkins/view/dsctest/
With the rewrite of the config parser to C it was missed that Hapy allowed
CR/LF within the values of the options. Changing the C parser to allow
it is a bit of work and having CR/LF within the value may lead to other
issues so it is now documented that CR/LF are not allowed in config option
values.
Fixes:
- The `-T` flag was just controlling pcap-thread usage of threads, it now
controls all usage of threads including how signals are caught.
- Fix program name, was incorrectly set so it would be reported as `/dsc`.
- Use thread safe functions (_r).
- Handle very long config lines by not having a static buffer, instead
let `getline()` allocate as needed.
- Use new activation in pcap-thread to activate the capturing of pcaps
after the initial interval sync have been done during start-up.
- Use factions of second for start-up interval sync and interval wait.
- Fix memory leaks if config options was specified more then once.
- Use new absolute timed run in pcap-thread to more exactly end capturing
at the interval.
- Fix config parsing, was checking for tab when should look for line feed.
- Exit correctly during pcap-thread run to honor `dump_reports_on_exit`.
- Use 100ms as default pcap-thread timeout, was 1s before but the old code
used 250ms.
- Various enhancements to logging of errors.
New config options/features:
- `pcap_buffer_size` can be used to increase the capture buffer within
pcap-thread/libpcap, this can help mitigate dropped packets by the
kernel during interval breaks.
- `no_wait_interval` will skip the interval sync that happens during
start-up and start capturing directly, the end of the interval will
still be the modulus of the interval.
- `pcap_thread_timeout` can be used to change the internal timeout use
in pcap-thread to wait for packets (default 100ms).
- Log non-fatal errors from pcap-thread w.r.t. setting the filter which
can indicate that the filter is running in userland because lack of
support or that it is too large for the kernel.
Special thanks to:
- Anand Buddhdev, RIPE NCC
- Klaus Darilion, NIC.AT
- Vincent Charrade, Nameshield
Commits:
ee59572 Fix #111, fix #116: Update pcap-thread to v2.0.0, remove debug
code
64befef Update copyright year
40a1fb4 Fix #139: Use 100ms as default pcap-thread timeout
2a07185 Fix #137: Graceful exit on signal during run
f1b3ec3 Issue #116: Try and make select issue more clear
950ea96 Fix #133: Return from `Pcap_run()` on signal/errors
667cc91 Issue #116: Add config option pcap_thread_timeout
3c9e073 Notice if non-fatal errors was detected during activation
4ea8f54 Fix #108: Document that CR/LF are not allowed within configuration
line
9fda332 Check for LF and not tab
15a1dc0 Use pcap-thread timed run to interface
1e98f8b Fix potential memory leaks if config options specified more then
once
a9b38e9 Add missing LF and indicate what config option was wrong if
possible
f8a2821 Use fractions of seconds for both start up interval sync and
timed run, always adjust for inter-run processing delay
f47069a Fix #121: Update to pcap-thread latest develop
fc13d73 Issue #116: Feature for not waiting on the interval sync
c832337 Fix #122: Update pcap-thread to v1.2.3 for fix in timed run
4739111 Add `pcap_buffer_size` config option
7d9bf90 Update pcap-thread to v1.2.2
ef43335 Make threads optional and default disabled
c2399cf getline() returns error on eof, don't report error if we are
5c671e6 Clarify config error message and report `getline()` error
8bd6a67 Fix #114: Handle very long lines
47b1e1a Use _r thread safe functions when possible
0f5d883 Update daemon.c
f18e3ea Update doc, -T now disables all usage of threads
57aacbe Honor the -T flag when installing signal handlers
2016-12-22 Jerry Lundström
Release 2.3.0
Rare lockup has been fixed that could happen if a signal was received
in the wrong thread at the wrong time due to `pcap_thread_stop()`
canceling and waiting on threads to join again. The handling of signals
have been improved for threaded and non-threaded operations.
A couple of bugfixes, one to fix loading of GeoIP ASN database and
another to use the lowest 32 bits of an IP address (being v4 or v6)
in the IP hash making it a bit more efficient for v6 addresses.
New functionality for the configure option `local_address`, you can now
specify a network mask (see `man 5 dsc.conf` for syntax).
Commits:
e286298 Fix CID 158968 Bad bit shift operation
c15db43 Update to pcap-thread v1.2.1
1ac06ac Move stopping process to not require a packet
597dd34 Handle signals better with and without pthreads
bcf99e8 Add RPM spec and ACLOCAL_AMFLAGS to build on CentOS 6
667fe69 fixed load geoIP ASN database from config-file
e1304d4 Fix #97: Add optional mask to `local_address` so you can
specify networks
5dae7dd Fix #96: Hash the lowest 32 bits of IP addresses
2016-10-15 Jerry Lundström
Release 2.2.1
Two bugfixes with one being critical (caused segfault, sorry for that)
- `pcap-thread` had an issue that threads where not closed on exit
of `pcap_thread_run()`, this only created many threads in my testing
environment but reports came in that it segfaulted.
- When started `dsc` (not in debug mode) it will wait to align with
the time, that did not get update with the configurable interval
change.
Commits:
6e3654b Fix #90: Update pcap-thread to v1.1.2 to fix a segfault issue
c9350a3 Fix #92: Honor configured interval when aligning
acaf617 minor typo
2016-10-10 Jerry Lundström
Release 2.2.0
Some big changes in this release are the removal of the C++ configure
parser library Hapy and the addition of pcap-thread to (hopefully)
handle capturing packet in a correct and efficiant way.
With that addition of pcap-thread comes new runtime options:
- `-m` sets monitor mode on interfaces
- `-i` sets immediate mode on interfaces
- `-T` disable the usage of threads in pcap thread
Bugfixes:
- d95190a fixes a small memory leak in `Pcap_init()` and a possible
bug where `-p` might never been used because of not being declared
external.
- 55e1056 added check for `netinet/ip_compat.h` for use in
`src/pcap_layers/pcap_layers.c`.
Commits:
0a1ce91 Fix coverity issues
5a1d410 Delete useless line (related to mayasd#84)
443db3e Check if the file was previously linked but not yet unlinked
(Tim CLERC.IM)
02a7621 Fix #82: Oops, `pcap_thread_set_filter()` had changed during
development and missed this
2a8aa29 Move definition of token struct inside and changed name to not
conflict on FreeBSD
43da964 Fix #9: Implement conf parser in C and remove dependency of
Hapy and C++
9f46f0d Update pcap thread to version 1.1.1
d95190a Use pcap thread, new options `-miT` and possible bugfix for
`-p` and a small memory leak fix
55e1056 Fix #77: Check for netinet/ip_compat
4e120f9 Fix travis script, only expand dir.
2016-06-28 Jerry Lundström
Release 2.1.1
Bugfixes:
- 22688c1 Fix pcap/select last_ts
In some cases `select` will return the fd set as if there are packets
to read but there aren't. That would case the last timestamp to not
advance and the `Pcap_run` loop to never finish.
This fix add a check on packets captured and sets last timestamp if
none where caught.
Commits:
d115b3f Correct configuration, missing `;`.
22688c1 Fix pcap/select last_ts
b6d3dd8 Fix package dependencies.
c8979c4 Add debian/ubuntu package files
2016-06-09 Jerry Lundström
Release 2.1.0
This release brings a couple of new features, the ability to change
the interval for which DSC writes out the statistics files and a new
indexer for AS numbers.
There was also an issue detected in Ubuntu 16.04 when reading pcap files
would result in doubling the statistics if running in daemon mode.
Looking at the strace it might be a issue with the kernel, libc, pcap or
a combination that results in the open file handle to reset and essentially
read all the data twice.
Credits:
- Klaus Darilion NIC.AT for interval changes and AS number indexer.
Commits:
8ab8632 Rewrote ASN indexer to remove the need for malloc/free. Removed
the need to malloc ipstr. Use same unknown tags as country
indexer.
89d4984 add ASN index: maps src-IP to AS number using GeoIP ASNum DB
8d5c6bf Fix #14: Remove ncap
c0f00e7 Add test for statistics_interval
cf9ede2 Add checks and documentation for statistics_interval
93eeecd new config option 'statistics_interval' (defaults to 60s)
a28f5d4 Fix #62: Do not go into daemon mode if reading offline files
2bc1abb Add information about puppet module
2016-06-01 Jerry Lundström
Release 2.0.0
This release brings a major update to the DSC software with the separation
of the Collector and the Presenter, this repository will only include
the Collector from now on.
Please read UPGRADE.md for information on upgrading from previous version.
Major changes / additions are:
- Use of Automake and rework of the Makefiles
- Conform to FHS 3.0
- Man-pages (man dsc, man dsc.conf)
- Continuous Integration testing using Travis-CI
- Compatibility testing on Debian, CentOS, FreeBSD and OpenBSD
- Use of Coverity Scan to find defects
- JSON output format, see output_format in dsc.conf(5).
- IPv6 support in country indexer and libgeoip is now runtime
configurable, see geoip_v4_dat / geoip_v6_dat in dsc.conf(5).
- Signal handling and optional write reports on exit, see
dump_reports_on_exit in dsc.conf(5).
- Upload scripts are deprecated
Credits:
- Klaus Darilion NIC.AT for GeoIP IPv6 patch.
- Michael Braunoeder NIC.AT for NXDOMAIN filter patch.
- L-root for overflow bugfix IP fragments.
- McStork for JSON output patch.
Changes since release candidate:
1be5148 Fix #57: Flush the pid file to write it out and add test for
pid file
0f79aa0 Use Semantic Versioning 2.0.0 semver.org
2016-05-10 Jerry Lundström
Release 2.0.0-rc.1
69ef9b4 Add -v to display version
7e5b403 Fix defects
0f64128 Add badges
f795ed3 Old automake needs AM_PROG_CC_C_O
27ae870 Fix #4: Remove old indexers, update indexers and filters
documentation, update authors.
d873411 Fix #38: Use locking to ensure we do not overwrite PID file
800fe83 Fix #35: GeoIP configurable. Fix debug and syslog in country
indexer. Add documentation and config example.
a1dcdf1 Avoid the checks for newer .dat files, this may have performance
impacts.
bb9a059 Add IPv6 support to country indexer and add some failure handling
03e16a8 Include config.h as this defines HAVE_LIBGEOIP. Thus, the geoip
code was not used yet.
7808d2e New dsyslog/dsyslogf/dfprint macro. Change fprintf to dfprintf.
fe47288 Fix #34: Create man-pages.
7a9b3c3 Fix #33: Handle most signals, new config option to dump reports
on exit.
4753eda Add a filter to track NXDOMAIN responses
1565952 Add info about DSP
05ef699 Fix make test in dist.
68bdc9b No default interface in conf example
2be98be Correct libexec directory. Install etc files in subdirectory.
319ac4e Only build dist. Build in a build directory.
96e0e73 Fix #16: conform to FHS. Automake tweaks.
bbcca74 Fix #23: Add missing changes for master branch
197ad52 No need to install autoconf/automake in Travis
eb95ee1 Add license to cron scripts
4f62420 Fix/add make dist and try it in Travis
ac4c634 Reconstruct repository to move out presenter. Update licenses.
Use pcap_layers as a git submodule. Use automake/autoconf.
89c7f4c Import patched pcap_layers.c code with buffer overflow bugfix
2787db2 Only wait a certain number of times for the files to appear
b5d911d Add the first test, simple run and compare gold files
32fd807 Fix #13: Document ability to read packets from pcap files
eee217e Rework some of the Makefiles based on some of the patches found in
the Debian packages
7a2a67e Fix #12: Add hash for dataset names and check for duplicates
ed1eba9 Add base64 for certain non-printable characters in JSON output and
remove extra new-line
580d543 Add output_format and JSON structure description to documentation
d2d1ed2 Fix #3: Reworked JSON output format implementation
2e2f90f Give option for additional output JSON/Extended JSON
a23b6af Ignore generated files
f4214f3 Add Travis CI. Remove old TODAY/tar commands. Add ifndef/def to
all .h files. Fix issue with arpa/nameser_compat.h on OpenBSD by
checking for the header file and only including it if it exists.
Rename configure.scan to configure.ac and change contact info and
version. Update configure using autoconf 2.69 on Ubuntu 14.04 LTS.
Set CC/CPP/CXX/CXXFLAGS in Makefile found by configure.
2016-01-11 Duane Wessels
added dfprintf() macro to improve code readability and avoid
multi-line if (debug_lvl) fprintf(...) statements.
2016-01-11 Duane Wessels
Commenting out ancount and nscount to silence compiler warnings
about unused variables.
2016-01-11 Duane Wessels
Patch from John Heidemann relating to TCP reassembly
- fix for multiple DNS messages per TCP connection that span
multiple segments
- some editorial code changes (== to >=)
- additional debugging statements
2016-01-11 Duane Wessels
Minor fix: debugging statement wasn't protected with debug_flag
check. (thanks John H).
2015-12-23 Duane Wessels
add prototype for dns_message_handle()
2015-12-23 Duane Wessels
Experimental feature to drop "received responses" and "sent queries"
since DSC is generally used to monitor authoritative servers that,
under normal operation, never receive responses nor send queries.
Currently protected with #ifdefs.
2015-12-15 Duane Wessels
Oops, forgot to set dns_message->server_ip_addr for new server_addr
indexer
2015-11-23 Duane Wessels
removing one level of the "dsc/dsc" top directory
2015-11-11 aqadeer
In pcap.c pcap_setnonblock doesn't accept captured files anymore and
must need a device from where it could do live capture. For offline
files to work, a simple check is added to by-pass this porblem.
2015-11-05 Duane Wessels
Added a 'server' indexer. This records the server (query destination)
IP address.
2015-11-05 Duane Wessels
Renamed "client_ipv4" to "client_ip" because it supports v4 and v6
2015-11-04 Duane Wessels
Update copy of pcap_layers.c from https://github.com/wessels/pcap_layers
2015-11-04 Duane Wessels
avoid "void *" pointer arithmetic (Yoshitaka Aharen)
2015-10-09 Duane Wessels
bugfix: handle receiving TCP dns length prefix out of order.
A user reported that when a TCP segment containing only the DNS message
length is received *after* the message it references (i.e., out-of-order),
then DSC goes into a 100% CPU loop. Confirmed that the code doesn't work
correctly when dnslen comes out-of-order, but I wasn't able to easily
reproduce the 100% CPU bug.
The fix is to add a "dnslen_bytes_seen_mask" variable that tracks which
of the two dnslen bytes we've seen. Once both bytes have been seen,
then we can proceed to reassembling the message buffer.
2015-10-09 Duane Wessels
cosmetic: rename ipv4 to ip4 to be consistent with ip6
2015-10-09 Duane Wessels
Regarding recent change to add pcap_layers library, forgot to add it
to Makefile.in (vs Makefile) before committing.
2015-10-09 Duane Wessels
Integrated https://github.com/wessels/pcap_layers with DSC collector.
The DSC code now includes a copy of the pcap_layers library, which does
a better job of extracting the layers of a pcap packet. In particular,
it does IP fragmentation reassembly, which is important for DSC and
RSSAC-002. At this time the DSC code still does TCP reassembly, however.
Removed USE_IPV6 ifdefs. IPV6 support is now always compiled.
2015-10-08 Duane Wessels
Removing "DMC *dns_message_callback" because there is only one that
would ever be used. The callback layer of indirection makes the code
a little confusing, and also this change is in preparation for bringing
in a third-party pcap layer handling library which will work slightly
differently.
2015-10-08 Duane Wessels
The "ip_message" code has not been in use since a commit back in 2012.
But the code lingered, until now.
2015-02-25 Duane Wessels
User reported a concern with the way dsc-xml-extractor.pl called
the operating system 'mv' command. It has been replaced with Perl's
File::Copy::mv().
2013-03-19 Duane Wessels
Fixed a bug in TCP reassembly when the DNS length field was
split between two segments.
2012-08-27 Sebastian Castro
Added 'tc_bit' indexer and dataset to track the frequency
of responses having the TC bit set. Useful with DNSSEC
signed zones.
2012-02-29 Duane Wessels
Added 'qr_aa_bits' dataset and graph. It shows the distribution
of QR/AA values in received messages and may be helpful in
detecting reflector attacks targeting your name server.