Skip to content

Releases: ameenmaali/urldedupe

v1.0.4 - Add --no-extensions and --query-strings-only flags

08 Jun 16:01
8df764b
Compare
Choose a tag to compare

Addition of 2 additional switches for further filtering down
results:

  • --no-extensions|-ne will allow you to discard any results that
    have extensions (.js, .jpg, .exe)
  • --query-strings-only|-qs will allow you to discard any results
    that do not have query strings

v1.0.3 - Dedupe similar URLs and performance improvements

07 Jun 17:53
515c855
Compare
Choose a tag to compare

v1.0.3 Updates

  • Several performance and code style improvements thanks to @larskraemer!
  • Added -s|--similar flag to detect and dedupe similar looking URLs, such as:
https://site.com/api/users/123
https://site.com/api/users/222
https://site.com/api/users/412/profile
https://site.com/users/photos/photo.jpg
https://site.com/users/photos/myPhoto.jpg
https://site.com/users/photos/photo.png

Becomes:

https://site.com/api/users/123
https://site.com/api/users/412/profile
https://site.com/users/photos/photo.jpg

v1.0.2 - Performance upgrades

03 Jun 05:10
Compare
Choose a tag to compare

Performance upgrades to decode method, removing unnecessary vector and final loop, and more.

v.1.0.1 - Use non-regex parsing by default

02 Jun 22:54
Compare
Choose a tag to compare

Adds a new CLI flag for regex parsing, which was previously by default. This is very slow for large data sets, and should only be used when absolute 100% accuracy is needed. In most cases, non-regex parsing is just as accurate and significantly quicker.

v1.0.0 - Initial Release

02 Jun 15:40
1eb4bcb
Compare
Choose a tag to compare
Create LICENSE