Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor -dns flag #380

Merged
merged 3 commits into from
Oct 16, 2023
Merged

refactor -dns flag #380

merged 3 commits into from
Oct 16, 2023

Conversation

dogancanbakir
Copy link
Member

Closes #374.

CLI output: 
$ go run . -u google.com -dns
  

  _____ _    _____  __
 |_   _| |  / __\ \/ /
   | | | |__\__ \>  < 
   |_| |____|___/_/\_\  v1.1.5

                projectdiscovery.io

[INF] Current tlsx version v1.1.5 (latest)
google.co.jp
google.com.br
google.es
recaptcha-cn.net
safeframe.googlesyndication-cn.com
gcp.gvt2.com
google.com
crowdsource.google.com
datacompute.google.com
googledownloads.cn
goo.gl
googlecommerce.com
google.com.ar
google.it
gstatic.cn
safenup.googlesandbox-cn.com
metric.gstatic.com
url.google.com
google.com.tr
ampproject.org.cn
googlevads-cn.com
googleoptimize-cn.com
g.doubleclick.cn
gstatic.com
yt.be
source.android.google.cn
origin-test.bdn.dev
android.com
ggpht.cn
youtu.be
google.cl
google.hu
googlevideo.com
google-analytics-cn.com
googleapis-cn.com
g.doubleclick-cn.net
googlesyndication-cn.com
gvt1-cn.com
youtube.com
developer.android.google.cn
google.fr
googleadapis.com
doubleclick-cn.net
googleflights-cn.net
googlesandbox-cn.com
youtube-nocookie.com
urchin.com
google.co.in
google.de
googleapis.cn
gstatic-cn.com
gkecnapps.cn
googletraveladservices-cn.com
flash.android.com
cloud.google.com
dartsearch-cn.net
googletagmanager-cn.com
admob-cn.com
google.ca
google.nl
google.pt
googleadservices-cn.com
doubleclick.cn
youtubekids.com
android.clients.google.com
google.pl
googlecnapps.cn
googleapps-cn.com
recaptcha.net.cn
ampproject.net.cn
fls.doubleclick.cn
gvt2-cn.com
gvt2.com
ytimg.com
appengine.google.com
google.co.uk
google.com.au
google.com.vn
widevine.cn
app-measurement-cn.com
gvt1.com
google.com.co
fls.doubleclick-cn.net
g.cn
youtubeeducation.com
developers.android.google.cn
google.com.mx
g.co
bdn.dev
googletagservices-cn.com
2mdn-cn.net
gcpcdn.gvt1.com
www.goo.gl
google-analytics.com
[INF] Connections made using crypto/tls: 1, zcrypto/tls: 0, openssl: 0
JSON output: 
$ go run . -u google.com -dns -json | jq
  

  _____ _    _____  __
 |_   _| |  / __\ \/ /
   | | | |__\__ \>  < 
   |_| |____|___/_/\_\  v1.1.5

                projectdiscovery.io

[INF] Current tlsx version v1.1.5 (latest)
[INF] Connections made using crypto/tls: 1, zcrypto/tls: 0, openssl: 0
{
  "timestamp": "2023-10-16T12:43:11.543740844Z",
  "host": "google.com",
  "ip": "216.239.38.120",
  "port": "443",
  "probe_status": true,
  "tls_version": "tls13",
  "cipher": "TLS_AES_128_GCM_SHA256",
  "not_before": "2023-09-18T08:19:26Z",
  "not_after": "2023-12-11T08:19:25Z",
  "subject_dn": "CN=*.google.com",
  "subject_cn": "*.google.com",
  "subject_an": [
    "*.google.com",
    "*.appengine.google.com",
    "*.bdn.dev",
    "*.origin-test.bdn.dev",
    "*.cloud.google.com",
    "*.crowdsource.google.com",
    "*.datacompute.google.com",
    "*.google.ca",
    "*.google.cl",
    "*.google.co.in",
    "*.google.co.jp",
    "*.google.co.uk",
    "*.google.com.ar",
    "*.google.com.au",
    "*.google.com.br",
    "*.google.com.co",
    "*.google.com.mx",
    "*.google.com.tr",
    "*.google.com.vn",
    "*.google.de",
    "*.google.es",
    "*.google.fr",
    "*.google.hu",
    "*.google.it",
    "*.google.nl",
    "*.google.pl",
    "*.google.pt",
    "*.googleadapis.com",
    "*.googleapis.cn",
    "*.googlevideo.com",
    "*.gstatic.cn",
    "*.gstatic-cn.com",
    "googlecnapps.cn",
    "*.googlecnapps.cn",
    "googleapps-cn.com",
    "*.googleapps-cn.com",
    "gkecnapps.cn",
    "*.gkecnapps.cn",
    "googledownloads.cn",
    "*.googledownloads.cn",
    "recaptcha.net.cn",
    "*.recaptcha.net.cn",
    "recaptcha-cn.net",
    "*.recaptcha-cn.net",
    "widevine.cn",
    "*.widevine.cn",
    "ampproject.org.cn",
    "*.ampproject.org.cn",
    "ampproject.net.cn",
    "*.ampproject.net.cn",
    "google-analytics-cn.com",
    "*.google-analytics-cn.com",
    "googleadservices-cn.com",
    "*.googleadservices-cn.com",
    "googlevads-cn.com",
    "*.googlevads-cn.com",
    "googleapis-cn.com",
    "*.googleapis-cn.com",
    "googleoptimize-cn.com",
    "*.googleoptimize-cn.com",
    "doubleclick-cn.net",
    "*.doubleclick-cn.net",
    "*.fls.doubleclick-cn.net",
    "*.g.doubleclick-cn.net",
    "doubleclick.cn",
    "*.doubleclick.cn",
    "*.fls.doubleclick.cn",
    "*.g.doubleclick.cn",
    "dartsearch-cn.net",
    "*.dartsearch-cn.net",
    "googletraveladservices-cn.com",
    "*.googletraveladservices-cn.com",
    "googletagservices-cn.com",
    "*.googletagservices-cn.com",
    "googletagmanager-cn.com",
    "*.googletagmanager-cn.com",
    "googlesyndication-cn.com",
    "*.googlesyndication-cn.com",
    "*.safeframe.googlesyndication-cn.com",
    "app-measurement-cn.com",
    "*.app-measurement-cn.com",
    "gvt1-cn.com",
    "*.gvt1-cn.com",
    "gvt2-cn.com",
    "*.gvt2-cn.com",
    "2mdn-cn.net",
    "*.2mdn-cn.net",
    "googleflights-cn.net",
    "*.googleflights-cn.net",
    "admob-cn.com",
    "*.admob-cn.com",
    "googlesandbox-cn.com",
    "*.googlesandbox-cn.com",
    "*.safenup.googlesandbox-cn.com",
    "*.gstatic.com",
    "*.metric.gstatic.com",
    "*.gvt1.com",
    "*.gcpcdn.gvt1.com",
    "*.gvt2.com",
    "*.gcp.gvt2.com",
    "*.url.google.com",
    "*.youtube-nocookie.com",
    "*.ytimg.com",
    "android.com",
    "*.android.com",
    "*.flash.android.com",
    "g.cn",
    "*.g.cn",
    "g.co",
    "*.g.co",
    "goo.gl",
    "www.goo.gl",
    "google-analytics.com",
    "*.google-analytics.com",
    "google.com",
    "googlecommerce.com",
    "*.googlecommerce.com",
    "ggpht.cn",
    "*.ggpht.cn",
    "urchin.com",
    "*.urchin.com",
    "youtu.be",
    "youtube.com",
    "*.youtube.com",
    "youtubeeducation.com",
    "*.youtubeeducation.com",
    "youtubekids.com",
    "*.youtubekids.com",
    "yt.be",
    "*.yt.be",
    "android.clients.google.com",
    "developer.android.google.cn",
    "developers.android.google.cn",
    "source.android.google.cn"
  ],
  "domains": [
    "google.com.tr",
    "google.fr",
    "googleadapis.com",
    "googleadservices-cn.com",
    "youtube-nocookie.com",
    "android.com",
    "android.clients.google.com",
    "google.com",
    "cloud.google.com",
    "fls.doubleclick-cn.net",
    "gstatic.com",
    "google.com.br",
    "google.com.mx",
    "google.pt",
    "recaptcha-cn.net",
    "dartsearch-cn.net",
    "gcp.gvt2.com",
    "flash.android.com",
    "youtubekids.com",
    "appengine.google.com",
    "gstatic.cn",
    "googledownloads.cn",
    "ampproject.net.cn",
    "admob-cn.com",
    "safenup.googlesandbox-cn.com",
    "youtubeeducation.com",
    "google.ca",
    "g.cn",
    "www.goo.gl",
    "youtube.com",
    "gvt2.com",
    "bdn.dev",
    "datacompute.google.com",
    "google.co.jp",
    "google.com.au",
    "recaptcha.net.cn",
    "googlevads-cn.com",
    "doubleclick.cn",
    "ggpht.cn",
    "google.com.ar",
    "gstatic-cn.com",
    "googleapps-cn.com",
    "gvt1-cn.com",
    "gvt2-cn.com",
    "googleflights-cn.net",
    "yt.be",
    "origin-test.bdn.dev",
    "crowdsource.google.com",
    "google.nl",
    "googlecnapps.cn",
    "googlesyndication-cn.com",
    "g.co",
    "google.pl",
    "doubleclick-cn.net",
    "googlesandbox-cn.com",
    "ytimg.com",
    "googlecommerce.com",
    "google.hu",
    "ampproject.org.cn",
    "googletagservices-cn.com",
    "2mdn-cn.net",
    "metric.gstatic.com",
    "gcpcdn.gvt1.com",
    "google.com.co",
    "fls.doubleclick.cn",
    "googletagmanager-cn.com",
    "developer.android.google.cn",
    "google.com.vn",
    "google.de",
    "googleapis.cn",
    "googletraveladservices-cn.com",
    "app-measurement-cn.com",
    "url.google.com",
    "youtu.be",
    "source.android.google.cn",
    "google.co.in",
    "google.it",
    "widevine.cn",
    "googleapis-cn.com",
    "goo.gl",
    "urchin.com",
    "developers.android.google.cn",
    "google.cl",
    "google-analytics-cn.com",
    "googleoptimize-cn.com",
    "g.doubleclick-cn.net",
    "safeframe.googlesyndication-cn.com",
    "google-analytics.com",
    "gkecnapps.cn",
    "google.co.uk",
    "google.es",
    "googlevideo.com",
    "g.doubleclick.cn",
    "gvt1.com"
  ],
  "serial": "6F:74:D0:38:20:CB:1B:45:10:CB:4C:1E:A2:7D:A2:EB",
  "issuer_dn": "CN=GTS CA 1C3, O=Google Trust Services LLC, C=US",
  "issuer_cn": "GTS CA 1C3",
  "issuer_org": [
    "Google Trust Services LLC"
  ],
  "fingerprint_hash": {
    "md5": "1ee175cb8553b1eb3c915d5b8514e40e",
    "sha1": "091e689fbd404b478dacbefeef35d652c1a0ec9f",
    "sha256": "b235f7c569490f2b2b861d2237e303337fe45a80ffec55dc140abda69e843d51"
  },
  "wildcard_certificate": true,
  "tls_connection": "ctls",
  "sni": "google.com"
}

@dogancanbakir dogancanbakir self-assigned this Oct 16, 2023
@dogancanbakir dogancanbakir linked an issue Oct 16, 2023 that may be closed by this pull request
expose unique hostnames functionality (-dns flag) in library #374
Closed
1 task
tarunKoyalwar
tarunKoyalwar approved these changes Oct 16, 2023
View reviewed changes
Copy link
Member

@tarunKoyalwar tarunKoyalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm !

did some minor changes

  • moved dedupe logic to clients package this way domains feature will now work even if we directly use a specific engine like ztls etc
  • retained global dedupe feature in cli mode using a global sync lock map
$  echo -e "google.com\nyoutube.com" | ./tlsx -dns | wc
  

  _____ _    _____  __
 |_   _| |  / __\ \/ /
   | | | |__\__ \>  < 
   |_| |____|___/_/\_\	v1.1.5

		projectdiscovery.io

[INF] Current tlsx version v1.1.5 (latest)
[INF] Connections made using crypto/tls: 2, zcrypto/tls: 0, openssl: 0
      94      94    1516

Note: json field is now renamed ( hostname -> domains ) to avoid collision in nuclei

@tarunKoyalwar tarunKoyalwar mentioned this pull request Oct 16, 2023
Merged
4 tasks
@ehsandeep ehsandeep merged commit 06dd68e into dev Oct 16, 2023
8 checks passed
@ehsandeep ehsandeep deleted the refactor_displaydns branch October 16, 2023 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tarunKoyalwar tarunKoyalwar tarunKoyalwar approved these changes

@ehsandeep ehsandeep ehsandeep approved these changes

Assignees

@dogancanbakir dogancanbakir

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

expose unique hostnames functionality (-dns flag) in library
3 participants
@dogancanbakir @ehsandeep @tarunKoyalwar