New APIs to be added. #7
Comments
Edu4rdSHL
added
help wanted
|
|
Hey @dtnml thanks a lot! Can you give reference links to the APIs documentations? That will make me easy to just read and start working in the implementations instead of looking for docs. |
|
Hi, Report URI just announced Certificate Transparency monitoring as well. |
|
Hi @Edu4rdSHL, Add Support for: threatcrowd.org Add Support for: certificatedetails.com Add support for: transparencyreport.google.com Add Support for github.com Add Support for: netcraft.com class NetcraftEnum(enumratorBaseThreaded):
def __init__(self, domain, subdomains=None, q=None, silent=False, verbose=True):
subdomains = subdomains or []
self.base_url = 'https://searchdns.netcraft.com/?restriction=site+ends+with&host={domain}'
self.engine_name = "Netcraft"
self.lock = threading.Lock()
super(NetcraftEnum, self).__init__(self.base_url, self.engine_name, domain, subdomains, q=q, silent=silent, verbose=verbose)
self.q = q
return
def req(self, url, cookies=None):
cookies = cookies or {}
try:
resp = self.session.get(url, headers=self.headers, timeout=self.timeout, cookies=cookies)
except Exception as e:
self.print_(e)
resp = None
return resp
def get_next(self, resp):
link_regx = re.compile('<A href="(.*?)"><b>Next page</b></a>')
link = link_regx.findall(resp)
link = re.sub('host=.*?%s' % self.domain, 'host=%s' % self.domain, link[0])
url = 'http://searchdns.netcraft.com' + link
return url
def create_cookies(self, cookie):
cookies = dict()
cookies_list = cookie[0:cookie.find(';')].split("=")
cookies[cookies_list[0]] = cookies_list[1]
# hashlib.sha1 requires utf-8 encoded str
cookies['netcraft_js_verification_response'] = hashlib.sha1(urllib.unquote(cookies_list[1]).encode('utf-8')).hexdigest()
return cookies
def get_cookies(self, headers):
if 'set-cookie' in headers:
cookies = self.create_cookies(headers['set-cookie'])
else:
cookies = {}
return cookies
def enumerate(self):
start_url = self.base_url.format(domain='example.com')
resp = self.req(start_url)
cookies = self.get_cookies(resp.headers)
url = self.base_url.format(domain=self.domain)
while True:
resp = self.get_response(self.req(url, cookies))
self.extract_domains(resp)
if 'Next page' not in resp:
return self.subdomains
break
url = self.get_next(resp)
def extract_domains(self, resp):
links_list = list()
link_regx = re.compile('<a href="http://toolbar.netcraft.com/site_report\?url=(.*)">')
try:
links_list = link_regx.findall(resp)
for link in links_list:
subdomain = urlparse.urlparse(link).netloc
if not subdomain.endswith(self.domain):
continue
if subdomain and subdomain not in self.subdomains and subdomain != self.domain:
if self.verbose:
self.print_("%s%s: %s%s" % (R, self.engine_name, W, subdomain))
self.subdomains.append(subdomain.strip())
except Exception:
pass
return links_list |
|
Hello, Threatcrowd is already implemented: https://github.com/Edu4rdSHL/findomain/blob/eda29344fe014f1a0034ededfc74b4daa941aa8e/src/lib.rs#L492-L500 certificatedetails.com doesn't provide a valid JSON. I will ONLY add APIs that reply with a proper JSON output like https://jonlu.ca/anubis/subdomains/google.com. |
|
I just looked at the WaybackMachine API, that should work with the following URLs... |
|
Closing, new APIs requests should be in new issues for easy management. |
Edu4rdSHL
removed
help wanted
Dear users, please put in comments APIs that you think should be added to
findomain, it will help me a lot to improve the tool.Note: what make findomain unique is that it only use APIs and doesn't do searchs in Google, etc. that's the secret why it's so faster. I haven't plans to add that to findomain then put only APIs (post or get) here still if they're not directly relationed with Certificate Transparency logs but can be used to discover subdomains.
Please, the following APIs are already implemented, make sure that the API that you want is not in that list:
Pull requests are more than welcome
The text was updated successfully, but these errors were encountered: