diff --git a/CHANGELOG b/CHANGELOG index 5754aa2..08591b2 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [v1.6.2] - 2021-12-11 +### Added +- Add the function of creating asset distribution map +- Optimized the dependency package version +- Added the configuration of hosts crash Serverless scan + ## [v1.6.1] - 2021-11-20 ### Added - Optimized the issue of CTRL+C exiting the program when executing system commands diff --git a/README.md b/README.md index 7ffc9e3..d4566ab 100644 --- a/README.md +++ b/README.md @@ -273,6 +273,30 @@ HostCrash 1.1.1.1 G:\host.txt ![](./images/searchcrashs.png) +**Serverless HostCrash Scan** + +Kunyu v1.6.2 adds an interesting feature that combines the cloud function to perform HOSTS collisions on the target. In this way, our scanned IP is effectively hidden to prevent it from being captured by the target situational awareness, and it also prevents WAF from banning the real IP. , And conceal the features. Through the following scanning effect, it can be found that the scanned IPs are all cloud service vendors and each scan is a random IP address. You can choose whether to enable it by configuring the cloud function address during initialization. + +**Configuration Guide:** [Configuration Method of Cloud Function](./doc/Serverless_EN.md) + +**Related technology:**https://www.anquanke.com/post/id/261551 + +**Situational Awareness Scanning Effect:** + +![](./images/serverless.png) + +**Asset distribution map** + +v1.6.2 adds the CreateMap command, which can generate a geographic location distribution map for the assets retrieved last time, and more vividly describe the mapping relationship between network space and real space. It is located in the same output directory as Excel, and the generated asset map is the same as the last time. The number of search results is related. + +**Generate distribution map** + +![](./images/createmap.png) + +**Web page** + +![](./images/map.png) + **Data result** All search results are saved in the user's root directory, and the directory is created based on the current timestamp. All query results of a single start are stored in an Excel format under one directory, giving a more intuitive experience. The output path can be returned through the ExportPath command. @@ -345,11 +369,11 @@ When using the Pocsuite command linkage, if it is a packaged Kunyu version, the **11. Kunyu can execute system commands as follows. ** **Windows:** - OS_SYSTEM = [**"ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver"**] + OS_SYSTEM = [**"ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver","nmap"]** **Linux/Mac：** -​ OS_SYSTEM = [**"ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname"**] +​ OS_SYSTEM = [**"ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname","nmap"]** **12, Kunyu operating environment** diff --git a/doc/README_CN.md b/doc/README_CN.md index 693e90b..65ee7d9 100644 --- a/doc/README_CN.md +++ b/doc/README_CN.md @@ -271,6 +271,30 @@ HostCrash 1.1.1.1 G:\host.txt ![](../images/searchcrashs.png) +**Serverless HostCrash Scan** + +Kunyu v1.6.2新增了一个有意思的功能，结合云函数对目标进行HOSTS碰撞，通过这样的方式有效的隐藏了我们的扫描IP防止被目标态势感知捕捉到，也防止了WAF对真实IP的封禁，并对特征进行了隐匿，通过下面的扫描效果可以发现扫描的IP均为云服务厂商且每次扫描均为随机IP地址，可以通过初始化时配置云函数地址的方式自主选择是否启用。 + +**配置导读：** [云函数的配置方法](./doc/Serverless_CN.md) + +**相关技术：**https://www.anquanke.com/post/id/261551 + +**态势感知扫描效果：** + +![](../images/serverless.png) + +**资产分布地图** + +v1.6.2新增CreateMap命令，可对上次检索的资产生成地理位置分布图，更形象的描述网络空间和现实空间的映射关系，与Excel位于相同的输出目录下，生成的资产图与上次搜索结果的条数相关。 + +**生成分布图** + +![](../images/createmap.png) + +**Web页面** + +![](../images/map.png) + **数据结果** 搜索的所有结果都保存在用户根目录下，并根据当前时间戳创建目录。单次启动的所有查询结果都在一个目录下，保存为Excel格式，给予更加直观的体验。可以通过ExportPath命令返回输出路径。 @@ -340,17 +364,17 @@ Kunyu的自动补全支持大小写，命令记录等，使用Tab进行补全， **11、Kunyu可执行系统命令如下。** **Windows:** - OS_SYSTEM = [**"ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver"**] + OS_SYSTEM = [**"ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver","nmap"**] **Linux/Mac：** -​ OS_SYSTEM = [**"ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname"**] +​ OS_SYSTEM = [**"ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname","nmap"**] **12、Kunyu运行环境** 这里建议使用Python3.2 — 3.9版本，Python3其他版本可能会有未知的报错，**Python2不可使用**。 -13、设置超时时间 +**13、设置超时时间** 如果HTTP请求没有得到及时响应，可以通过增大timeout时间解决，如:set timeout = 50 diff --git a/doc/Serverless_CN.md b/doc/Serverless_CN.md new file mode 100644 index 0000000..d5151ec --- /dev/null +++ b/doc/Serverless_CN.md @@ -0,0 +1,71 @@ +# Kunyu Serverless HOSTS碰撞配置 + +首先进入配置云函数界面，选择自定义创建，执行环境选为Python3.6，地域都可以，当然针对中国的目标最好选为国内的位置，函数名称任意 + +![](../images/serverless_1.png) + +填入函数代码，具体代码如下： + +![](../images/serverless_2.png) + +```python +# -*- coding: utf8 -*- +import requests + +def main_handler(event, context): + headers=event["headers"] + ip = headers["ip"] + header_new={ + "Host":headers["hosts"], + "User-Agent":headers["user-agent"], + "Accept-Encoding": "gzip, deflate", + "Accept-Language": "zh-CN,zh;q=0.9,ko;q=0.8", + "Connection":"close" + } + try: + r = requests.get(ip,headers=header_new,timeout=10,verify=False) + if r.status_code == 200: + r.encoding = "gbk2312" + return r.text + except Exception as err: + print(err) + + return False +``` + +在高级配置中，执行超时时间设置为10秒，如果超时时间默认时较小，可能导致返回失败请求结果。 + +![](../images/serverless_3.png) + +创建触发器，具体配置如下，注意关闭集成响应。 + +![](../images/serverless_4.jpg) + +编辑API配置的路径为/，然后点击立即完成 + +![](../images/serverless_5.png) + +配置成功后获取到API网关域名如图： + +![](../images/serverless_6.png) + +两个任选其一即可，复制出来并进行初始化操作。 + +**命令：** + +``` +kunyu init --serverless "API网关地址" +``` + +然后正常进行HOSTS爆破功能即可。 + +![](../images/serverless_7.png) + +**示例：** + +![](../images/serverless_8.png) + +**态势感知效果：** + +![](../images/serverless.png) + diff --git a/doc/Serverless_EN.md b/doc/Serverless_EN.md new file mode 100644 index 0000000..1c70373 --- /dev/null +++ b/doc/Serverless_EN.md @@ -0,0 +1,72 @@ +# Kunyu Serverless HOSTS collision configuration + +First enter the configuration cloud function interface, select custom creation, the execution environment is Python 3.6, the region is fine, of course, it is best to choose the domestic location for the goal of China, and the function name is arbitrary + +![](../images/serverless_1.png) + +Fill in the function code, the specific code is as follows: + +![](../images/serverless_2.png) + +```python +# -*- coding: utf8 -*- +import requests + +def main_handler(event, context): + headers=event["headers"] + ip = headers["ip"] + header_new={ + "Host":headers["hosts"], + "User-Agent":headers["user-agent"], + "Accept-Encoding": "gzip, deflate", + "Accept-Language": "zh-CN,zh;q=0.9,ko;q=0.8", + "Connection":"close" + } + try: + r = requests.get(ip,headers=header_new,timeout=10,verify=False) + if r.status_code == 200: + r.encoding = "gbk2312" + return r.text + except Exception as err: + print(err) + + return False +``` + +In the advanced configuration, the execution timeout time is set to 10 seconds. If the timeout time is small by default, it may cause the failed request result to be returned. + +![](../images/serverless_3.png) + +Create a trigger, the specific configuration is as follows, pay attention to close the integrated response. + +![](../images/serverless_4.jpg) + +Edit the path of the API configuration to /, and then click Finish now + +![](../images/serverless_5.png) + +After the configuration is successful, the domain name of the API gateway is obtained as shown in the figure: + +![](../images/serverless_6.png) + +![](../images/serverless_6.png) + +You can choose one of the two, copy it out and initialize it. + +**Order:** + +``` +kunyu init --serverless "API gateway address" +``` + +Then perform the HOSTS blasting function normally. + +![](../images/serverless_7.png) + +**Example:** + +![](../images/serverless_8.png) + +**Situational Awareness Effect:** + +![](../images/serverless.png) \ No newline at end of file diff --git a/doc/Serverless_cloud_Code.py b/doc/Serverless_cloud_Code.py new file mode 100644 index 0000000..44850c4 --- /dev/null +++ b/doc/Serverless_cloud_Code.py @@ -0,0 +1,22 @@ +# -*- coding: utf8 -*- +import requests + +def main_handler(event, context): + headers=event["headers"] + ip = headers["ip"] + header_new={ + "Host":headers["hosts"], + "User-Agent":headers["user-agent"], + "Accept-Encoding": "gzip, deflate", + "Accept-Language": "zh-CN,zh;q=0.9,ko;q=0.8", + "Connection":"close" + } + try: + r = requests.get(ip,headers=header_new,timeout=10,verify=False) + if r.status_code == 200: + r.encoding = "gbk2312" + return r.text + except Exception as err: + print(err) + + return False \ No newline at end of file diff --git a/images/createmap.png b/images/createmap.png new file mode 100644 index 0000000..13f8c8e Binary files /dev/null and b/images/createmap.png differ diff --git a/images/infos.png b/images/infos.png index 763042e..ca3a6b7 100644 Binary files a/images/infos.png and b/images/infos.png differ diff --git a/images/map.png b/images/map.png new file mode 100644 index 0000000..ee723c5 Binary files /dev/null and b/images/map.png differ diff --git a/images/serverless.png b/images/serverless.png new file mode 100644 index 0000000..7533b6a Binary files /dev/null and b/images/serverless.png differ diff --git a/images/serverless_1.png b/images/serverless_1.png new file mode 100644 index 0000000..cd28ebb Binary files /dev/null and b/images/serverless_1.png differ diff --git a/images/serverless_2.png b/images/serverless_2.png new file mode 100644 index 0000000..fd4a239 Binary files /dev/null and b/images/serverless_2.png differ diff --git a/images/serverless_3.png b/images/serverless_3.png new file mode 100644 index 0000000..a479908 Binary files /dev/null and b/images/serverless_3.png differ diff --git a/images/serverless_4.jpg b/images/serverless_4.jpg new file mode 100644 index 0000000..8ca6ba0 Binary files /dev/null and b/images/serverless_4.jpg differ diff --git a/images/serverless_5.png b/images/serverless_5.png new file mode 100644 index 0000000..eab7b48 Binary files /dev/null and b/images/serverless_5.png differ diff --git a/images/serverless_6.png b/images/serverless_6.png new file mode 100644 index 0000000..38f9d0b Binary files /dev/null and b/images/serverless_6.png differ diff --git a/images/serverless_7.png b/images/serverless_7.png new file mode 100644 index 0000000..2fbaf5f Binary files /dev/null and b/images/serverless_7.png differ diff --git a/images/serverless_8.png b/images/serverless_8.png new file mode 100644 index 0000000..a5f765d Binary files /dev/null and b/images/serverless_8.png differ diff --git a/images/setinfo.png b/images/setinfo.png index 3c6f85c..756fb34 100644 Binary files a/images/setinfo.png and b/images/setinfo.png differ diff --git a/kunyu/config/__version__.py b/kunyu/config/__version__.py index 0859bc4..8dda16b 100644 --- a/kunyu/config/__version__.py +++ b/kunyu/config/__version__.py @@ -15,7 +15,7 @@ __python_version__ = sys.version.split()[0] __platform__ = platform.platform() __url__ = "https://github.com/knownsec/Kunyu" -__version__ = '1.6.1' +__version__ = '1.6.2' __author__ = '风起' __Team__ = 'KnownSec 404 Team' __author_email__ = 'onlyzaliks@gmail.com' @@ -35,7 +35,6 @@ kunyu is Cyberspace Search Engine auxiliary tools {{datil}} - """.format(version=__version__, url=__url__) __help__ = """ @@ -65,4 +64,5 @@ kunyu init --username "404@knownsec.com" --password "P@ssword" kunyu init --seebug "012345200157abcdef981bcc89a1452c34d62b8c" kunyu init --apikey "01234567-acbd-0000" --seebug "a73503200157" (recommend) + kunyu init --serverless "https://service-xxxxx-xxxxxxx.sh.apigw.tencentcs.com:443" """ diff --git a/kunyu/config/setting.py b/kunyu/config/setting.py index 8af5e8b..60eace5 100644 --- a/kunyu/config/setting.py +++ b/kunyu/config/setting.py @@ -32,9 +32,9 @@ # Set executable system commands OS_SYSTEM = [] if PLATFORM == "Windows": - OS_SYSTEM = ["ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver"] + OS_SYSTEM = ["ipconfig", "dir", "whoami", "ping", "telnet", "cd", "findstr", "chdir","find", "mysql", "type", "curl", "netstat", "tasklist", "taskkill", "tracert", "del", "ver", "nmap"] else: - OS_SYSTEM = ["ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname"] + OS_SYSTEM = ["ifconfig", "ls", "cat", "pwd", "whoami", "ping", "find", "grep", "telnet", "mysql", "cd", "vi", "more", "less", "curl", "ps", "netstat", "rm", "touch", "mkdir", "uname", "nmap"] # Kunyu OUTPUT File Path OUTPUT_PATH = os.path.expanduser('~/kunyu/output/') @@ -52,10 +52,6 @@ "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50", "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; rv:11.0) like Gecko", - "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", - "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)", - "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)", - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11", @@ -71,9 +67,6 @@ "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0", "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124", "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)", - "UCWEB7.0.2.37/28/999", - "NOKIA5700/ UCWEB7.0.2.37/28/999", - "Openwave/ UCWEB7.0.2.37/28/999", "Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999", "Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.100 Safari/537.36", diff --git a/kunyu/core/__init__.py b/kunyu/core/__init__.py index 4d45e6e..204d825 100644 --- a/kunyu/core/__init__.py +++ b/kunyu/core/__init__.py @@ -41,6 +41,7 @@ parser_init_console.add_argument("--password", help='ZoomEye Password') parser_init_console.add_argument("--seebug", help='ZoomEye Password') parser_init_console.add_argument("--output", help='Set Output File Path') +parser_init_console.add_argument("--serverless", help='Set Serverless API') args = parser.parse_args() @@ -73,6 +74,11 @@ def initial_config(): conf.add_section("path") conf.set("path", "output", setting.OUTPUT_PATH) + # Set Serverless API Address Config + if not conf.has_section("Serverapi"): + conf.add_section("Serverapi") + conf.set("Serverapi", "serverless", "None") + # Verify the login status of the ZoomEye account def _get_login(): param = '{{"username": "{}", "password": "{}"}}'.format(args.username, args.password) @@ -106,6 +112,10 @@ def _get_login(): if args.output: conf.set("path", "output", args.output) + # Used for CrashHost function + if args.serverless: + conf.set("Serverapi", "serverless", args.serverless) + except requests.HTTPError as err: print("\033[31;1m{}\033[0m".format(err)) print(__help__.format(datil=init)) diff --git a/kunyu/core/console.py b/kunyu/core/console.py index 5e4dacc..d41fbae 100644 --- a/kunyu/core/console.py +++ b/kunyu/core/console.py @@ -17,7 +17,6 @@ from rich.console import Console from kunyu.config import setting -from rich.console import Console from kunyu.utils.log import logger from kunyu.lib.export import createdir from kunyu.core.zoomeye import ZoomEye @@ -32,6 +31,7 @@ cmd = "cls" if PLATFORM == "Windows" else "clear" console = Console(color_system="auto", record=True) + def readline_available(): """ Check if the readline is available. By default @@ -206,7 +206,7 @@ def start(self): sys.exit(0) except Exception as err: - # console.print(err) + console.print(err) continue diff --git a/kunyu/core/crash.py b/kunyu/core/crash.py index fa1e312..354273b 100644 --- a/kunyu/core/crash.py +++ b/kunyu/core/crash.py @@ -24,6 +24,7 @@ console = Console(color_system="auto", record=True) ZOOMEYE_KEY = conf.get("zoomeye", "apikey") ZOOMEYE_TOKEN = conf.get("login", "token") +SERVERLESS_API = conf.get("Serverapi", "serverless") class HostScan: @@ -130,7 +131,7 @@ def _get_ip_file(self, ip): """ ip_list = [] if self.__is_valid_ip(ip): - ip_list.append(ip) + ip_list.append(ip) else: for ip_address in get_file(ip): ip_list.append(ip_address) @@ -142,8 +143,10 @@ def host_scan(self, search, ip): :param search: Ways to obtain domain names :param ip: IP address to be collided """ - resp = [] - crash_list = [] + resp, crash_list = [], [] + tmp_list = [] + status = False + if SERVERLESS_API != "None": status = True # http and https protocol collision protocol = ['http://{}/', 'https://{}/'] self.params["q"] = search @@ -152,14 +155,18 @@ def host_scan(self, search, ip): for server in protocol: for ip_address in url: urls = server.format(ip_address) + resp_url = SERVERLESS_API if status else urls for domain in domain_list: - headers = {'Host': domain.strip(), - 'User-Agent': random.choice(UA), - 'ip': urls, - } + headers = { + 'User-Agent': random.choice(UA), + 'ip': urls, + } + # The cloud function sets header hosts + if status:headers["hosts"] = domain.strip() + else:headers["host"] = domain.strip() # Concurrent requests through the encapsulated coroutine module resp.append(grequests.get( - urls, headers=headers, timeout=2, verify=False) + resp_url, headers=headers, timeout=5, verify=False) ) res_list = grequests.map(resp) @@ -167,13 +174,19 @@ def host_scan(self, search, ip): try: # Determine the HTTP status code that needs to be obtained if res.status_code == 200 or res.status_code == 302 or res.status_code == 301: - res.encoding = 'gbk2312' - # Get the title of the returned result - title = re.findall('(.+)', res.text) - crash_list.append( - [res.request.headers['ip'], res.request.headers['Host'], title[0]] - ) + if res.text != "false": + res.encoding = 'gbk2312' + # Get the title of the returned result + title = re.findall('(.+)', res.text) + tmp_list = [res.request.headers['ip'], title[0]] + if status: + tmp_list.append(res.request.headers['hosts']) + # Unicode decoding of the information returned by cloud functions + tmp_list[1] = title[0].encode('utf8').decode('unicode_escape') + else:tmp_list.append(res.request.headers['host']) + crash_list.append(tmp_list) except Exception: continue return crash_list + diff --git a/kunyu/core/createmap.py b/kunyu/core/createmap.py new file mode 100644 index 0000000..e507c0f --- /dev/null +++ b/kunyu/core/createmap.py @@ -0,0 +1,86 @@ +#!/usr/bin/env python +# encoding: utf-8 +""" +@author: 风起 +@contact: onlyzaliks@gmail.com +@File: createmap.py +@Time: 2021/12/6 8:55 +""" + + +def create_data_map(data, data_map_name): + html = str(data_map_name) + body = '{width: 100%;height: 100%;overflow: hidden;margin:0;font-family:"微软雅黑";}' + f = open(html, 'w', encoding='utf-8') + message = """ + + + + + + + + + Kunyu（坤舆）资产分布图 + + +
+ + + + + + """ % (body, data) + f.write(message) + f.close() diff --git a/kunyu/core/keyword.py b/kunyu/core/keyword.py index 290353a..d1f42d7 100644 --- a/kunyu/core/keyword.py +++ b/kunyu/core/keyword.py @@ -81,4 +81,4 @@ def get_keyword_sensitive(self, banner): self.sensitive_params.append(ip) if self.sensitive_params: - return self.sensitive_params \ No newline at end of file + return self.sensitive_params diff --git a/kunyu/core/zoomeye.py b/kunyu/core/zoomeye.py index 0013cb0..12b18c0 100644 --- a/kunyu/core/zoomeye.py +++ b/kunyu/core/zoomeye.py @@ -11,6 +11,7 @@ import sys import json import random +import datetime import requests import platform @@ -27,7 +28,9 @@ from pocsuite3.lib.core.option import init_options import kunyu.lib.encode as encode -from kunyu.config.setting import UA, USER_INFO_API, HOST_SEARCH_API, WEB_SEARCH_API, DOMAIN_SEARCH_API, HOST_SCAN_INFO, SEMSITIVE_INFO +from kunyu.config.setting import UA, USER_INFO_API, HOST_SEARCH_API, WEB_SEARCH_API, DOMAIN_SEARCH_API, HOST_SCAN_INFO, \ + SEMSITIVE_INFO +from kunyu.core.createmap import create_data_map from kunyu.lib.export import export_xls from kunyu.lib.batchfile import get_file from kunyu.core.crash import HostScan @@ -45,6 +48,7 @@ params = {} + class ZoomeyeSearch(object): def __init__(self, method): self.auth = None @@ -158,27 +162,25 @@ def _dork_search(self, url, search, page, sub_type): def _user_info(self): return USER_INFO_API + # Set Global variate class GlobalVar: timeout_resp = 30 + def set_timeout_resp(timeout_resp): GlobalVar.timeout_resp = timeout_resp def get_timeout_resp(*args): return GlobalVar.timeout_resp + # The Display class of the tool class ZoomEye: from kunyu.config.setting import ZOOMEYE_FIELDS_HOST, ZOOMEYE_FIELDS_WEB, ZOOMEYE_FIELDS_INFO, ZOOMEYE_FIELDS_DOMAIN from kunyu.utils.convert import convert - raw_data_params = {} - ssl_data_params = {} - sensitive_params = [] - page = 1 - dtype = 0 - stype = "v4" - btype = "host" - timeout = 30 + ssl_data_params, raw_data_params, sensitive_params, scatter_params = {}, {}, [], [] + page, dtype, timeout = 1, 0, 30 + stype, btype = "v4", "host" # Global commands List help = """Global commands: @@ -197,28 +199,33 @@ class ZoomEye: SearchKeyWord Query sensitive information by keyword Pocsuite3 Invoke the pocsuite component ExportPath Returns the path of the output file + CreateMap Generate an IP distribution heat map clear Clear the console screen show Show can set options help Print Help info exit Exit KunYu & """ # ZoomEye Command List - Command_Info = ["help", "info", "set", "Seebug", "SearchWeb", "SearchHost", "SearchIcon", "HostCrash", "SearchBatch", - "SearchCert", "SearchDomain", "EncodeHash", "Pocsuite3", "ExportPath", "show", "clear", "view", "views", - "SearchKeyWord", "exit"] + Command_Info = ["help", "info", "set", "Seebug", "SearchWeb", "SearchHost", "SearchIcon", "HostCrash", + "SearchBatch", "SearchCert", "SearchDomain", "EncodeHash", "Pocsuite3", "ExportPath", + "show", "clear", "view", "DirectoryCrash", "views", "SearchKeyWord", "CreateMap", "exit"] def __init__(self): self.fields_tables = None + def __params_clear(self): + self.raw_data_params.clear() + self.ssl_data_params.clear() + self.sensitive_params.clear() + self.scatter_params.clear() + def __command_search(self, search, types="host"): """"The raw data obtained is processed and finally displayed on the terminal, which is also one of the most core codes :param search: zoomeye grammar search statement :param types: Dynamically set according to the interface used """ - self.raw_data_params.clear() - self.ssl_data_params.clear() - self.sensitive_params.clear() + self.__params_clear(self) GlobalVar.set_timeout_resp(self.timeout) table = Table(show_header=True, style="bold") global total, api_url, result, FIELDS, export_host @@ -259,11 +266,12 @@ def __command_search(self, search, types="host"): data_isp = data.geoinfo.isp except: pass - """ + + # Set the Latitude and longitude information if data.geoinfo.location: lat = data.geoinfo.location.lat lon = data.geoinfo.location.lon - """ + # Set the output field table.add_row(str(num), data.ip, str(data.portinfo.port), str(data.portinfo.service), str(data.portinfo.app), str(data_isp), str(data.geoinfo.country.names.en), @@ -274,6 +282,9 @@ def __command_search(self, search, types="host"): str(data.portinfo.app), str(data_isp), str(data.geoinfo.country.names.en), str(data.geoinfo.city.names.en), str(title), str(data.timestamp).split("T")[0]] + # Set scatter_params info + self.scatter_params.append({"lng": str(lon), "lat": str(lat), "ip": data.ip}) + # Reset the element self.raw_data_params[num] = data.portinfo.banner @@ -366,8 +377,8 @@ def command_info(cls, *args): ) info.user_info.expired_at = None if info.user_info.expired_at == "" else info.user_info.expired_at console.log("User Information:", style="green") - table.add_row(str(info.user_info.name),str(info.user_info.role), - str(search_quota),str(info.user_info.expired_at)) + table.add_row(str(info.user_info.name), str(info.user_info.role), + str(search_quota), str(info.user_info.expired_at)) console.print(table) logger.info("User information retrieval is completed\n") @@ -451,10 +462,10 @@ def command_encodehash(cls, args): )) except ArithmeticError: - logger.warning("Please specify the encryption mode") + return logger.warning("Please specify the encryption mode") except Exception as err: - logger.warning(err) + return logger.warning(err) @classmethod # Get SeeBug vulnerability information @@ -507,10 +518,13 @@ def command_view(cls, serial): else: console.log("Banner Information is:\n", style="green") console.print(raw_data) - print() + except ArithmeticError: - logger.warning("No retrieval operation is performed or the length of the dictionary key value is exceeded") - return + return logger.warning( + "No retrieval operation is performed or the length of the dictionary key value is exceeded") + + finally: + print() @classmethod # look over ssl row_data info @@ -530,11 +544,11 @@ def command_views(cls, serial): else: console.log("SSL Banner Information is:\n", style="green") console.print(ssl_raw_data) - print() except ArithmeticError: - logger.warning("SSL Banner information is empty") - return - pass + return logger.warning("SSL Banner information is empty") + + finally: + print() @classmethod def command_searchkeyword(cls, *args, **kwargs): @@ -562,6 +576,22 @@ def command_searchkeyword(cls, *args, **kwargs): except Exception: return + @classmethod + def command_createmap(cls, *args, **kwargs): + """ + Create a network space asset distribution map + You can use the CreateMap command to generate a resource distribution map after a search + """ + # Check whether Scatter Params is empty + if len(cls.scatter_params) == 0: return logger.warning("Asset List is Empty,Please search and try again") + from kunyu.config import setting + # Generating an export path + __filename = datetime.datetime.now().strftime("ScatterGram_%H%M%S.html") + path = os.path.join(setting.OUTPUT_PATH, __filename) + create_data_map(cls.scatter_params, path) + # Output Export path + logger.info(path) + @classmethod def command_hostcrash(cls, args): """ @@ -586,7 +616,7 @@ def command_hostcrash(cls, args): # Set output list content for res in result_list: table.add_row( - str(res[0]), str(res[1]), str(res[2]) + str(res[0]), str(res[2]),str(res[1]) ) # Output table list to console console.print(table) @@ -595,7 +625,7 @@ def command_hostcrash(cls, args): export_xls(result_list, HOST_SCAN_INFO) # End of function execution except ArithmeticError: - logger.warning("Please Host IP and Domain\n") + return logger.warning("Please Host IP and Domain") except KeyboardInterrupt: return diff --git a/kunyu/lib/encode.py b/kunyu/lib/encode.py index 50ff034..36c3e08 100644 --- a/kunyu/lib/encode.py +++ b/kunyu/lib/encode.py @@ -17,8 +17,8 @@ import ssl import mmh3 import requests - from tld import get_fld + from kunyu.utils.log import logger from kunyu.config.setting import IP_ADDRESS_REGEX, HTTP_CHECK_REGEX diff --git a/kunyu/lib/export.py b/kunyu/lib/export.py index c0d9cb4..cd6d70b 100644 --- a/kunyu/lib/export.py +++ b/kunyu/lib/export.py @@ -27,6 +27,7 @@ OUTPUT_PATH = conf.get("path", "output") + def createdir(): # Create the results output directory. __dirnamae = datetime.datetime.now().strftime("%Y%m%d%H%M") diff --git a/requirements.txt b/requirements.txt index bf0287e..56894bc 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,10 +1,10 @@ -mmh3~=3.0.0 -requests~=2.26.0 -tld~=0.12.6 -xlwt~=1.3.0 -grequests~=0.6.0 -rich~=10.7.0 -colorama~=0.4.4 -pocsuite3~=1.7.8 -colorlog >= 4.7.2 -setuptools~=57.4.0 \ No newline at end of file +mmh3 +requests +tld +xlwt +grequests +rich +colorama +pocsuite3 +colorlog +setuptools \ No newline at end of file