Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tool_arsenal): incorrect regex version numbers #1086

Merged
merged 6 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion web/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Download Go packages
RUN go install -v -v github.com/bp0lr/gauplus@latest
RUN go install -v github.com/jaeles-project/gospider@latest
RUN go install -v github.com/tomnomnom/gf@latest
RUN go install -v github.com/tomnomnom/unfurl@latest
Expand Down
47 changes: 15 additions & 32 deletions web/fixtures/external_tools.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
version_lookup_command: naabu -version
update_command: go install -v github.com/projectdiscovery/naabu/v2/cmd/naabu@latest
install_command: go install -v github.com/projectdiscovery/naabu/v2/cmd/naabu@latest
version_match_regex: ' (\d+\.)?(\d+\.)?(\*|\d+)'
version_match_regex: '(\b\d+\.\d+\.\d+\b)'
is_default: true
is_subdomain_gathering: false
is_github_cloned: false
Expand Down Expand Up @@ -171,40 +171,23 @@
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 10
fields:
logo_url: null
name: gauplus
description: Get all URLs
github_url: https://github.com/bp0lr/gauplus
license_url: https://github.com/bp0lr/gauplus/blob/main/LICENSE
version_lookup_command: gauplus -version
update_command: go install -v -v github.com/bp0lr/gauplus@latest
install_command: go install -v -v github.com/bp0lr/gauplus@latest
version_match_regex: '[vV]*(\d+\.)?(\d+\.)?(\*|\d+)'
is_default: true
is_subdomain_gathering: false
is_github_cloned: true
github_clone_path: /usr/src/github/gauplus
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 11
fields:
logo_url: null
name: OneForAll
description: A powerful subdomain integration tool.
github_url: https://github.com/shmilylty/OneForAll
license_url: https://github.com/shmilylty/OneForAll/blob/master/LICENSE
version_lookup_command: python3 /usr/src/github/OneForAll/oneforall.py version
version_lookup_command: cat /usr/src/github/OneForAll/oneforall.py
update_command: git pull
install_command: git clone https://github.com/shmilylty/OneForAll
version_match_regex: 'v(\d+\.)?(\d+\.)?(\*|\d+)'
version_match_regex: v\d+\.\d+\.\d+\b
is_default: true
is_subdomain_gathering: true
is_github_cloned: true
github_clone_path: /usr/src/github/OneForAll
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 12
pk: 11
fields:
logo_url: https://raw.githubusercontent.com/laramies/theHarvester/master/theHarvester-logo.png
name: theHarvester
Expand All @@ -215,17 +198,17 @@
names, subdomains, IPs and URLs using multiple public data sources.
github_url: https://github.com/laramies/theHarvester
license_url: https://github.com/laramies/theHarvester/blob/master/README/LICENSES
version_lookup_command: null
version_lookup_command: cat /usr/src/github/theHarvester/theHarvester/lib/version.py
update_command: git pull
install_command: git clone https://github.com/laramies/theHarvester
version_match_regex: (\d+\.)?(\d+\.)?(\*|\d+)
version_match_regex: \b\d+\.\d+\.\d+\b
is_default: true
is_subdomain_gathering: false
is_github_cloned: true
github_clone_path: /usr/src/github/theHarvester
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 13
pk: 12
fields:
logo_url: null
name: ctfr
Expand All @@ -244,7 +227,7 @@
github_clone_path: /usr/src/github/ctfr
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 14
pk: 13
fields:
logo_url: https://user-images.githubusercontent.com/8293321/174841003-01a62bad-2ecf-4874-89c4-efa53dd56884.png
name: tlsx
Expand All @@ -261,14 +244,14 @@
github_clone_path: null
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 15
pk: 14
fields:
logo_url: https://avatars.githubusercontent.com/u/79084675?v=4
name: netlas
description: Non-intrusive Internet Scanner.
github_url: https://github.com/netlas-io/netlas-python
license_url: https://github.com/netlas-io/netlas-python/blob/master/LICENSE
version_lookup_command: null
version_lookup_command: pip3 show netlas
update_command: pip3 install netlas --update
install_command: pip3 install netlas
version_match_regex: (\d+\.)?(\d+\.)?(\*|\d+)
Expand All @@ -278,7 +261,7 @@
github_clone_path: null
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 16
pk: 15
fields:
logo_url: https://github.githubassets.com/images/icons/emoji/unicode/1f98a.png
name: dalfox
Expand All @@ -296,7 +279,7 @@
github_clone_path: null
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 17
pk: 16
fields:
logo_url: https://user-images.githubusercontent.com/8293321/196779266-421c79d4-643a-4f73-9b54-3da379bbac09.png
name: katana
Expand All @@ -313,7 +296,7 @@
github_clone_path: null
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 18
pk: 17
fields:
logo_url: https://user-images.githubusercontent.com/25837540/90128972-fc3bdf00-dd91-11ea-8c3b-0d6f4e8c6ba3.png
name: crlfuzz
Expand All @@ -330,7 +313,7 @@
github_clone_path: null
subdomain_gathering_command: null
- model: scanEngine.installedexternaltool
pk: 19
pk: 18
fields:
logo_url: null
name: gau
Expand All @@ -345,4 +328,4 @@
is_subdomain_gathering: false
is_github_cloned: false
github_clone_path: null
subdomain_gathering_command: null
subdomain_gathering_command: null
3 changes: 0 additions & 3 deletions web/reNgine/tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -1759,21 +1759,18 @@ def fetch_url(self, urls=[], ctx={}, description=None):
# Tools cmds
cmd_map = {
'gau': f'gau',
'gauplus': f'gauplus -random-agent',
'hakrawler': 'hakrawler -subs -u',
'waybackurls': 'waybackurls',
'gospider': f'gospider -S {input_path} --js -d 2 --sitemap --robots -w -r',
'katana': f'katana -list {input_path} -silent -jc -kf all -d 3 -fs rdn',
}
if proxy:
cmd_map['gau'] += f' --proxy "{proxy}"'
cmd_map['gauplus'] += f' -p "{proxy}"'
cmd_map['gospider'] += f' -p {proxy}'
cmd_map['hakrawler'] += f' -proxy {proxy}'
cmd_map['katana'] += f' -proxy {proxy}'
if threads > 0:
cmd_map['gau'] += f' --threads {threads}'
cmd_map['gauplus'] += f' -t {threads}'
cmd_map['gospider'] += f' -t {threads}'
cmd_map['katana'] += f' -c {threads}'
if custom_header:
Expand Down