Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerize Every Analyzer #607

Closed
wants to merge 32 commits into from
Closed
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
7169e3c
Adding Dockerfiles to analyzers
milesflo Feb 8, 2020
15bcf08
Multithreaded build step and refactored w/ library
milesflo Feb 8, 2020
6c0d9c1
Added alpine dependency detection in builder script
milesflo Feb 9, 2020
36666b4
Resolved yara-python related build errors
milesflo Feb 9, 2020
da53fee
Fixed dependency detection for eml_parser pip
milesflo Feb 9, 2020
c170c71
Removing redundant dependencies from eml_parser
milesflo Feb 9, 2020
8364f34
Restructuring alpine dependency step
milesflo Feb 9, 2020
47d4e7f
Reverting FileInfo Dockerfile to manual control state
milesflo Feb 10, 2020
dfa3c4f
Merged upstream into master
milesflo Feb 10, 2020
ac51197
[WIP] Updated dockerfiles. Pending tests
milesflo Feb 10, 2020
e4cc256
Updating new configs to semver
milesflo Feb 10, 2020
61f3c2e
Reverting MalwareCluster to manual control
milesflo Feb 10, 2020
eb2f146
tmp
milesflo Feb 10, 2020
d92ec4a
Correctly stylizing PN AutoFocus
milesflo Feb 10, 2020
e8cd23b
Joining label operations to a single statement
milesflo Feb 10, 2020
4aab251
Adding vendor, version, and authors key to dockerfiles
milesflo Feb 10, 2020
c555258
Excluding dockerfile description from Analyzers w/ many services
milesflo Feb 10, 2020
942ed58
Adding source to image labels
milesflo Feb 10, 2020
8f6466a
Added authors label string sorting to prevent FP update operation
milesflo Feb 10, 2020
039ec17
Fixed error where new Dockerfile would be empty
milesflo Feb 10, 2020
b7b4222
Minor script naming convention improvement
milesflo Feb 10, 2020
e4de803
Refactored scripts to use relative filepaths _correctly_
milesflo Feb 10, 2020
a844c0d
Fixed Dockerfile entrypoint where configs were incorrect
milesflo Feb 11, 2020
6ee32ed
Fixing dependency err for DomainTools and DomainToolsIris
milesflo Feb 11, 2020
b64ca1b
DNSDB Python3 version up
milesflo Feb 11, 2020
f00eb02
Including script to verify dockerfile will build&exec
milesflo Feb 11, 2020
b6b1fad
Resolving Permission denied bug w/ sweeping exec chmod
milesflo Feb 11, 2020
41069e7
Updated Hippocampe to use requests over urllib
milesflo Feb 11, 2020
c02a2af
Typo
milesflo Feb 11, 2020
d100a51
Final passthrough resolving dep declarations & execution paths
milesflo Feb 11, 2020
f9e0e53
Fixed abuse_finder. Rediculous.
milesflo Feb 11, 2020
5bcd4dc
Merge branch 'master' into master
milesflo Feb 11, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion analyzers/AbuseIPDB/AbuseIPDB.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "AbuseIPDB",
"version": "1.0",
"version": "1.0.0",
"author": "Matteo Lodi",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-v3",
Expand Down
24 changes: 24 additions & 0 deletions analyzers/AbuseIPDB/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="Matteo Lodi" \
description="Determine whether an IP was reported or not as malicious by AbuseIPDB" \
license="AGPL-v3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="AbuseIPDB" \
url="https://github.com/TheHive-Project/Cortex-Analyzers" \
vendor="TheHive" \
version="1.0.0"

WORKDIR /worker

COPY . AbuseIPDB

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r AbuseIPDB/requirements.txt

ENTRYPOINT AbuseIPDB/abuseipdb.py
5 changes: 3 additions & 2 deletions analyzers/Abuse_Finder/Abuse_Finder.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
{
"name": "Abuse_Finder",
"version": "3.0",
"version": "3.0.0",
"author": "CERT-BDF",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
"description": "Find abuse contacts associated with domain names, URLs, IPs and email addresses.",
"dataTypeList": ["ip", "domain", "fqdn", "url", "mail"],
"baseConfig": "Abuse_Finder",
"command": "Abuse_Finder/abusefinder.py"
"command": "Abuse_Finder/abusefinder.py",
"dockerImage": "cortexneurons/abuseipdb:1"
}
22 changes: 20 additions & 2 deletions analyzers/Abuse_Finder/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
FROM python:3
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="CERT-BDF" \
description="Find abuse contacts associated with domain names, URLs, IPs and email addresses." \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="Abuse_Finder" \
url="https://github.com/TheHive-Project/Cortex-Analyzers" \
vendor="TheHive" \
version="3.0.0"

WORKDIR /worker

COPY . Abuse_Finder

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r Abuse_Finder/requirements.txt
ENTRYPOINT Abuse_Finder/abusefinder.py

ENTRYPOINT Abuse_Finder/abusefinder.py
2 changes: 2 additions & 0 deletions analyzers/Abuse_Finder/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
cortexutils
future
abuse_finder
pythonwhois==2.2.2
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{
"name": "AUTOFOCUS_GetSampleAnalysis",
"version": "1.0",
"name": "AutoFocus_GetSampleAnalysis",
"version": "1.0.0",
"author": "ANSSI",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
"baseConfig": "Autofocus",
"baseConfig": "AutoFocus",
"config": {
"service": "get_sample_analysis"
},
"description": "Get full analysis from a sample based on its hash",
"dataTypeList": ["hash"],
"command": "Autofocus/analyzer.py",
"command": "AutoFocus/analyzer.py",
"configurationItems": [
{
"name": "apikey",
"description": "Autofocus API key",
"description": "AutoFocus API key",
"type": "string",
"multi": false,
"required": true
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{
"name": "AUTOFOCUS_SearchIOC",
"version": "1.0",
"name": "AutoFocus_SearchIOC",
"version": "1.0.0",
"author": "ANSSI",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
"baseConfig": "Autofocus",
"baseConfig": "AutoFocus",
"config": {
"service": "search_ioc"
},
"description": "Search samples in Autofocus based on a single IOC",
"description": "Search samples in AutoFocus based on a single IOC",
"dataTypeList": ["domain","fqdn","user-agent","imphash","ip","mutex","tag","url"],
"command": "Autofocus/analyzer.py",
"command": "AutoFocus/analyzer.py",
"configurationItems": [
{
"name": "apikey",
"description": "Autofocus API key",
"description": "AutoFocus API key",
"type": "string",
"multi": false,
"required": true
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{
"name": "AUTOFOCUS_SearchJSON",
"version": "1.0",
"name": "AutoFocus_SearchJSON",
"version": "1.0.0",
"author": "ANSSI",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
"baseConfig": "Autofocus",
"baseConfig": "AutoFocus",
"config": {
"service": "search_json"
},
"description": "Search samples in Autofocus with a full search query in JSON",
"description": "Search samples in AutoFocus with a full search query in JSON",
"dataTypeList": ["other"],
"command": "Autofocus/analyzer.py",
"command": "AutoFocus/analyzer.py",
"configurationItems": [
{
"name": "apikey",
"description": "Autofocus API key",
"description": "AutoFocus API key",
"type": "string",
"multi": false,
"required": true
Expand Down
25 changes: 25 additions & 0 deletions analyzers/AutoFocus/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

RUN apk add --no-cache gcc git musl-dev

LABEL author="ANSSI" \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="AutoFocus" \
url="https://github.com/TheHive-Project/Cortex-Analyzers" \
vendor="TheHive" \
version="1.0.0"

WORKDIR /worker

COPY . AutoFocus

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r AutoFocus/requirements.txt

ENTRYPOINT AutoFocus/analyzer.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,14 @@ def run(self):
records = self.execute_autofocus_service()
self.report(records)

except AFSampleAbsent as e: # Sample not in Autofocus
self.error('Unknown sample in Autofocus')
except AFSampleAbsent as e: # Sample not in AutoFocus
self.error('Unknown sample in AutoFocus')
except AFServerError as e: # Server error
self.unexpectedError(e)
except AFClientError as e: # Client error
self.unexpectedError(e)
except Exception: # Unknown error
self.unexpectedError("Unknown error while running Autofocus analyzer")
self.unexpectedError("Unknown error while running AutoFocus analyzer")

if __name__ == '__main__':
AutoFocusAnalyzer().run()
2 changes: 1 addition & 1 deletion analyzers/BackscatterIO/BackscatterIO_Enrichment.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "BackscatterIO_Enrichment",
"version": "1.0",
"version": "1.0.0",
"author": "brandon@backscatter.io",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "APLv2",
Expand Down
2 changes: 1 addition & 1 deletion analyzers/BackscatterIO/BackscatterIO_GetObservations.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "BackscatterIO_GetObservations",
"version": "1.0",
"version": "1.0.0",
"author": "brandon@backscatter.io",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "APLv2",
Expand Down
23 changes: 23 additions & 0 deletions analyzers/BackscatterIO/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="brandon@backscatter.io" \
license="APLv2" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="BackscatterIO" \
url="https://github.com/TheHive-Project/Cortex-Analyzers" \
vendor="TheHive" \
version="1.0.0"

WORKDIR /worker

COPY . BackscatterIO

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r BackscatterIO/requirements.txt

ENTRYPOINT BackscatterIO/backscatter-io.py
1 change: 1 addition & 0 deletions analyzers/C1fApp/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
input
2 changes: 1 addition & 1 deletion analyzers/C1fApp/C1fApp_osint.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "C1fApp",
"version": "1.0",
"version": "1.0.0",
"author": "etz69",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
Expand Down
24 changes: 24 additions & 0 deletions analyzers/C1fApp/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="etz69" \
description="Query C1fApp OSINT Aggregator for IPs, domains and URLs" \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="C1fApp" \
url="https://github.com/TheHive-Project/Cortex-Analyzers" \
vendor="TheHive" \
version="1.0.0"

WORKDIR /worker

COPY . C1fApp

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r C1fApp/requirements.txt

ENTRYPOINT C1fApp/cifquery.py
2 changes: 1 addition & 1 deletion analyzers/C1fApp/README
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ C1fApp {

To test the analyzer from cmdline

python cifquery.py < input
python3 cifquery.py < input

Testing
--------
Expand Down
2 changes: 1 addition & 1 deletion analyzers/CERTatPassiveDNS/CERTatPassiveDNS.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"author": "Nils Kuhnert, CERT-Bund",
"license": "AGPL-V3",
"url": "https://github.com/BSI-CERT-Bund/cortex-analyzers",
"version": "2.0",
"version": "2.0.0",
"description": "Checks CERT.at Passive DNS for a given domain.",
"dataTypeList": ["domain", "fqdn", "ip"],
"baseConfig": "CERTatPassiveDNS",
Expand Down
24 changes: 24 additions & 0 deletions analyzers/CERTatPassiveDNS/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="Nils Kuhnert, CERT-Bund" \
description="Checks CERT.at Passive DNS for a given domain." \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="CERTatPassiveDNS" \
url="https://github.com/BSI-CERT-Bund/cortex-analyzers" \
vendor="TheHive" \
version="2.0.0"

WORKDIR /worker

COPY . CERTatPassiveDNS

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r CERTatPassiveDNS/requirements.txt

ENTRYPOINT CERTatPassiveDNS/certat_passivedns.py
2 changes: 1 addition & 1 deletion analyzers/CIRCLPassiveDNS/CIRCLPassiveDNS.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"author": "Nils Kuhnert, CERT-Bund",
"license": "AGPL-V3",
"url": "https://github.com/BSI-CERT-Bund/cortex-analyzers",
"version": "2.0",
"version": "2.0.0",
"description": "Check CIRCL's Passive DNS for a given domain or URL.",
"dataTypeList": ["domain", "url", "ip"],
"baseConfig": "CIRCL",
Expand Down
24 changes: 24 additions & 0 deletions analyzers/CIRCLPassiveDNS/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="Nils Kuhnert, CERT-Bund" \
description="Check CIRCL's Passive DNS for a given domain or URL." \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="CIRCLPassiveDNS" \
url="https://github.com/BSI-CERT-Bund/cortex-analyzers" \
vendor="TheHive" \
version="2.0.0"

WORKDIR /worker

COPY . CIRCLPassiveDNS

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r CIRCLPassiveDNS/requirements.txt

ENTRYPOINT CIRCLPassiveDNS/circl_passivedns.py
2 changes: 1 addition & 1 deletion analyzers/CIRCLPassiveSSL/CIRCLPassiveSSL.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"author": "Nils Kuhnert, CERT-Bund",
"license": "AGPL-V3",
"url": "https://github.com/BSI-CERT-Bund/cortex-analyzers",
"version": "2.0",
"version": "2.0.0",
"description": "Check CIRCL's Passive SSL for a given IP address or a X509 certificate hash.",
"dataTypeList": ["ip", "certificate_hash", "hash"],
"baseConfig": "CIRCL",
Expand Down
24 changes: 24 additions & 0 deletions analyzers/CIRCLPassiveSSL/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### AUTOGEN ###
# THIS FILE IS UPDATED BY `utils/dockerfile_builder.py`
# DO NOT EDIT IT DIRECTLY

# Guessing base image from source code shebang
FROM python:3-alpine

LABEL author="Nils Kuhnert, CERT-Bund" \
description="Check CIRCL's Passive SSL for a given IP address or a X509 certificate hash." \
license="AGPL-V3" \
source="https://github.com/TheHive-Project/Cortex-Analyzers" \
title="CIRCLPassiveSSL" \
url="https://github.com/BSI-CERT-Bund/cortex-analyzers" \
vendor="TheHive" \
version="2.0.0"

WORKDIR /worker

COPY . CIRCLPassiveSSL

# Project determined to be Python, installing deps
RUN pip install --no-cache-dir -r CIRCLPassiveSSL/requirements.txt

ENTRYPOINT CIRCLPassiveSSL/circl_passivessl.py
Loading